summaryrefslogtreecommitdiff
path: root/git_refs.py
AgeCommit message (Collapse)Author
2026-03-17git_refs: read refs via git plumbing for files/reftableGavin Mak
Replace direct `packed-refs` file parsing with `git for-each-ref` plumbing to support both `files` and `reftable` backends. Bug: 476209856 Change-Id: I2ad8ff8f3382426600f15370c997f9bc17165485 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/550401 Commit-Queue: Gavin Mak <gavinmak@google.com> Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Gavin Mak <gavinmak@google.com>
2023-10-21cleanup: convert exceptions to OSErrorJason R. Coombs
In Python 3, these exceptions were merged into OSError, so switch everything over to that. Change-Id: If876a28b692de5aa5c62a3bdc8c000793ce52c63 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/390376 Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com> Commit-Queue: Mike Frysinger <vapier@google.com> Tested-by: Mike Frysinger <vapier@google.com>
2023-10-21cleanup: delete redundant "r" open modeJason R. Coombs
Change-Id: I86ebb8c5a9dc3752e8a25f4b11b64c5be3a6429e Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/390375 Tested-by: Mike Frysinger <vapier@google.com> Commit-Queue: Mike Frysinger <vapier@google.com> Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
2023-10-20delete Python 2 (object) compatMike Frysinger
Bug: 302871152 Change-Id: I39636d73a6e1d69efa8ade74f75c5381651e6dc8 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/390054 Commit-Queue: Mike Frysinger <vapier@google.com> Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com> Tested-by: Mike Frysinger <vapier@google.com>
2023-08-22isort: format codebasev2.36Mike Frysinger
Change-Id: I6f11d123b68fd077f558d3c21349c55c5f251019 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/383715 Reviewed-by: Gavin Mak <gavinmak@google.com> Tested-by: Mike Frysinger <vapier@google.com> Commit-Queue: Mike Frysinger <vapier@google.com>
2023-03-22Format codebase with black and check formatting in CQGavin Mak
Apply rules set by https://gerrit-review.googlesource.com/c/git-repo/+/362954/ across the codebase and fix any lingering errors caught by flake8. Also check black formatting in run_tests (and CQ). Bug: b/267675342 Change-Id: I972d77649dac351150dcfeb1cd1ad0ea2efc1956 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/363474 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Gavin Mak <gavinmak@google.com> Commit-Queue: Gavin Mak <gavinmak@google.com>
2022-11-03Set tracing to always on and save to .repo/TRACE_FILE.Joanna Wang
- add `--trace_to_stderr` option so stderr will include trace outputs and any other errors that get sent to stderr - while TRACE_FILE will only include trace outputs piggy-backing on: https://gerrit-review.googlesource.com/c/git-repo/+/349154 Change-Id: I3895a84de4b2784f17fac4325521cd5e72e645e2 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/350114 Reviewed-by: LaMont Jones <lamontjones@google.com> Tested-by: Joanna Wang <jojwang@google.com>
2021-02-28git_refs: fix crash with binary . files in .git/refs/Mike Frysinger
On macOS, the Finder app likes to poop .DS_Store files in every path that the user browses. If the user pokes around the .git/ tree, it could generate a .DS_Store file in there too. When repo goes to read all the local refs, it tries to decode this binary file as UTF-8 and subsequently crashes. Since paths that begin with . are not valid refs, ignore them like we already do with paths that end in .lock. Also bump the check up to ignore dirs that match since that follows the git rules: they apply to any component in its path, not just the final path (name). We don't implement the full valid ref algorithm that git employs as it's a bit complicated, and we only really need to focus on what will practically show up locally. Bug: https://crbug.com/gerrit/14162 Change-Id: I6519f990e33cc58a72fcb00c0f983ad3285ace3d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298662 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Michael Mortensen <mmortensen@google.com>
2021-01-06strip python2-only coding:utf-8 & print_function settingsMike Frysinger
We're committed to Python 3 at this point, so clean up boilerplate. Bug: https://crbug.com/gerrit/10418 Change-Id: Ib1719ba2eb65c53b94881a1a1bf203ddfcaaafed Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/292382 Reviewed-by: Chris Mcdonald <cjmcdonald@google.com> Tested-by: Mike Frysinger <vapier@google.com>
2020-02-29project: fix m/ pseudo ref handling with git worktreesMike Frysinger
Since most ref namespaces are shared among all worktrees, trying to set the pseudo m/<branch> in the common git repo ends up clobbering each other when using shared checkouts. For example, in CrOS: <project path="src/third_party/kernel/v3.8" name="chromiumos/third_party/kernel" revision="refs/heads/chromeos-3.8" /> <project path="src/third_party/kernel/v3.10" name="chromiumos/third_party/kernel" revision="refs/heads/chromeos-3.10" /> Trying to set m/master in chromiumos/third_party/kernel.git/ will keep clobbering the other. Instead, when using git worktrees, lets set the m/ pseudo ref to point into the refs/worktree/ namespace which is unique to each git worktree. So we have in the common dir: chromiumos/third_party/kernel.git/: refs/remotes/m/master: ref: refs/worktree/m/master And then in each worktree we point refs/worktree/m/master to the respective manifest revision expression. Now people can use the m/master in each git worktree and have it resolve to the right commit for that worktree. Bug: https://crbug.com/gerrit/12404 Change-Id: I78814bdd5dd67bb13218c4c6ccd64f8a15dd0a52 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/256952 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-12Fix various whitespace issues reported by pyflakesDavid Pursehouse
- E201 whitespace after '[' - E202 whitespace before '}' - E221 multiple spaces before operator - E222 multiple spaces after operator - E225 missing whitespace around operator - E226 missing whitespace around arithmetic operator - E231 missing whitespace after ',' - E261 at least two spaces before inline comment - E271 multiple spaces after keyword Fixed automatically with autopep8: git ls-files | grep py$ | xargs autopep8 --in-place \ --select E201,E202,E221,E222,E225,E226,E231,E261,E271 Change-Id: I367113eb8c847eb460532c7c2f8643f33040308c Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254601 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: David Pursehouse <dpursehouse@collab.net>
2019-11-12use open context managers in more placesMike Frysinger
Use open() as a context manager to simplify the close logic and make the code easier to read & understand. This is also more Pythonic. Change-Id: I579d03cca86f99b2c6c6a1f557f6e5704e2515a7 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/244734 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
2019-08-27rename local trace moduleMike Frysinger
There is a standard Python "trace" module, so having a local trace.py prevents us being able to import that. Rename the module to avoid. Change-Id: I23e29ec95a2204bb168a641323d05e76968d9b57 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/234832 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
2019-06-13set default file encoding to utf-8Mike Frysinger
There's no reason to support any other encoding in these files. This only affects the files themselves and not streams they open. Bug: https://crbug.com/gerrit/10418 Change-Id: I053cb40cd3666ce5c8a0689b9dd938f24ca765bf
2018-10-22Add support for long pathsRenaud Paquay
* Add more file i/o wrappers in platform_utils to allow using long paths (length > MAX_PATH) on Windows. * Paths using the long path syntax ("\\?\" prefix) should never escape the platform_utils API surface area, so that this specific syntax is not visible to the rest of the repo code base. * Forward many calls from os.xxx to platform_utils.xxx in various place to ensure long paths support, specifically when repo decides to delete obsolete directories. * There are more places that need to be converted to support long paths, this commit is an initial effort to unblock a few common use cases. * Also, fix remove function to handle directory symlinks Change-Id: If82ccc408e516e96ff7260be25f8fd2fe3f9571a
2017-08-31Handle Windows line endings when reading binary filesRenaud Paquay
Without this change, '.git\HEAD' files, for examples, are sometime read incorrectly resulting in the current branch to be reset to "master" when running a "repo init -b xxx" on an already initialized repository. Change-Id: I48c7ef85ff81626edf156914329a560e14252f2a
2017-08-25When starting a branch, do not use a tag or change value for branch.mergeZac Livingston
When starting a branch, branch.merge is set to project revision unless the revision is a SHA1. In that case, branch.merge is set to dest_branch if defined or manifest default revision otherwise. This special handling allows repo upload to work when the project revision is a SHA1. Extend the special handling to also happen when the project revision is a tag value or a change value so that repo upload will work in those case as well. Change-Id: Iff81ece40e770cd02535e80dcb023564d42dcf47
2013-11-21Fix some python3 encoding issuesChirayu Desai
* Add .decode('utf-8') where needed * Add 'b' to `open` where needed, and remove where unnecessary Change-Id: I0f03ecf9ed1a78e3b2f15f9469deb9aaab698657
2013-04-18Some fixes for supporting python3Chirayu Desai
* Fix imports. * Use python3 syntax. * Wrap map() calls with list(). * Use list() only wherever needed. (Thanks Conley!) * Fix dictionary iteration methods (s/iteritems/items/). * Make use of sorted() in appropriate places * Use iterators directly in the loop. * Don't use .keys() wherever it isn't needed. * Use sys.maxsize instead of sys.maxint TODO: * Make repo work fully with python3. :) Some of this was done by the '2to3' tool [1], by applying the needed fixes in a way that doesn't break compatibility with python2. Links: [1]: http://docs.python.org/2/library/2to3.html Change-Id: Ibdf3bf9a530d716db905733cb9bfef83a48820f7 Signed-off-by: Chirayu Desai <cdesai@cyanogenmod.org>
2012-10-30Even more coding style cleanupDavid Pursehouse
Fixing some more pylint warnings: W1401: Anomalous backslash in string W0623: Redefining name 'name' from outer scope W0702: No exception type(s) specified E0102: name: function already defined line n Change-Id: I5afcdb4771ce210390a79981937806e30900a93c
2012-10-09Coding style cleanupDavid Pursehouse
Fix the following issues reported by pylint: C0321: More than one statement on a single line W0622: Redefining built-in 'name' W0612: Unused variable 'name' W0613: Unused argument 'name' W0102: Dangerous default value 'value' as argument W0105: String statement has no effect Also fixed a few cases of inconsistent indentation. Change-Id: Ie0db839e7c57d576cff12d8c055fe87030d00744
2012-08-23Remove unused importsDavid Pursehouse
There are several imports that are not used. Remove them. Change-Id: I2ac3be66827bd68d3faedcef7d6bbf30ea01d3f2
2011-11-29Improve error handling when reading loose refsv1.7.8Shawn O. Pearce
When repo is trying to figure out branches the repository has by traversing refs/heads, add exception handling for readline. Change-Id: If3b2a3720c6496f52f629aa9a2539f186d6ec882
2009-04-18Make usage of open safer by setting binary mode and closing fdsShawn O. Pearce
Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-04-18Fix `repo --trace` to show ref and config loadsShawn O. Pearce
The value of the varible TRACE was copied during the import, which happens before the --trace option can be processed. So instead we now use a function to determine if the value is set, as the function can be safely copied early during import. Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-04-17Enable tracing of ref scans and config unpicklingShawn O. Pearce
These are not as expensive as spawning a git command, but they are not free either. We want to keep track of how many times we wind up calling them on any particular operation. Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-04-17Avoid unnecessary git symbolic-ref calls during repo syncShawn O. Pearce
If the m/BRANCH ref is already pointing at the value set in the manifest there is no reason to set it again. Leave it alone, thus saving a full fork+exec call. Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-04-17Improve repo sync performance by avoid git forksShawn O. Pearce
By resolving the current HEAD and the manifest revision using pure Python, we can in the common case of "no changes" avoid a lot of git operations and directly jump out of the local sync method. This reduces the no-op `repo sync -l` time for Android's 114 projects from more than 6s to under 0.8s. Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-04-17Implement git ref reading purely in PythonShawn O. Pearce
Its much faster to read the refs from 114 projects when the reader is pure Python and just doing file IO than forking 114 git commands and parsing their output. The reader caches refs based upon file mtimes. If any single ref file has been modified since the last read, we re-read the entire repository's ref namespace. This simplifies the code as we don't need to worry about shooting down symbolic-refs, but it may cause more IO than is necessary if only one ref gets updated. This change drops `repo branches` in Android from 1.658s to 0.206s. Likewise, `repo sync` improves dramatically as well. Signed-off-by: Shawn O. Pearce <sop@google.com>