Skip to content

fix(gitclone): skip ensure-refs fetches that cannot help - #453

Merged
alecthomas merged 3 commits into
mainfrom
aat/skip-pointless-ensure-fetch
Nov 17, 2026
Merged

alecthomas merged 3 commits into
mainfrom
aat/skip-pointless-ensure-fetch

Conversation

@alecthomas

@alecthomas alecthomas commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

EnsureRefs fetched from upstream whenever a request was unsatisfied, and it kept no record that a fetch had already failed to help. Any request the mirror could not satisfy even after fetching therefore cost a full upstream fetch on every call. #452 fixed one instance of this, HEAD. These cases remained:

  • A ref pinned to a SHA the mirror has moved past, because the check was exact equality.
  • An annotated tag pinned to its commit, because the mirror records the tag object's SHA.
  • A ref or commit that upstream does not have, such as a deleted branch, a misspelled name, or a commit removed by a force push.

Separately, a ref requested with an empty SHA was satisfied as soon as it existed locally, so it never refreshed a stale mirror. That is not what callers asking for HEAD expect.

The approach is to fetch only when upstream has something the mirror lacks:

  • A pinned ref is satisfied when its SHA is reachable from the ref.
  • An unsatisfied request compares the mirror's refs with git ls-remote and fetches only if they differ. This bypasses RefCheckInterval, so a commit pushed a moment ago is still found.
  • An entry upstream could not satisfy is remembered for RefCheckInterval, so a caller repeating a hopeless request makes no upstream call.
  • An empty SHA now also requires the ref to be current. For HEAD and branches this uses the existing cached check in EnsureRefsUpToDate, so frequent HEAD requests cost at most one git ls-remote per RefCheckInterval. That check compares only branches, so other refs, such as tags, are compared directly with upstream instead.

Two behaviours change for callers. A request for a ref or commit that upstream lacks now returns fetched=false, with missing commits still listed. An empty-SHA request now fails if upstream cannot be reached, where before it succeeded from the mirror alone.

The upstream comparison is shared with EnsureRefsUpToDate through a new behindUpstream method, which takes a filter for the refs to compare. Tests cover each case above, plus the memory of unsatisfiable entries.

🤖 Generated with Claude Code

EnsureRefs fetched from upstream whenever a request was unsatisfied,
and kept no record that a fetch had already failed to help. A ref
pinned to a SHA the mirror had moved past, a tag pinned to its commit,
or a ref or commit upstream lacks cost a full fetch on every call.

Compare refs with upstream first and fetch only when upstream has
something the mirror lacks. Treat a pinned ref as satisfied when its
SHA is reachable from the ref. Remember entries upstream could not
satisfy for RefCheckInterval, so repeats make no upstream call.

An empty SHA now also requires the mirror to be current, using the
cached staleness check. Before, it never refreshed a stale mirror,
which is not what callers asking for HEAD expect.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@alecthomas
alecthomas marked this pull request as ready for review November 17, 2026 05:04
@alecthomas
alecthomas requested a review from a team as a code owner November 17, 2026 05:04
@alecthomas
alecthomas requested review from inez and removed request for a team November 17, 2026 05:04
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-17T05:43:23.340399Z 051b4ed New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 884742fa77

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/gitclone/manager.go Resolved
Comment thread internal/gitclone/manager.go Resolved
A repeat request for a missing ref with an empty SHA skipped the
upstream comparison, as intended, but then ran the freshness check
because the request contained an empty SHA. That reached upstream again
and could fail if upstream was down, despite the remembered entry.

Run the freshness check only for empty-SHA refs the mirror has. Also
drop descriptive comments from short private helpers, which the
repository's comment rules do not allow.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 43a242fccc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/gitclone/manager.go Resolved
An empty SHA requires the ref to be current, but the freshness check
compared only branches. A tag that moved upstream while no branch
changed was returned stale indefinitely.

HEAD and branches keep the cached branch check, so frequent HEAD
requests still cost at most one ls-remote per RefCheckInterval. Other
empty-SHA refs, such as tags, are compared directly with upstream.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alecthomas
alecthomas merged commit 7e42c30 into main Nov 17, 2026
8 checks passed
@alecthomas
alecthomas deleted the aat/skip-pointless-ensure-fetch branch November 17, 2026 05:58
alecthomas added a commit that referenced this pull request Nov 17, 2026
Comments in this codebase had drifted into long blocks that restate the
code, and the comment rules in `AGENTS.md` were not enforced. Each
review ended up repeating the same corrections, most recently on #453.
This change turns those rules into linters and brings the existing code
into line, so the rules hold without anyone having to remember them.

The approach is two new `tsk` linters plus one config change:

- `commentshape` limits every comment to one summary line, optionally
followed by a blank comment line and up to two lines of explanation.
Directive lines such as `//nolint:x`, `//go:build` and `// #nosec` do
not count. Both limits are configurable.
- `privatedoc` rejects a doc comment on an unexported function under 20
lines, since those should be clear from their name and body. A comment
that explains why can be kept with `//nolint:privatedoc // <reason>` on
the `func` line.
- The revive exclusion that hid "exported X should have comment" is
removed, so the existing rule that public declarations are always
documented is now enforced.

The rest of the diff is the cleanup those rules required: about 570
comments shortened, a small number deleted where they only restated the
code, and doc comments added to the 162 exported declarations that had
none. Only comments change, apart from whitespace. No
`//nolint:commentshape` was needed.

Two things are worth a reviewer's attention:

- Shortening lost some detail. The comments most affected are
`Tiered.Open` and `statAndHeaders` in `internal/cache`, `ParallelGet`
and `OpenGitSnapshotParallel` in `client`, `Repack` in
`internal/gitclone`, and `ServeHTTP` in `internal/strategy/handler`.
- A few comments were corrected, not just shortened, because the old
text no longer matched the code: the interceptor ordering in
`config.Load`, the HCL example on `HostConfig`, and the metadata storage
note on `NewS3`.

`just fmt` now skips paths under `testdata/`, because the linter
fixtures are separate modules that hold deliberately malformed comments.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants