-
Notifications
You must be signed in to change notification settings - Fork 2.2k
attempt to fix merge ff with multiple remotes and multiple lfs commits #6171
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3,6 +3,8 @@ package commands | |
| import ( | ||
| "os" | ||
|
|
||
| "github.com/git-lfs/git-lfs/v3/git" | ||
| "github.com/git-lfs/git-lfs/v3/lfs" | ||
| "github.com/git-lfs/git-lfs/v3/tr" | ||
| "github.com/rubyist/tracerx" | ||
| "github.com/spf13/cobra" | ||
|
|
@@ -18,6 +20,8 @@ func postMergeCommand(cmd *cobra.Command, args []string) { | |
| os.Exit(1) | ||
| } | ||
|
|
||
| fetchMissingLfsObjects("ORIG_HEAD", "HEAD") | ||
|
|
||
| // Skip entire hook if lockable read only feature is disabled | ||
| if !cfg.SetLockableFilesReadOnly() { | ||
| os.Exit(0) | ||
|
|
@@ -46,6 +50,41 @@ func postMergeCommand(cmd *cobra.Command, args []string) { | |
| } | ||
| } | ||
|
|
||
| func fetchMissingLfsObjects(pre, post string) { | ||
| remote := cfg.Remote() | ||
| if r := git.FirstRemoteForTreeish(post); r != "" { | ||
| remote = r | ||
| } | ||
|
Comment on lines
+55
to
+57
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Our only other use of the The All of which is to say that I think if we are going to fetch Git LFS objects in situations where we did not previously do so, we should select the remote(s) from which to fetch following the same logic we do now when "smudging". For one thing, as was pointed out during the review of PR #5066 when the |
||
|
|
||
| q := newDownloadQueue( | ||
| getTransferManifestOperationRemote("download", remote), | ||
| remote, | ||
| ) | ||
|
|
||
| gitscanner := lfs.NewGitScanner(cfg, func(p *lfs.WrappedPointer, err error) { | ||
| if err != nil { | ||
| LoggedError(err, tr.Tr.Get("Scanner error: %s", err)) | ||
| return | ||
| } | ||
| lfs.LinkOrCopyFromReference(cfg, p.Oid, p.Size) | ||
| if cfg.LFSObjectExists(p.Oid, p.Size) { | ||
| return | ||
| } | ||
|
|
||
| tracerx.Printf("post-merge: found LFS object %s (%s)", p.Oid, p.Name) | ||
| q.Add(downloadTransfer(p)) | ||
| }) | ||
|
|
||
| if err := gitscanner.ScanRefRange(post, pre, nil); err != nil { | ||
| LoggedError(err, tr.Tr.Get("Scanner error: %s", err)) | ||
| } | ||
|
|
||
| q.Wait() | ||
| for _, err := range q.Errors() { | ||
| LoggedError(err, tr.Tr.Get("Download error: %s", err)) | ||
| } | ||
| } | ||
|
|
||
| func init() { | ||
| RegisterCommand("post-merge", postMergeCommand, nil) | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -225,7 +225,7 @@ func (c *Configuration) IsDefaultRemote() bool { | |
| } | ||
|
|
||
| func (c *Configuration) AutoDetectRemoteEnabled() bool { | ||
| return c.Git.Bool("lfs.remote.autodetect", false) | ||
| return c.Git.Bool("lfs.remote.autodetect", true) | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Alas, as I noted in my initial comments on this PR, we can't change this option's default setting at this time. In general, we try to avoid altering the defaults for our configuration options unless there's a critical bug which we need to correct. Barring that type of problem, if we do change the default settings for our options, that change should be part of a new major version of the Git LFS client and should be accompanied by a suitably lengthy period of advance notice to our users. |
||
| } | ||
|
|
||
| func (c *Configuration) SearchAllRemotesEnabled() bool { | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we want to introduce this type of behaviour to our post-merge hook program, I think we should make it optional, and also not the default. Perhaps we could introduce an
lfs.fetchPostMergeconfiguration option for this purpose, whose default value isfalse.My rationale for this suggestion is based on two concerns. First, I suspect that users may be surprised if a
git mergeoperation, which typically requires only local access, now requires connectivity to a remote in order to retrieve Git LFS objects, and possibly even new credentials.Second, while some users may wish to retrieve all the Git LFS objects referenced in intermediate commits in the histories of the merged branches (for instance, if they wish to then push the merged result to another remote, as in your case), other users may want to keep as few Git LFS objects as possible in local storage. These users typically deal with large repositories where the Git LFS client's duplicative local storage is a burden, and so they may not want to automatically fetch a full history's worth of objects when they perform a merge.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What if we move it to pre push hook ?
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interesting question! I think my comments would largely all still apply. In part that's because it might be surprising to a user if they push to remote A and are asked for credentials for remote B in order to download Git LFS objects, especially if remote A is just another copy of the repository on their local disk.
I skimmed through the Git mailing list archives for discussions of a post-fetch hook, and the topic has come up a few times. One relevant early message from the Git maintainer outlines conditions for a new hook, one of which would seem to broadly apply to the Git LFS use case:
In another message from a thread about a potential "tweak-fetch" hook the Git maintainer points out that a typical
git pulloperation invokesgit fetchand thengit merge. So if we have to work with just the existing hooks provided by Git, I think the post-merge hook is (somewhat) more appropriate than the pre-push hook.In practice, of course, we primarily rely on the "smudge" operation to download the Git LFS objects the user requires in order to perform the final updates of the working tree, both when
git pullis run but also when the user runsgit checkoutorgit merge.The challenge with this, as you've pointed out, is that we expect Git LFS objects to be available from the default Git LFS remote the user has configured, and we only "see" the objects in the current checkout, not any intermediate objects in the Git history (with the caveat that not all Git LFS users want those intermediate objects to be automatically retrieved).
A post-fetch hook, if one existed, would presumably inform the hook program about the remote from which Git objects were fetched, as well as the IDs of those objects. Hypothetically, a Git LFS post-fetch hook might then be able to download any corresponding Git LFS objects before any subsequent checkout stage, so our "smudge" filter would find that all the Git LFS objects necessary for the working tree checkout already exist locally and thus wouldn't need to download anything at all.
We'd want such a hook program to respect all the
lfs.fetch*configuration options, likelfs.fetchExclude.I'm slightly uncertain, though, as to how our existing remote server discovery process could or should apply if the remote provided to the post-fetch hook differs from, say, an explicitly configured
lfs.urlsetting. I think we'd need to work through the full range of potential conditions here and make sure we weren't doing anything too surprising for users.But that's all hypothetical, of course, since Git doesn't have a post-fetch hook right now.