Skip to content

/ fd

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Should -X behavior match xargs? #441

Closed
kbd opened this issue May 13, 2019 · 5 comments
Closed

Should -X behavior match xargs? #441

kbd opened this issue May 13, 2019 · 5 comments
Labels
feature-request good first issue idea

Comments

@kbd
Copy link

@kbd kbd commented May 13, 2019 •
edited

As discussed in this bat issue, the behavior of fd's -X flag is different from the behavior of fd | xargs. My initial impression upon reading the docs was that -X's behavior would be equivalent to piping to xargs, and I was surprised when that wasn't the case.

Shell session for reference:

$ for i in a b c d; do echo $i > $i.txt; done
$ fd
a.txt
b.txt
c.txt
d.txt
$ fd -X bat -p
c
b
a
d
$ fd | xargs bat -p
a
b
c
d
$ fd --version
fd 7.3.0

In short, I would expect -X's order to be the order returned by a normal call to fd (while I would expect -x's order to be random because the docs make clear that operation is parallelized).
@sharkdp
Copy link
Owner

@sharkdp sharkdp commented May 31, 2019 •
edited

Thank you for reporting this.

I'm going to summarize from what I said in the linked ticket:

  • fd's output order is not deterministic (even if it often looks like it actually is, especially in small examples). This means that the execution order can not be "predicted" by first running fd … and then running fd … | xargs ….
  • The same is true for -x/--exec and -X/--exec-batch. fd doesn't make any promises on the order of the search results. And I don't think we should (it would hurt performance).

That being said, I don't really know why the order of fd -X echo seems to be reversed from what fd outputs. If someone want's to look into this, 👍.
@sharkdp sharkdp added the question label May 31, 2019
@tavianator
Copy link
Contributor

@tavianator tavianator commented Jun 4, 2019

I believe this can be explained by the buffering logic, specifically

fd/src/walk.rs

Line 231 in 5cbd840

buffer.sort();

@sharkdp
Copy link
Owner

@sharkdp sharkdp commented Jun 6, 2019 •
edited

Oh, of course. I completely forgot about this.

If the search finishes within the buffering time (100 ms, by default), we actually sort the search results before printing them to the console.

For fd -X, we currently don't do this. I think it might be reasonable to do something similar because we are waiting for all results anyway. Given that we are currently limited by the amount of search results (for -X), the time to sort all results is probably negligible. If we would not be restricted, it might make sense to only sort the results if there are less than some fixed number.
MarcoIeni added a commit to crash-g/fd that referenced this issue Nov 30, 2019
@MarcoIeni
exec/mod.rs: added reason why we sort elements with -X
aed7462 
see sharkdp#441 (comment)
@MarcoIeni MarcoIeni mentioned this issue Nov 30, 2019
Closed
@MarcoIeni MarcoIeni mentioned this issue Jan 2, 2020
Merged
@sharkdp
Copy link
Owner

@sharkdp sharkdp commented Jan 2, 2020

Closed via #524 by @MarcoIeni and @crash-g.
@sharkdp sharkdp closed this Jan 2, 2020
@sharkdp
Copy link
Owner

@sharkdp sharkdp commented Mar 22, 2020

Released in fd 7.5.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature-request good first issue idea
Projects
None yet
Milestone
No milestone
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
@kbd @tavianator @sharkdp
You can’t perform that action at this time.