Compute depth for broken symlinks so --min-depth keeps them - #2039
Compute depth for broken symlinks so --min-depth keeps them#2039hexbinoct wants to merge 4 commits into
Conversation
tmccombs
left a comment
There was a problem hiding this comment.
Unfortunately, this doesn't work properly if the --absolute-path option is used. Probably because we are comparing the absolute path to a relative path, and not getting a match.
In particular, my test case looked like:
❯ tree
.
├── a
│ ├── b
│ │ ├── c
│ │ │ ├── d
│ │ │ │ └── foo -> /noexist
│ │ │ └── foo -> /noexist
│ │ ├── foo -> /noexist
│ │ └── two -> /tmp/tmp.od15gRvlEL/a/two
│ ├── foo -> /noexist
│ └── two
│ └── c
│ └── foo -> /noexist
├── foof -> /noexist
└── l -> a/b
9 directories, 6 files
❯ fd -L foo l -a
/tmp/tmp.od15gRvlEL/a/b/c/d/foo
/tmp/tmp.od15gRvlEL/a/b/c/foo
/tmp/tmp.od15gRvlEL/a/b/foo
/tmp/tmp.od15gRvlEL/a/b/two/c/foo
Notice that /tmp/tmp.od15gRvlEL/a/b/foo is printed even though it's relative path would be l/foo which isn't deep enough for the --min-path condition.
| BrokenSymlink(PathBuf), | ||
| // Broken symlinks are surfaced by the walker as errors that carry no depth, | ||
| // so we record the depth (relative to the search root) at creation time. | ||
| BrokenSymlink(PathBuf, usize), |
There was a problem hiding this comment.
It would probably be best to change this to be a struct-like variant, to make it more clear what the usize value is.
There was a problem hiding this comment.
Done. It is now a struct-like variant with named path and depth fields, so the recorded depth is clear at every match and construction site.
| /// fall back to the full component count, which keeps the entry visible rather | ||
| /// than silently dropping it. |
There was a problem hiding this comment.
This will show it for --min-depth, but will probably hide it if --max-depth or --exact-depth is used. Which, I'm not sure what we should do in that case.
There was a problem hiding this comment.
I checked this and it actually behaves the same as a normal entry. --max-depth (and the upper bound of --exact-depth) is enforced by the walker itself through its own internal depth, and --min-depth (and the lower bound of --exact-depth) is the only one applied here using the computed depth. So broken symlinks are kept and dropped at the same depths as real files for all three flags.
To confirm, I placed a real file next to each broken symlink and verified that --min-depth, --max-depth and --exact-depth select the same set for both, in relative and --absolute-path modes, for depths 0 to 4. I also added regression tests test_max_depth_broken_symlink and test_exact_depth_broken_symlink that exercise both the real directory route and the followed symlink route.
|
Thanks for catching this. You're right that comparing an absolute entry path against a relative root would miss, so I changed the depth computation to put both the path and the roots into absolute form before stripping, and to pick the deepest matching root. That removes the relative vs absolute mismatch. I also tried to reproduce the original symptom on the current code, to be sure I was fixing the right thing. With your tree the depths come out correct even before the change, because To double check, I put a real file next to each broken symlink and confirmed If you have a setup where the root stays relative under |
|
I tested the current PR branch locally on macOS to check the original #1017 case and the Checks run:
For the manual
The commands used
|
|
Thanks a lot for taking the time to build the tree and run this independently on macOS, @bugprone. Good to have the |
6998ab6 to
df27ff6
Compare
|
Rebased onto current master to clear a merge conflict. The only adaptation was in the broken symlink arm in On sequencing: whenever the |
93361ce to
70c4d0f
Compare
|
Rebased onto master to pick up #2068. The clippy failure here was the new |
When following links, a broken symlink is surfaced by the walker as an error that carries no depth, so fd stored it with an unknown depth and the --min-depth filter dropped it for any minimum value. Record the depth when the entry is created by counting the path components relative to the matching search root, which is what the issue discussion suggested. Broken symlinks are now filtered by depth like any other entry. Fixes sharkdp#1017
The depth for a broken symlink was derived by stripping the search root from its path and counting the remaining components. Compare the path and the roots in absolute form before stripping, so the match holds whether or not --absolute-path has already made the roots absolute, and pick the deepest matching root. This avoids an absolute path failing to match a relative root and falling back to an inflated depth. Add a regression test that exercises the broken symlink case together with --absolute-path.
Address review feedback. Make DirEntryInner::BrokenSymlink a struct-like variant with named path and depth fields, so the recorded depth is clear at every use site. Add regression tests that a broken symlink is filtered by --max-depth and --exact-depth the same way a normal entry is, both through the real directory tree and through a followed symlink.
The depth of a broken symlink was derived from its path by stripping the matching search root, because the walker did not report a depth on the errors that carry broken symlinks. ignore 0.4.28 fills that depth in, so read it from the error instead and drop the path arithmetic. The path based version could not express one case. When two search roots overlap, the walker visits the same broken symlink once per root at a different depth each time, and both visits carry the same path, so a single path derived answer had to serve both. Searching roots a and a/b with --min-depth 2 dropped a/b/blink even though a real file in its place was kept.
70c4d0f to
b3a8ff6
Compare
Fixes #1017.
Problem
When following links, a broken symlink is reported by the walker as an error instead of a normal entry, so fd wraps it in
DirEntry::broken_symlink. That variant carried no depth, andDirEntry::depth()returnedNonefor it. The--min-depthfilter inwalk.rstreats a missing depth as below the minimum, so a broken symlink was filtered out for any--min-depthvalue, including--min-depth 0. The original report shows it:Fix
The depth now comes from the walker itself.
ignorerecords a depth on the errors that carry broken symlinks, sowalk.rsreadserr.depth()before taking the error apart and passes it toDirEntry::broken_symlink.DirEntry::depth()returns it, and broken symlinks are filtered by depth like every other entry.Earlier revisions of this PR derived the depth from the path by stripping the matching search root, because the walker did not report one at the time. @tmccombs then added it upstream in BurntSushi/ripgrep#3458 and #3464, which is what this now uses. The path arithmetic is deleted, along with the
current_dir()call it made for every broken symlink.That upstream depth also fixes a case the path version could not express. When two search roots overlap, the walker visits the same broken symlink once per root at a different depth each time, and both visits carry the same path, so one path derived number had to answer for both. Searching roots
aanda/bwith--min-depth 2droppeda/b/blinkwhile a real file in the same place was kept.ignoreis bumped to 0.4.28, which is the first release where the depth actually arrives on these errors.Error::depth()andWithDepthalready existed in 0.4.26, so I bisected 0.4.26 through 0.4.33 to find the real floor: 0.4.26 and 0.4.27 returnNone, 0.4.28 onward return the right depth.Tests
Five tests cover
--min-depth,--min-depthwith--absolute-path,--max-depth,--exact-depth, and the overlapping roots case. Each was checked to fail on the code before it and pass after. The fullcargo testsuite passes, fmt and clippy are clean, and the tests were run pinned to exactly 0.4.28 as well as to the newestignore, since a caret requirement resolving to 0.4.33 would not have proved the floor.Verified on Linux, including a matrix over
--min-depth,--max-depthand--exact-depthfrom 0 to 4 comparing a broken symlink against a real file at the same depth, in both relative and--absolute-pathmodes. There is a CHANGELOG entry.Note on tooling
I used an AI assistant (Claude) to help write this change. I have reviewed it, I understand how it works, and I am happy to answer questions or make changes.