Skip to content

transfer/indexer: say which file blocks a download and what to do (#200) - #201

Merged
0g-peterzhb merged 1 commit into
mainfrom
name-file-in-existence-errors
Aug 24, 2026
Merged

transfer/indexer: say which file blocks a download and what to do (#200)#201
0g-peterzhb merged 1 commit into
mainfrom
name-file-in-existence-errors

Conversation

@0g-peterzhb

@0g-peterzhb 0g-peterzhb commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Fixes #200. Supersedes #199, which took the other approach and was dropped.

The problem

checkFileExistence reported both outcomes anonymously, so the user saw:

Failed to download file: Failed to check file existence: File already exists

No path anywhere. These fire for two very different files.

For the caller's own destination it's merely terse — they passed --file out.dat and can guess.

For a fragment temp file it misleads. Fragments download to <root>.temp in the process working directory. If one survives an attempt — the copy into the output failed (a full disk, which I hit myself today), os.Remove failed, the process was killed mid-iteration — every later attempt reports File already exists for a file the caller never created. So they inspect the out.dat they asked for, find nothing wrong with it, delete it, retry, and get the identical error, with nothing pointing at the real obstacle.

Change

Name the file in both errors, and at the six fragment sites say what to do:

a previous attempt left this fragment file behind; remove it and retry: 0x1111.temp: File already exists

The hint deliberately does not repeat the path — checkFileExistence supplies it — and there's an assertion that it appears exactly once.

The sentence lives in one place, FragmentLeftBehindError, rather than being copied at each of the six fragment sites. It's exported because indexer.Client runs the same loops; the alternative was two unexported copies. Sharing the constructor rather than a bare string constant means the wrapping can't drift either.

The guidance is fragment-specific on purpose: for the caller's own destination, "remove it" would be bad advice, since they may well want to keep the file they already have.

ErrFileAlreadyExists stays wrapped, not replaced, so the errors.Is checks that depend on it keep matching — download_dir.go relies on that in three places. Asserted, not assumed.

Why not reuse the leftover instead

That was #199, and I closed it. Reuse means trusting checkFileExistence as a completeness signal, and it compares only the Merkle root, never the size. Because the root covers zero-padded chunks, a file short by less than a chunk of trailing zeros has the same root:

 16 bytes -> 0xdb4eca15…be3dc
256 bytes -> 0xdb4eca15…be3dc
same root for different sizes? true

The fragment path never validates the assembled output, so a short temp would silently produce a short result. Telling the caller to remove the file requires no such trust.

Testing

Test
TestCheckFileExistence_NamesTheFile both errors carry the path; the matching case still satisfies errors.Is(err, ErrFileAlreadyExists); a different hash stays a distinct, fatal error
TestCheckFileExistence_MissingFileIsNotAnError an absent path is not an error
..._LeftoverFragmentTellsUserToRemoveIt the fragment failure names 0x1111.temp, says "remove it and retry", and mentions the path exactly once

The last test asserts the literal words rather than referencing FragmentLeftBehindError. A test that reads the same constant it verifies would pass for any message, including nonsense — the point is that the user sees actionable wording, so that belongs as a literal in the test.

RED on main: "the message must say which file", twice. go build, go vet, gofmt and go test -count=1 ./... (14 packages) all pass.


This change is Reviewable

@0g-peterzhb
0g-peterzhb force-pushed the name-file-in-existence-errors branch from c206b6e to 4a5b714 Compare August 24, 2026 16:34
@0g-peterzhb 0g-peterzhb changed the title transfer: name the file in the existence errors (#200) transfer/indexer: say which file blocks a download and what to do (#200) Aug 24, 2026
checkFileExistence reported both outcomes without saying which file it meant, so what
reached the user was "Failed to download file: Failed to check file existence: File
already exists" with no path at all.

For the caller's own destination that is merely terse. For a fragment temp file it
misleads: fragments are downloaded to <root>.temp in the process working directory, and
if one survives an attempt - the copy into the output failed, os.Remove failed, the
process was killed mid-iteration - every later attempt reports "File already exists"
for a file the caller never created. They inspect the output file they asked for, find
nothing wrong, delete it, retry, and get the same error, with nothing indicating what
is actually blocking them.

Name the file in both errors, and at the fragment sites add what to do about it via a
single shared FragmentLeftBehindError rather than six copies of the sentence. The hint
does not repeat the path, since checkFileExistence now supplies it. It is deliberately
fragment-specific: the same ErrFileAlreadyExists is reported for the caller's own
destination, where "remove it" would be bad advice since they may want to keep the file
they already have. Exported because indexer.Client runs the same fragment loops; the
alternative was two unexported copies.

ErrFileAlreadyExists stays wrapped rather than replaced, so the errors.Is checks that
depend on it keep matching - download_dir relies on it in three places.

Reusing the leftover instead was tried in #199 and dropped: checkFileExistence compares
only the merkle root, never the size, and because the root covers zero-padded chunks a
file short by under a chunk of trailing zeros has the same root - 16 bytes and 256
bytes of the same padded content produce identical roots. Telling the caller to remove
it needs no such trust.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@0g-peterzhb
0g-peterzhb force-pushed the name-file-in-existence-errors branch from 4a5b714 to 206e5c7 Compare August 24, 2026 19:27
@0g-peterzhb
0g-peterzhb merged commit 7d4a015 into main Aug 24, 2026
2 checks passed
@0g-peterzhb
0g-peterzhb deleted the name-file-in-existence-errors branch August 24, 2026 19:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

transfer: File already exists errors do not say which file, misleading fragment retries

1 participant