Fix stream_file position accounting for multi-buffer operations on Windows - #1762
Open
gycherish wants to merge 1 commit into
Open
Fix stream_file position accounting for multi-buffer operations on Windows#1762gycherish wants to merge 1 commit into
gycherish wants to merge 1 commit into
Conversation
…ndows. win_iocp_file_service advances the file position by the size of the whole buffer sequence, but win_iocp_handle_service submits only the sequence's first non-empty buffer, ReadFile and WriteFile taking one buffer each. The position therefore moves past data that has not been transferred, and the composed read/write operations resume from there: a scatter write leaves a hole in the file, and a scatter read reports end_of_file with the later buffers unfilled. Advance by the buffer that is actually submitted. That size is known before the operation is initiated, so the accounting stays where it is today and no completion handler is involved. A single-buffer sequence is unaffected, its first buffer being the whole sequence. Only the Windows backend maintains the position itself, IOCP requiring an explicit offset per operation; io_uring and the reactive backends leave it to the kernel and pass the whole sequence to writev in one operation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1346.
win_iocp_file_serviceadvances the file position by the size of the whole buffer sequence:uint64_t offset = impl.offset_; impl.offset_ += asio::buffer_size(buffers); handle_service_.async_write_some_at(impl, offset, buffers, handler, io_ex);but
win_iocp_handle_servicesubmits only the sequence's first non-empty buffer,ReadFileandWriteFiletaking a single buffer each:asio::const_buffer buffer = buffer_sequence_adapter<asio::const_buffer, ConstBufferSequence>::first(buffers); return do_write(impl, offset, buffer, ec);So the position moves past data that has not been transferred, and the composed operation resumes
from there. A scatter write leaves a hole in the file; a scatter read reports
end_of_filewith thelater buffers unfilled and no error the caller can act on.
All four sites are affected:
write_some,async_write_some,read_some,async_read_some.The fix
Advance by the buffer that is actually submitted. That size is known before the operation is
initiated, so the accounting stays exactly where it is today — no completion handler is involved and
nothing has to reach
implafter the operation finishes.A single-buffer sequence is unaffected: its first buffer is the whole sequence, so
first(buffers).size() == buffer_size(buffers).buffer_sequence_adapteris already visible in this header throughwin_iocp_handle_service.hpp.Scope
This makes a multi-buffer sequence behave like a single-buffer one. It does not change what happens
when the OS transfers less than the buffer it was given — a short read at EOF still over-advances,
exactly as it does today for a one-buffer sequence. Covering that would mean advancing by
bytes_transferredfrom the completion handler; that is a separate concern from the gaps reportedin #1346, and this change does not depend on it.
Only Windows is affected, and structurally so: IOCP needs an explicit offset per operation, so
win_iocp_file_serviceis the one backend that tracks the stream position itself.io_uring_file_serviceforwards to the descriptor service and keeps nooffset_— the kernel ownsthe position, and
io_uring_prep_writevtakes the whole sequence in one SQE.random_access_fileand the
*_atoverloads are unaffected either way, taking an explicit offset and never touchingimpl.offset_.