Optimizations for readers #192
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
acogoluegnes
added a commit
to rabbitmq/rabbitmq-server
that referenced
this pull request
Jul 4, 2025
This saves a system call by sending the frame header and the chunk header at the same time. References rabbitmq/osiris#192
acogoluegnes
added a commit
to rabbitmq/rabbitmq-server
that referenced
this pull request
Jul 23, 2025
This saves a system call by sending the frame header and the chunk header at the same time. References rabbitmq/osiris#192
The callback is used in RabbitMQ to send the frame header. This commit changes the callback to expect a binary result and to send it along the chunk header. This saves a system call and improves performance with small chunks.
acogoluegnes
force-pushed
the
send-file-improvements
branch
4 times, most recently
from
5fdaec1 to
09ef637
Compare
Reading a chunk header then using the sendfile syscall to send the chunk data is not optimal for small chunks, as it amounts to 2 system calls for a small amount of bytes. This commit uses the size of the last chunk to try to read the whole chunk when the library reads the chunk header. If the last chunk data size was under 4096 bytes, the library does not only read the chunk header, but reads ahead to try to read the whole and thus to save the sendfile call later. This optimization should improve the read throughput for streams with small chunks, e.g. streams with 1-message chunks.
acogoluegnes
force-pushed
the
send-file-improvements
branch
from
c5572f0 to
bda8f17
Compare
acogoluegnes
added a commit
to rabbitmq/rabbitmq-server
that referenced
this pull request
Jul 24, 2025
This saves a system call by sending the frame header and the chunk header at the same time. References rabbitmq/osiris#192
acogoluegnes
changed the title
kjnilsson
force-pushed
the
send-file-improvements
branch
from
c16bed9 to
b50de25
Compare
Start with default filter size and use actual size after non-zero value has been read. Attempt to read ahead from parse_header instead of returning the header and no data. The decision is based on the availability of previous read ahead data.
acogoluegnes
added a commit
to rabbitmq/rabbitmq-server
that referenced
this pull request
Aug 1, 2025
This saves a system call by sending the frame header and the chunk header at the same time. References rabbitmq/osiris#192
acogoluegnes
added a commit
to rabbitmq/rabbitmq-server
that referenced
this pull request
Aug 4, 2025
This saves a system call by sending the frame header and the chunk header at the same time. References rabbitmq/osiris#192
acogoluegnes
added a commit
to rabbitmq/rabbitmq-server
that referenced
this pull request
Aug 4, 2025
This saves a system call by sending the frame header and the chunk header at the same time. References rabbitmq/osiris#192
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A couple of optimizations when sending chunks to readers: