Skip to content

Conversation

@RedKinda
Copy link

Motivation

Currently IO operations for fs::File is implemented using spawn_blocking to perform IO, which is not great for an async runtime. This PR implements IO operations using io_uring. The fs::write function already uses io_uring for async operations, and this PR extends the uring functionality into impl AsyncWrite for fs::File.

For full discussion see here #7684 and subsequently on discord https://discord.com/channels/500028886025895936/810724255046172692/1427735341003051079

Solution

The fs::File struct already implements an internal buffer when writing using spawn_blocking, which is similar to what io_uring requires. Therefore in poll_write, we can check if io_uring is initialized and supported, and if not fall back to the spawn_blocking implementation, without requiring any API changes.

This PR only implements AsyncWrite, and once this PR is accepted, I am happy to also open a PR for AsyncRead for File using the same structure.

@ADD-SP ADD-SP added C-enhancement Category: A PR with an enhancement or bugfix. A-tokio Area: The main tokio crate M-fs Module: tokio/fs labels Oct 27, 2025
}

cfg_io_uring! {
struct BoxedOp<T>(Pin<Box<dyn Future<Output = T> + Send + Sync + 'static>>);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the Future need to be Sync ?
I think it is not really necessary.

Copy link
Author

@RedKinda RedKinda Oct 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In principle you could avoid Sync using src/util/sync_wrapper.rs. But for this use-case, it does not matter.

let handle = BoxedOp(Box::pin(async move {
let (r, buf, _fd) = op.await;
match r {
Ok(_n) => (Operation::Write(Ok(())), buf),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is a chance that the buffer is not fully drained here.
Write(Ok()) should be returned only if buf.is_empty().

I think you need to wrap the write in a loop and handle it with something like:

Ok(0) => break (Operation::Write(Err(io::ErrorKind::WriteZero.into())), buf),
Ok(_) if buf.is_empty() => break (Operation::Write(Ok(())), buf),
Ok(_) => continue, // more to write
Err(e) => break (Operation::Write(Err(e)), buf),

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thanks and fixed

let handle = BoxedOp(Box::pin(async move {
let (r, buf, _fd) = op.await;
match r {
Ok(_n) => (Operation::Write(Ok(())), buf),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above - make sure the buf is empty before saying Ok().

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-tokio Area: The main tokio crate C-enhancement Category: A PR with an enhancement or bugfix. M-fs Module: tokio/fs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants