Skip to content

fix: ignore again... #125

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 52 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
90c23be
Add entity-manager dep
rklaehn Jul 8, 2025
90db719
Remove unused dependencies
rklaehn Jul 8, 2025
24c7f2c
Merge branch 'lose-futures-buffered-dep' into entity-manager
rklaehn Jul 8, 2025
126d834
Merge branch 'main' into entity-manager
rklaehn Jul 8, 2025
264ed84
Add import
rklaehn Jul 8, 2025
a859674
Add helpers for set and update
rklaehn Jul 8, 2025
baf4d54
Make bipolar clippy happy again
rklaehn Jul 8, 2025
66f8540
Move message sending into meta
rklaehn Jul 8, 2025
9193b08
Merge branch 'remove-db-public-send' into entity-manager
rklaehn Jul 8, 2025
8cae7cd
WIP
rklaehn Jul 8, 2025
161a70b
Replace slots with EntityHandler
rklaehn Jul 8, 2025
169261a
Get rid of the weak reference
rklaehn Jul 8, 2025
6fe2e07
Inline entity manager and streamline the dispatcher code a bit
rklaehn Jul 9, 2025
fd9faab
revert a few changes to minimize the diff
rklaehn Jul 9, 2025
8ac1b78
codespell
rklaehn Jul 9, 2025
011746d
remove debug code
rklaehn Jul 9, 2025
def9abc
more comments
rklaehn Jul 9, 2025
3d603b0
remove options from BaoFileHandleInner
rklaehn Jul 9, 2025
e65bd07
Get rid of BaoFileHandleInner and all that indirection
rklaehn Jul 9, 2025
e81b1f3
change a few fns to take HashContext directly instead of 2 args
rklaehn Jul 9, 2025
27df6da
more using HashContext instead of TaskContext
rklaehn Jul 9, 2025
77fbb05
First step for removing the slot mutex
rklaehn Jul 9, 2025
ae95f28
prepare BaoFileHandle to be the full state
rklaehn Jul 9, 2025
c786a04
Flatten the state we keep per active blob
rklaehn Jul 10, 2025
94df43f
Remove debug statements
rklaehn Jul 10, 2025
48ffd48
Remove unused code
rklaehn Jul 10, 2025
a540002
Remove more debug code
rklaehn Jul 10, 2025
6e237f8
instrument the per-hash tasks as well
rklaehn Jul 10, 2025
47cab9c
Add property based tests for the entity manager
rklaehn Jul 10, 2025
00c54f3
Add a property based test that tests basic functionality of the entit…
rklaehn Jul 10, 2025
664beb8
clippy
rklaehn Jul 10, 2025
7b1b71b
Merge branch 'main' into entity-manager
rklaehn Jul 10, 2025
e584ad1
fmt
rklaehn Jul 10, 2025
4412be9
Add tests for the busy and dead cases which are hard to trigger witho…
rklaehn Jul 10, 2025
c26e555
clippy again
rklaehn Jul 10, 2025
002469a
Merge branch 'entity-manager' into slim-down-state
rklaehn Jul 11, 2025
7857e38
Merge branch 'slim-down-state' into remove-slot-mutex
rklaehn Jul 11, 2025
f559598
Move things around in preparation for introducing traits
rklaehn Jul 11, 2025
bc6bc71
Group high level fns.
rklaehn Jul 11, 2025
632131f
Introduce traits to structure the API
rklaehn Jul 11, 2025
8004115
Merge branch 'main' into entity-manager
rklaehn Jul 11, 2025
4020bd4
Merge branch 'entity-manager' into slim-down-state
rklaehn Jul 11, 2025
1665781
Merge branch 'slim-down-state' into remove-slot-mutex
rklaehn Jul 11, 2025
9a8547a
clippy
rklaehn Jul 11, 2025
c57f259
Merge branch 'main' into entity-manager
rklaehn Jul 23, 2025
331a3aa
Merge branch 'entity-manager' into slim-down-state
rklaehn Jul 23, 2025
e2b76fb
Merge branch 'slim-down-state' into remove-slot-mutex
rklaehn Jul 23, 2025
42bbbec
Merge branch 'main' into entity-manager
rklaehn Jul 24, 2025
b595923
Merge branch 'entity-manager' into slim-down-state
rklaehn Jul 24, 2025
6d39d40
Merge branch 'slim-down-state' into remove-slot-mutex
rklaehn Jul 24, 2025
c6fc5ad
instrument the failing test
rklaehn Jul 24, 2025
3b72fca
Add wait_idle api call.
rklaehn Jul 24, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 22 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ testresult = "0.4.1"
tracing-subscriber = { version = "0.3.19", features = ["fmt"] }
tracing-test = "0.2.5"
walkdir = "2.5.0"
atomic_refcell = "0.1.13"
iroh = { version = "0.90", features = ["discovery-local-network"]}

[features]
Expand Down
15 changes: 15 additions & 0 deletions src/api.rs
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ pub mod downloader;
pub mod proto;
pub mod remote;
pub mod tags;
use crate::api::proto::WaitIdleRequest;
pub use crate::{store::util::Tag, util::temp_tag::TempTag};

pub(crate) type ApiClient = irpc::Client<proto::Command, proto::Request, proto::StoreService>;
Expand Down Expand Up @@ -314,6 +315,7 @@ impl Store {
Request::ClearProtected(msg) => local.send((msg, tx)),
Request::SyncDb(msg) => local.send((msg, tx)),
Request::Shutdown(msg) => local.send((msg, tx)),
Request::WaitIdle(msg) => local.send((msg, tx)),
}
})
});
Expand All @@ -332,6 +334,19 @@ impl Store {
Ok(())
}

/// Waits for the store to become completely idle.
///
/// This is mostly useful for tests, where you want to check that e.g. the
/// store has written all data to disk.
///
/// Note that a store is not guaranteed to become idle, if it is being
/// interacted with concurrently. So this might wait forever.
pub async fn wait_idle(&self) -> irpc::Result<()> {
let msg = WaitIdleRequest;
self.client.rpc(msg).await?;
Ok(())
}

pub(crate) fn from_sender(client: ApiClient) -> Self {
Self { client }
}
Expand Down
5 changes: 5 additions & 0 deletions src/api/proto.rs
Original file line number Diff line number Diff line change
Expand Up @@ -134,11 +134,16 @@ pub enum Request {
#[rpc(tx = oneshot::Sender<super::Result<()>>)]
SyncDb(SyncDbRequest),
#[rpc(tx = oneshot::Sender<()>)]
WaitIdle(WaitIdleRequest),
#[rpc(tx = oneshot::Sender<()>)]
Shutdown(ShutdownRequest),
#[rpc(tx = oneshot::Sender<super::Result<()>>)]
ClearProtected(ClearProtectedRequest),
}

#[derive(Debug, Serialize, Deserialize)]
pub struct WaitIdleRequest;

#[derive(Debug, Serialize, Deserialize)]
pub struct SyncDbRequest;

Expand Down
41 changes: 40 additions & 1 deletion src/api/remote.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1064,8 +1064,15 @@ mod tests {
use testresult::TestResult;

use crate::{
api::blobs::Blobs,
protocol::{ChunkRangesSeq, GetRequest},
store::fs::{tests::INTERESTING_SIZES, FsStore},
store::{
fs::{
tests::{create_n0_bao, test_data, INTERESTING_SIZES},
FsStore,
},
mem::MemStore,
},
tests::{add_test_hash_seq, add_test_hash_seq_incomplete},
util::ChunkRangesExt,
};
Expand Down Expand Up @@ -1117,6 +1124,38 @@ mod tests {
Ok(())
}

async fn test_observe_partial(blobs: &Blobs) -> TestResult<()> {
let sizes = INTERESTING_SIZES;
for size in sizes {
let data = test_data(size);
let ranges = ChunkRanges::chunk(0);
let (hash, bao) = create_n0_bao(&data, &ranges)?;
blobs.import_bao_bytes(hash, ranges.clone(), bao).await?;
let bitfield = blobs.observe(hash).await?;
if size > 1024 {
assert_eq!(bitfield.ranges, ranges);
} else {
assert_eq!(bitfield.ranges, ChunkRanges::all());
}
}
Ok(())
}

#[tokio::test]
async fn test_observe_partial_mem() -> TestResult<()> {
let store = MemStore::new();
test_observe_partial(store.blobs()).await?;
Ok(())
}

#[tokio::test]
async fn test_observe_partial_fs() -> TestResult<()> {
let td = tempfile::tempdir()?;
let store = FsStore::load(td.path()).await?;
test_observe_partial(store.blobs()).await?;
Ok(())
}

#[tokio::test]
async fn test_local_info_hash_seq() -> TestResult<()> {
let sizes = INTERESTING_SIZES;
Expand Down
2 changes: 1 addition & 1 deletion src/hash.rs
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ impl From<&[u8; 32]> for Hash {

impl PartialOrd for Hash {
fn partial_cmp(&self, other: &Self) -> Option<std::cmp::Ordering> {
Some(self.0.as_bytes().cmp(other.0.as_bytes()))
Some(self.cmp(other))
}
}

Expand Down
1 change: 1 addition & 0 deletions src/metrics.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ use iroh_metrics::{Counter, MetricsGroup};

/// Enum of metrics for the module
#[allow(missing_docs)]
#[allow(dead_code)]
#[derive(Debug, Default, MetricsGroup)]
#[metrics(name = "iroh-blobs")]
pub struct Metrics {
Expand Down
Loading
Loading