[Storage] Refactor stored chunk data pack #7983

zhangchiqing · 2025-09-30T03:08:41Z

This PR:

Introduces a new StoredChunkDataPacks store and refactors NewChunkDataPacks to depend on it, wiring it through node startup and CLI tools (execution builder, read-badger, rollback cmd). This splits storage of chunk packs from the protocol DB.
Changes the write path: chunkDataPacks.Store(...) now returns a closure that writes the ChunkID→StoredChunkDataPack.ID mapping inside the protocol DB batch; only LockInsertOwnReceipt is held. Improves atomicity and clarifies failure modes.
Updates rollback to batch-remove multiple chunk data packs at once (BatchRemove(chunkIDs, writeBatch, chunkBatch)), simplifying error handling.
Verification requester API becomes more informative: RequestQualifierFunc now returns (bool, string) and MaxAttemptQualifier includes a reason when unqualified.

…stored-chunk-data-pack

codecov-commenter · 2025-10-01T00:26:27Z

Codecov Report

❌ Patch coverage is 42.35589% with 230 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
storage/mock/stored_chunk_data_packs.go	0.00%	74 Missing ⚠️
storage/chunk_data_packs_stored.go	0.00%	44 Missing ⚠️
storage/mock/chunk_data_packs.go	0.00%	31 Missing ⚠️
storage/store/chunk_data_packs.go	71.71%	22 Missing and 6 partials ⚠️
storage/store/chunk_data_packs_stored.go	67.79%	14 Missing and 5 partials ⚠️
engine/execution/state/state.go	75.00%	5 Missing and 2 partials ⚠️
engine/verification/requester/qualifier.go	53.84%	6 Missing ⚠️
module/pruner/pruners/chunk_data_pack.go	0.00%	6 Missing ⚠️
cmd/execution_builder.go	0.00%	4 Missing ⚠️
...ck-executed-height/cmd/rollback_executed_height.go	62.50%	3 Missing ⚠️
... and 4 more

📢 Thoughts on this report? Let us know!

zhangchiqing · 2025-10-02T23:55:52Z

cmd/util/cmd/rollback-executed-height/cmd/rollback_executed_height.go

+	}
+
+	if len(chunkIDs) > 0 {
+		err := chunks.BatchRemove(chunkIDs, writeBatch, chunkBatch)


BatchRemove now takes two batch, one for batch operation on protocol db, the other on chunk data pack db.

zhangchiqing · 2025-10-02T23:57:27Z

engine/execution/state/state.go

 }

+// SaveExecutionResults saves all data related to the execution of a block.
+// It is concurrent-safe


It is concurrent safe because chunk data packs store is conflict-free (storing data by hash), and protocol data requires a lock to store, which will be synchronized.

Suggested change

// It is concurrent-safe

// It is concurrent safe because chunk data packs store is conflict-free (storing data by hash), and protocol data requires a lock to store, which will be synchronized.

zhangchiqing · 2025-10-02T23:58:17Z

engine/verification/requester/qualifier.go

 //
 // The return value of this function determines whether the chunk request can be dispatched to the network.
-type RequestQualifierFunc func(attempts uint64, lastRequested time.Time, retryAfter time.Duration) bool
+type RequestQualifierFunc func(attempts uint64, lastRequested time.Time, retryAfter time.Duration) (bool, string)


This change is an minor improvement for debugging why chunk data pack is not received on VN.

zhangchiqing · 2025-10-03T00:01:21Z

module/block_iterator/executor/executor_test.go

-		// prepare data
-		cdps := make([]*flow.ChunkDataPack, 0, blockCount)
+		// prepare data - create execution receipts for blocks
+		receipts := make([]*flow.ExecutionReceipt, 0, blockCount)


Previously this tests was using chunk data pack for testing the iterator (pruner), but now the chunk data pack is more complex to store, I switched it to execution receipt, which is simpler, because it is also indexed by block ID.

zhangchiqing · 2025-10-03T00:02:38Z

module/pruner/pruners/chunk_data_pack.go


+	// remove all chunk data packs in a single batch operation
+	if len(chunkIDs) > 0 {
+		err := p.chunkDataPacks.BatchRemoveStoredChunkDataPacksOnly(chunkIDs, batchWriter)


The chunk data pack pruner will only prune the store chunk data pack in the chunk data pack database, it won't remove the chunkID and CDP ID mapping in the protocol database.

zhangchiqing · 2025-10-03T00:04:51Z

storage/operation/prefix.go


 	// legacy codes (should be cleaned up)
-	codeChunkDataPack                      = 100
+	_                                      = 100 // codeChunkDataPack (100) is deprecated


We are deprecating the old table, it's still backward compatible if we roll out carefully.

We need to roll out to ENs one by one, so that an updated node won't be required for the old chunks since they are already been sealed by other ENs.

Under the same assumptions that this is backward-compatible, I think modifying the StoredChunkDataPack.ID() function would also be backward compatible. In either case, we are changing the way chunk data packs are stored such that we cannot retrieve pre-upgrade chunk data packs.

zhangchiqing · 2025-10-03T00:05:13Z

storage/operation/prefix.go

+	codeChunkDataPackID              = 112
+	codeStoredChunkDataPack          = 113


112 and 113 are the new tables to replace the deprecated 100 (codeChunkDataPack)

AlexHentschel

I focused my first first review pass on the core logic. I think there is a good amount of comments and suggestions for a first round of revisions, after which I would continue with the review.

My impression is that there is a subtle foot gun and the possibility to significantly simplify the ID computation of Chunk Data Pack and StoredChunkDataPack (specifically make their ID consistent).

AlexHentschel · 2025-10-07T23:39:28Z

storage/chunk_data_packs_stored.go

+// StoredChunkDataPack is an in-storage representation of chunk data pack.
+// Its prime difference is instead of an actual collection, it keeps a collection ID hence relying on maintaining
+// the collection on a secondary storage.
+type StoredChunkDataPack struct {


I think conceptually, this represents data that the EN made external commitments to, so it should not be modified (?) Hence, we might want to declare it as an immutable object for out linter to protect it:

flow-go/model/flow/header.go

Line 183 in 469d93b

//structwrite:immutable - mutations allowed only within the constructor

👍 I think both this and ChunkDataPack should be marked immutable

AlexHentschel · 2025-10-07T23:43:42Z

storage/chunk_data_packs_stored.go

+	CollectionID      flow.Identifier
+	SystemChunk       bool


I would change this to a pointer *flow.Identifier. Mirrors the behaviour of ChunkDataPack.Collection which is nil. Alternatively, you could use flow.ZeroID to represent unset specifically, i.e. system chunk.

AlexHentschel · 2025-10-08T00:16:58Z

storage/chunk_data_packs_stored.go

+func (c StoredChunkDataPack) ID() flow.Identifier {
+	return flow.MakeID(c)
+}


In my opinion, we are introducing redundant identifiers here, that hash exactly the same data differently. I think that is confusing.

Suggestion:

when computing the ID of a regular Chunk Data Pack, we should compute the collection ID first and hash that, instead of the collection's full object structure:

flow-go/model/flow/chunk.go

Lines 206 to 209 in af1d4da

// ID returns a collision-resistant hash of the ChunkDataPack struct.

func (c *ChunkDataPack) ID() Identifier {

return MakeID(c)

}

The block header does something very similar with the timeout certificate:

flow-go/model/flow/header.go

Line 267 in 469d93b

LastViewTCID: h.LastViewTC.ID(),

As a result, no matter whether you have the data structure with the full collection, or the reduced representation with only the collection hash/ID, in both cases you have the same ID for the overall chunk data pack.

In a way, we are saying: it represents the same data, so it should have the same hash.

I think there is a variety of ways how we can implement this. I think the following would be very clean:

we could introduce a common data structure, "ChunkDataPackHeader"

type ChunkDataPackHeader struct { ChunkID Identifier // ID of the chunk this data pack is for StartState StateCommitment // commitment for starting state ProofHash Identifier // proof for all registers touched (read or written) during the chunk execution CollectionID Identifier // collection executed in this chunk; nil for system chunk // ExecutionDataRoot is the root data structure of an execution_data.BlockExecutionData. // It contains the necessary information for a verification node to validate that the // BlockExecutionData produced is valid. ExecutionDataRoot BlockExecutionDataRoot }

ProofHash and CollectionID have different types compared to Proof and Collection in the original chunk data pack. ChunkDataPack.ID would then compute the ID via ChunkDataPackHeader.ID()

Similarly, StoredChunkDataPack could compute It's ID also via ChunkDataPackHeader. Thereby, it would be totally clear that both representation generate the same ID (ideally we also have a couple tests).

@jordanschalm would love your thoughts here, since I feel my proposal is very close to the malleability approach / design (uniform pattern throughout the code base)

100% agree with making ChunkDataPack and StoredChunkDataPack have the same ID.

We just need to be careful that changing the ID definition here will be safe to deploy as a rolling upgrade, since Testnet has already been sporked.

AlexHentschel · 2025-10-08T00:24:29Z

storage/chunk_data_packs_stored.go

+// Equals compares two StoredChunkDataPack for equality.
+// The second return value is a string describing the first found mismatch, or empty if they are equal.
+func (c StoredChunkDataPack) Equals(other StoredChunkDataPack) (bool, string) {


I find this an odd method signature. How about

Suggested change

// Equals compares two StoredChunkDataPack for equality.

// The second return value is a string describing the first found mismatch, or empty if they are equal.

func (c StoredChunkDataPack) Equals(other StoredChunkDataPack) (bool, string) {

// EnsureEquals compares two StoredChunkDataPack for equality. It returns a [storage.ErrDataMismatch] if

// and only if they are not equal.

func (c StoredChunkDataPack) EnsureEquals(other StoredChunkDataPack) error {

AlexHentschel · 2025-10-08T00:26:55Z

storage/chunk_data_packs_stored.go

+	return sc
+}
+
+func ToStoredChunkDataPacks(cs []*flow.ChunkDataPack) []*StoredChunkDataPack {


Suggested change

func ToStoredChunkDataPacks(cs []*flow.ChunkDataPack) []*StoredChunkDataPack {

func ToStoredChunkDataPacks(cs []*flow.ChunkDataPack) []*StoredChunkDataPack {// ToStoredChunkDataPack converts the given ChunkDataPacks to their reduced representation,

// by replacing the full collection data with its ID (collision-resistant hash).

func ToStoredChunkDataPack(c *flow.ChunkDataPack) *StoredChunkDataPack {

AlexHentschel · 2025-10-08T03:38:02Z

storage/store/chunk_data_packs.go

+		return nil, fmt.Errorf("cannot retrieve stored chunk data pack ID for chunk %x: %w", chunkID, err)
+	}
+
+	// Then retrieve the actual stored chunk data pack using the ID


Suggested change

// Then retrieve the actual stored chunk data pack using the ID

// Then retrieve the reduced representation of the chunk data pack via its ID.

AlexHentschel · 2025-10-08T03:43:33Z

storage/store/chunk_data_packs.go

+		return nil, fmt.Errorf("cannot retrieve stored chunk data pack %x for chunk %x: %w", storedChunkDataPackID, chunkID, err)
 	}

 	chdp := &flow.ChunkDataPack{


Please prefer to use the constructor. Those are really helpful for enforcing immutability and basic data consistency (a lot of constructors with basic data consistency checks have been added during the malleability work).

Suggestion:

first inspect the StoredChunkDataPack whether it represents the system chunk

if not system chunk, retrieve the collection data

construct flow.ChunkDataPack (you probably have to go through a UntrustedChunkDataPack). Sorry, its a bit more code but helps to guarantee immutability throughout the code base.

fyi @jordanschalm for thoughts and recommendations.

AlexHentschel · 2025-10-08T03:54:20Z

storage/store/chunk_data_packs_stored.go

+
+	cache := newCache(collector, metrics.ResourceStoredChunkDataPack,
+		withLimit[flow.Identifier, *storage.StoredChunkDataPack](byIDCacheSize),
+		withStore(operation.InsertStoredChunkDataPack),


Suggested change

withStore(operation.InsertStoredChunkDataPack),

withStore(operation.InsertStoredChunkDataPack),

withRemove[flow.Identifier, *storage.StoredChunkDataPack](operation.RemoveStoredChunkDataPack),

and then you can implement BatchRemove below as

func (ch *StoredChunkDataPacks) BatchRemove(ids []flow.Identifier, rw storage.ReaderBatchWriter) error { for _, id := range ids { err := ch.byIDCache.RemoveTx(rw, id) if err != nil { return fmt.Errorf("cannot remove chunk data pack: %w", err) } } return nil }

(the logic you implemented in batchRemove is essentially already integrated into the cache)

AlexHentschel · 2025-10-08T04:07:55Z

storage/store/chunk_data_packs_stored.go

+// BatchRemove removes multiple StoredChunkDataPacks cs keyed by their IDs in a batch using the provided
+// No error returns are expected during normal operation, even if none of the referenced objects exist in storage.
+func (ch *StoredChunkDataPacks) BatchRemove(ids []flow.Identifier, rw storage.ReaderBatchWriter) error {


⚠️ ❓

I am not sure if I am understanding this correctly. I am concerned that this function is extremely error prone. My thinking:

we have the chunk data pack database (which I like to call "execution database")

and then we have the protocol database

here, we assume that ReaderBatchWriter is for the "execution database".

Here are the two usages of this function:

flow-go/storage/store/chunk_data_packs.go

Line 158 in 8dbfdc1

err := ch.stored.BatchRemove(storedChunkDataPackIDs, chunkDataPackDBBatch)

I think the usage here is correct.

flow-go/storage/store/chunk_data_packs.go

Line 200 in 8dbfdc1

err := ch.stored.BatchRemove(storedChunkDataPackIDs, chunkDataPackDBBatch)

But here, I think the ReaderBatchWriter instance it is receiving is for the protocol database (well, it should be, despite the misleading name).

I hope this illustrates that the instance of ReaderBatchWriter is very easily mixed up.

My gut feeling:

I think StoredChunkDataPacks can operate entirely without ReaderBatchWriter. Reasoning:

we anyway can't guarantee atomic write across two different data bases. So we might as well only focus on atomic writes for the protocol database.

For storing chunks in the "execution database" we just use operations that do their own write batch internally, but to the outside, no write batches are exposed.

AlexHentschel · 2025-10-08T04:24:31Z

storage/store/chunk_data_packs.go

-		ch.byChunkIDCache.Remove(chunkID)
-	})
-	return operation.RemoveChunkDataPack(rw.Writer(), chunkID)
+func (ch *ChunkDataPacks) BatchRemoveStoredChunkDataPacksOnly(chunkIDs []flow.Identifier, chunkDataPackDBBatch storage.ReaderBatchWriter) error {


⚠️

I think this usage here of chunkDataPackDBBatch is incorrect. If I am not mistaken, we are using the same batch to operate on the protocol database (here) and the chunk data pack database (here).

In my opinion, it would be doable to have StoredChunkDataPacks work without ReaderBatchWriter instances (see my previous comment). Then, for ChunkDataPacks, you would only need a single storage.ReaderBatchWriter instance for the protocol database (applies also to ChunkDataPacks.BatchRemove),

jordanschalm · 2025-10-07T21:34:45Z

cmd/util/cmd/rollback-executed-height/cmd/rollback_executed_height.go

+	}
+
+	if len(chunkIDs) > 0 {
+		err := chunks.BatchRemove(chunkIDs, writeBatch, chunkBatch)


Suggested change

err := chunks.BatchRemove(chunkIDs, writeBatch, chunkBatch)

err := chunks.BatchRemove(chunkIDs, protocolDBatch, chunkDBBatch)

jordanschalm · 2025-10-07T21:41:13Z

engine/execution/state/state.go

 }

+// SaveExecutionResults saves all data related to the execution of a block.
+// It is concurrent-safe


Suggested change

// It is concurrent-safe

// It is concurrent safe because chunk data packs store is conflict-free (storing data by hash), and protocol data requires a lock to store, which will be synchronized.

jordanschalm · 2025-10-07T21:46:36Z

engine/verification/requester/qualifier.go

 //
 // The return value of this function determines whether the chunk request can be dispatched to the network.
-type RequestQualifierFunc func(attempts uint64, lastRequested time.Time, retryAfter time.Duration) bool
+type RequestQualifierFunc func(attempts uint64, lastRequested time.Time, retryAfter time.Duration) (bool, string)


Suggested change

type RequestQualifierFunc func(attempts uint64, lastRequested time.Time, retryAfter time.Duration) (bool, string)

type RequestQualifierFunc func(attempts uint64, lastRequested time.Time, retryAfter time.Duration) (requestQualifies bool, reasonMsg string)

jordanschalm · 2025-10-07T21:47:30Z

engine/verification/requester/requester.go


 // canDispatchRequest returns whether chunk data request for this chunk ID can be dispatched.
-func (e *Engine) canDispatchRequest(chunkID flow.Identifier) bool {
+func (e *Engine) canDispatchRequest(chunkID flow.Identifier) (bool, string) {


Suggested change

func (e *Engine) canDispatchRequest(chunkID flow.Identifier) (bool, string) {

func (e *Engine) canDispatchRequest(chunkID flow.Identifier) (canDispatch bool, reasonMsg string) {

jordanschalm · 2025-10-07T21:55:09Z

module/metrics/labels.go

 	ResourceChunkDataPack                         = "chunk_data_pack"                           // execution node
+	ResourceStoredChunkDataPack                   = "chunk_data_pack_stored"                    // execution node


Suggested change

ResourceChunkDataPack = "chunk_data_pack" // execution node

ResourceStoredChunkDataPack = "chunk_data_pack_stored" // execution node

ResourceChunkIDToChunkDataPackIndex = "chunk_data_pack_index" // execution node

ResourceChunkDataPack = "chunk_data_pack" // execution node

I would invert the naming here

jordanschalm · 2025-10-09T18:49:54Z

storage/chunk_data_packs.go

+	// 1. First phase: Store chunk data packs (StoredChunkDataPack) by its hash (storedChunkDataPackID) in chunk data pack database.
+	// 2. Second phase: Create index mappings from ChunkID to storedChunkDataPackID in protocol database


Suggested change

// 1. First phase: Store chunk data packs (StoredChunkDataPack) by its hash (storedChunkDataPackID) in chunk data pack database.

// 2. Second phase: Create index mappings from ChunkID to storedChunkDataPackID in protocol database

// 1. First phase: Store chunk data packs (StoredChunkDataPack) by its hash (storedChunkDataPackID) in chunk data pack database.

// This phase occurs immediately when Store is called

// 2. Second phase: Create index mappings from ChunkID to storedChunkDataPackID in protocol database

// This phase is deferred until the caller of Store invokes the returned functor.

[copied from implementation doc]

jordanschalm · 2025-10-09T18:50:52Z

storage/chunk_data_packs.go

+	// The two-phase approach ensures that:
+	//   - Chunk data pack content is stored atomically in the chunk data pack database
+	//   - Index mappings are created within the same atomic batch update in protocol database


I think the most important aspect to understand about atomicity in this function is that the two phases are NOT committed atomically w.r.t. one another. I think this section is trying to say that each write batch independently is committed atomically, but I'm worried it makes it sound like the whole operation is atomic, especially "within the same atomic batch".

Suggested change

// The two-phase approach ensures that:

// - Chunk data pack content is stored atomically in the chunk data pack database

// - Index mappings are created within the same atomic batch update in protocol database

// CAUTION: Chunk data pack content is stored immediately, and index mappings are stored later.

// This means that the operations performed by Store as a whole are NOT atomic.

[copied from implemention godoc]

jordanschalm · 2025-10-09T18:52:32Z

storage/chunk_data_packs_stored.go

+// StoredChunkDataPack is an in-storage representation of chunk data pack.
+// Its prime difference is instead of an actual collection, it keeps a collection ID hence relying on maintaining
+// the collection on a secondary storage.
+type StoredChunkDataPack struct {


👍 I think both this and ChunkDataPack should be marked immutable

jordanschalm · 2025-10-09T19:03:34Z

engine/execution/state/state.go

+	}
+
+	return storage.WithLock(s.lockManager, storage.LockInsertOwnReceipt, func(lctx lockctx.Context) error {
+		// The batch update writes all execution result data in a single atomic operation.


Suggested change

// The batch update writes all execution result data in a single atomic operation.

// The batch update writes all execution result data (except chunk data pack!) atomically.

I think the most important detail to get across in the docs is that this operation as a whole is NOT atomic.

jordanschalm · 2025-10-09T19:05:12Z

storage/operation/prefix.go

+	codeChunkDataPackID              = 112
+	codeStoredChunkDataPack          = 113


Suggested change

codeChunkDataPackID = 112

codeStoredChunkDataPack = 113

codeIndexChunkDataPackByChunkID = 112

codeChunkDataPack = 113

zhangchiqing added 11 commits September 29, 2025 10:44

add StoreChunkDataPacks

05faf10

update store chunk data pack

a75653e

fix test case

82f14f0

fix for execution state

e690d56

fix test

70f3cc6

fix builder

c998e28

update testutil engine

9ab5e0a

update chunk data packs and tests

6a4b59e

fix executor tests

17821a5

fix lint

848c9e7

update comments

fe584bd

zhangchiqing mentioned this pull request Sep 30, 2025

[Storage] Refactor insert chunk data pack #7939

Merged

zhangchiqing added 12 commits September 30, 2025 11:30

update comments in execution state

cbc5873

refactor BatchRemove

936f38b

add comments for BatchRemove

7ad596e

Merge branch 'leo/refactor-insert-chunk-data-pack' into leo/refactor-…

e875957

…stored-chunk-data-pack

use two databases for chunk data pack tests

044325d

add test cases for BatchRemove

d8e884d

update StoreChunkDataPack.Equals

24744c5

fix pruner tests

9d287d6

update mocks

be6f1ce

fix tests

5e0e6a1

update comments

c37af9c

update tests

7fa1ef6

Base automatically changed from leo/refactor-insert-chunk-data-pack to master October 1, 2025 00:14

Merge branch 'master' into leo/refactor-stored-chunk-data-pack

d22661a

zhangchiqing force-pushed the leo/refactor-stored-chunk-data-pack branch from e3a3b6b to d22661a Compare October 1, 2025 00:22

zhangchiqing added 2 commits October 1, 2025 21:40

Merge branch 'master' into leo/refactor-stored-chunk-data-pack

58cd332

fix integration tests

8dbfdc1

zhangchiqing added 3 commits October 2, 2025 12:04

fix tests

9d7cddd

fix tests

792c69f

Merge branch 'master' into leo/refactor-stored-chunk-data-pack

d7a2241

zhangchiqing commented Oct 2, 2025

View reviewed changes

Merge branch 'master' into leo/refactor-stored-chunk-data-pack

506dd4c

zhangchiqing commented Oct 3, 2025

View reviewed changes

zhangchiqing added 2 commits October 2, 2025 17:08

revert tests

64a1cbd

update comments

4a4f529

zhangchiqing marked this pull request as ready for review October 3, 2025 00:46

zhangchiqing requested a review from a team as a code owner October 3, 2025 00:46

zhangchiqing requested review from jordanschalm and AlexHentschel October 3, 2025 00:46

AlexHentschel added 5 commits October 7, 2025 16:50

minor documentation extensions.

af1d4da

minor punctuation improvements

6389c09

documented that non-existent keys are no-ops for remove operations

944e733

clarified documentation

f8abc95

minor polishing of error documentation

bd59da0

AlexHentschel reviewed Oct 8, 2025

View reviewed changes

jordanschalm reviewed Oct 9, 2025

View reviewed changes

	// It is concurrent-safe
	// It is concurrent safe because chunk data packs store is conflict-free (storing data by hash), and protocol data requires a lock to store, which will be synchronized.

	// ID returns a collision-resistant hash of the ChunkDataPack struct.
	func (c *ChunkDataPack) ID() Identifier {
	return MakeID(c)
	}

-func ToStoredChunkDataPacks(cs []*flow.ChunkDataPack) []*StoredChunkDataPack {
+func ToStoredChunkDataPacks(cs []*flow.ChunkDataPack) []*StoredChunkDataPack {// ToStoredChunkDataPack converts the given ChunkDataPacks to their reduced representation,
+// by replacing the full collection data with its ID (collision-resistant hash).
+func ToStoredChunkDataPack(c *flow.ChunkDataPack) *StoredChunkDataPack {

	// Then retrieve the actual stored chunk data pack using the ID
	// Then retrieve the reduced representation of the chunk data pack via its ID.

	withStore(operation.InsertStoredChunkDataPack),
	withStore(operation.InsertStoredChunkDataPack),
	withRemove[flow.Identifier, *storage.StoredChunkDataPack](operation.RemoveStoredChunkDataPack),

	err := chunks.BatchRemove(chunkIDs, writeBatch, chunkBatch)
	err := chunks.BatchRemove(chunkIDs, protocolDBatch, chunkDBBatch)

	type RequestQualifierFunc func(attempts uint64, lastRequested time.Time, retryAfter time.Duration) (bool, string)
	type RequestQualifierFunc func(attempts uint64, lastRequested time.Time, retryAfter time.Duration) (requestQualifies bool, reasonMsg string)

	func (e *Engine) canDispatchRequest(chunkID flow.Identifier) (bool, string) {
	func (e *Engine) canDispatchRequest(chunkID flow.Identifier) (canDispatch bool, reasonMsg string) {

		ResourceChunkDataPack = "chunk_data_pack" // execution node
		ResourceStoredChunkDataPack = "chunk_data_pack_stored" // execution node

		// 1. First phase: Store chunk data packs (StoredChunkDataPack) by its hash (storedChunkDataPackID) in chunk data pack database.
		// 2. Second phase: Create index mappings from ChunkID to storedChunkDataPackID in protocol database

	// The batch update writes all execution result data in a single atomic operation.
	// The batch update writes all execution result data (except chunk data pack!) atomically.

[Storage] Refactor stored chunk data pack #7983

Are you sure you want to change the base?

[Storage] Refactor stored chunk data pack #7983

Uh oh!

Conversation

zhangchiqing commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov-commenter commented Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zhangchiqing Oct 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AlexHentschel left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

⚠️ ❓

Uh oh!

AlexHentschel Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

⚠️

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

zhangchiqing commented Sep 30, 2025 •

edited

Loading

codecov-commenter commented Oct 1, 2025 •

edited

Loading

zhangchiqing Oct 3, 2025 •

edited

Loading

AlexHentschel Oct 8, 2025 •

edited

Loading