proposal: NeoFS block storage format

Signed-off-by: Ekaterina Pavlova <[email protected]>

Author: AliceInHunterland

neofs-blocks.mediawiki

@@ -0,0 +1,104 @@
+<pre>
+  NEP: TBD
+  Title: NeoFS block storage format
+  Author: Ekaterina Pavlova <[email protected]>
+  Type: Informational
+  Status: Draft
+  Created: 2025-04-07
+</pre>
+
+==Abstract==
+
+This proposal outlines the specification for storing Neo blockchain blocks within
+NeoFS container.
+
+==Motivation==
+
+Neo node synchronization via P2P requires all headers and blocks (~700 bytes each, over 1 GB per year with
+15-second blocks) to remain available, limiting scalability as chains grow. Storing blocks in NeoFS
+provides a distributed alternative, reducing local storage, enabling on-demand fetches and allowing to drop
+blocks and headers from storage for most nodes.
+
+==Specification==
+
+===Block Storage Schema===
+
+A single NeoFS container is used to store blocks of a single Neo network. Each container includes a fixed
+metadata attribute named `Magic` whose value is the network’s magic number in decimal representation. Each
+block is stored in a serialized representation following standard Neo block serialization rules as a
+separate NeoFS object with a unique OID and the following metadata attributes:
+
+{| class="wikitable"
+! Attribute name !! Attribute value
+|-
+| Block || String representation of the block index in base 10.
+|-
+| Hash || Hexadecimal string of the block’s hash.
+|-
+| PrevHash || Hexadecimal string of the previous block’s hash.
+|-
+| BlockTime || Millisecond-precision timestamp when the block was created.
+|-
+| Timestamp || Second-precision timestamp when the block was uploaded.
+|}
+
+These attributes are stored as NeoFS object metadata; the list of attributes MAY be extended.
+The object MAY include additional data after the block.
+
+===Synchronization Algorithm===
+
+Blocks stored in NeoFS and those synchronized via P2P both use the standard Neo block
+serialization format, ensuring compatibility across different synchronization methods.
+The NeoFS container serves as verified storage for downloading blocks.
+[https://github.com/nspcc-dev/neofs-api/blob/b7548cfb16426256f0a92c6d70a3ec794a6facb0/object/service.proto#L170 NeoFS SearchV2 API]
+should be used to fetch ordered chunks of block OIDs by Block attribute. Additional
+GE/GT/LE/LT SearchV2 filters may be applied to the Block attribute value to filter
+out required range of blocks.
+[https://github.com/nspcc-dev/neofs-api/blob/b7548cfb16426256f0a92c6d70a3ec794a6facb0/object/service.proto#L48 NeoFS Get API]
+should be used to fetch blocks by retrieved OIDs.
+
+Downloaded blocks are inserted into the blockchain using the same logic as in the
+P2P synchronization protocol.
+
+==Rationale==
+
+Storing each block as a self-contained NeoFS object moves historical data off full nodes
+while preserving byte-exact compatibility with the P2P format. Standard metadata makes
+blocks easily discoverable without custom indices, and NeoFS gives durability and fast
+parallel fetches. The proposed scheme is implemented as an experimental extension of NeoGo
+node ([https://github.com/nspcc-dev/neo-go/issues/3496 nspcc-dev/neo-go#3496]) and
+proved to be efficient.
+
+== References ==
+
+* [https://github.com/neo-project/neo/issues/3463 Using NeoFS to store blocks and snapshots]
+
+== Test cases ==
+
+An example of a [https://rest.fs.neo.org/v1/objects/3RCdP3ZubyKyo8qFeo7EJPryidTZaGCMdUjqFJaaEKBV/by_id/Fu7yQzspvLJwSGJNK64xeeyMdWXtU5B5b1es6KSxUag1 block object]:
+<pre>
+ID: Fu7yQzspvLJwSGJNK64xeeyMdWXtU5B5b1es6KSxUag1
+CID: 3RCdP3ZubyKyo8qFeo7EJPryidTZaGCMdUjqFJaaEKBV
+Owner: NVvY1FF67XJ2GTVhy9FqiZGC4jEQtvjmHt
+CreatedAt: 28202
+Size: 697
+HomoHash: 45c98e627910d9b915d58368493789ce49ca194626f16ea5bf6b57bb2cce462921a1d3faf682d252a804b0555ca48905286222ee9209b3ff1ce4677a082ffd4d
+Checksum: fa6cedddfec3a61157c4a12e25f81e85c0f92aac70317d6df7fe193f983b4917
+Type: REGULAR
+Attributes:
+  Block=1
+  Primary=0
+  Hash=5f3fbb43d1e516fe07771e2e873ebc9e2810662401acf603a775aace486220bd
+  PrevHash=1f4d1defa46faa5e7b9b8d3f79a06bec777d7c26c4aa5f6f5899a291daa87c15
+  BlockTime=1627894840919
+  Timestamp=1734362616 (2024-12-16 18:23:36 +0300 MSK)
+ID signature:
+  public key: 02a4920745d86db224c179c936606dc0e4620edad13d568ef036da279352e45f2b
+  signature: 0443ff20d15952759b5101a7d223d70eb992260fd9ad5aecb404a97448b2ea54bd1ad783b12ccddba0097dd6608b55ccac9215c5715f9589ec6a555542ead6dc00
+[Block Binary Data]
+</pre>
+
+== Implementation ==
+
+* Go: [https://github.com/nspcc-dev/neo-go/blob/965ae2746cd1a01aba3b4114e1ea26ca7349f34b/cli/util/upload_bin.go utility tool that uploads blocks to NeoFS container]
+* Go: [https://github.com/nspcc-dev/neo-go/blob/965ae2746cd1a01aba3b4114e1ea26ca7349f34b/pkg/services/blockfetcher/blockfetcher.go node service that fetches blocks from NeoFS container as an alternative to P2P synchronization]