Memory Bundle Format Compatibility

Bundle Format Compatibility

Spector stores each partition and each namespace runtime in a single mmap'd bundle file. This page states which bundle format versions a release reads and writes, and what happens when it meets one it does not recognise.

Current support

Release	Writes	Reads	Refuses
current main	1	1	anything outside [1, 1]

The range is declared in code, not in documentation, by two constants in BundleFileLayout:

public static final int SCHEMA_VERSION = 1;              // the version this binary writes
public static final int MIN_READABLE_SCHEMA_VERSION = 1;  // the oldest it will read

BundleDirectory.read accepts a bundle only when its recorded version falls in the closed interval [MIN_READABLE_SCHEMA_VERSION, SCHEMA_VERSION]. The table above is generated by reading those constants, so it cannot drift from the code without a test failing.

What happens on a version it cannot read

The open is refused, with a SpectorStorageException carrying FILE_FORMAT_INVALID and a message naming the file's version, the readable range, and the remedy:

Situation	Message says
Version above the range	The bundle was written by a newer Spector; upgrade this binary to read it.
Version below the floor	The bundle predates the oldest supported format; migrate it with an older release first.
The two recorded copies disagree	Refuses rather than guessing which copy is authoritative.
Partition bundle opened as runtime, or vice versa	Names both roles, so it is clear which file was pointed at what.

Why refusal rather than a best effort

The bundle directory records where every region begins. Misreading it does not produce a clean error at the point of the mistake — it produces records read from the wrong offsets, which surfaces much later as corrupt data with no obvious cause.

There is a second, sharper reason. A bundle is opened READ_WRITE, and close() rewrites the directory with the current version. Before this gate existed, an unfamiliar bundle was not merely accepted: it was relabelled as the current version on close, destroying the only evidence that another binary had written it. The next open then saw a well-formed, current-version file whose regions had been laid out by a different writer.

This matches the discipline already applied elsewhere in the kernel — HebbianGraphMemory throws on an unrecognised magic rather than starting fresh, and RegionPreamble.readShape throws on an unknown shape ordinal.

Versions are a flat integer, not major/minor

Every on-disk version in the kernel is a single int. There is deliberately no major/minor split in any binary header: introducing one would mean either reinterpreting bytes already written or claiming reserved space, for no gain over a declared readable range.

note: Not to be confused with SchemaVersion

`SchemaVersion` — which does have major, minor and patch — versions the **namespace directory layout** and
is persisted as a JSON string (`{"schemaVersion": "2.0.0"}`). It never appears in a bundle header. The two
version schemes are independent.

Where the version is recorded

A bundle stores its format version twice, and both copies are checked:

Location	Offset	Written by
Bundle RegionPreamble	file offset 0, field at +4	BundleDirectory.write
BundleSubHeader	file offset 64, field at +4	BundleDirectory.write

They are written together, so a disagreement means one was rewritten independently — corruption worth refusing rather than silently resolving.

Region versions are now validated (resolved in #1015)

Beyond the bundle-level version, each region carries its own version, also stored twice — once in its RegionEntry in the directory and once in the region's own preamble. A partition bundle has five such regions (SEMANTIC, EPISODIC, PROCEDURAL, TEXT, STRENGTH); runtime bundles have more.

As of PR #1015, RegionVersionRegistry.validateRegionVersions() is called during BundleDirectory.read() to validate that each region slice's preamble schema version agrees with its directory entry and is at least 1. Regions whose preambles use non-SMKM magic (e.g., graph regions with custom headers) are skipped rather than rejected, preserving backward compatibility.

When the first layout version bump occurs, RegionVersionRegistry should be extended with explicit per-RegionId acceptable version ranges (currently all layouts declare schemaVersion() == 1).

The V3 → V4 migration

V4 bundles replaced the earlier layout of one file per region. That migration still works, and is detected by directory shape, not by any version field:

Condition	Meaning
partition.bundle present	Already V4; nothing to do
partition.bundle absent and any of semantic.mem, episodic.mem, procedural.mem, text.dat present	V3 store; migrate
Neither present	Empty partition; nothing to do

PartitionManager.openFrozenPartition invokes BundleMigrationCli.migratePartition reflectively — the reflection exists only so spector-memory needs no dependency on spector-cli. Migration copies each region into a new bundle, verifies record counts, and renames the originals with a .v3bak suffix rather than deleting them.

Because a V3 store has no partition.bundle at all, it never reaches the bundle open path, and the version gate cannot affect the migration.

Verified on both architectures

The frozen-bundle fixtures in BundleFormatCompatibilityTest are real bundle files checked into the repository and opened on every CI run, on x86_64 and aarch64. A fixture written on one architecture and opened on the same architecture would prove nothing about byte order or alignment; opening it on both is what makes it a compatibility test.