[Protocol Versioning] Document versioning mechanics for dynamic protocol state

[AlexHentschel]

Overarching Product Goal

This issue is part of Epic #6999. The overall goal is to evolve Versioning of Execution Stack to use the Dynamic Protocol State, and subsequently employ this versioning information for

1. coordinating EN HCUs (see Flip 298 for more context on product relevance and impact)
2. the ANs to supports script execution across breaking HCU version boundaries (see OKR description for more context on product relevance and impact)

This issue is part of 1.

Description and Details for this issue

We want to describe the conceptual information flow and processing of the new versioning information for the Execution Stack

Governance Transaction 
   → EN Version Beacon Smart Contract (state-full component) 
        → EN Version Beacon Service Event 
             → Dynamic Protocol State (state-full component)

We have the potentially state-full components Version Beacon Smart Contract and Dynamic Protocol State. The Governance Transaction and EN Version Beacon Service Event can be viewed as messages carrying information that is ingested by these components and may induce an update of their states.

Definition of Done

Notion doc describing

1. the information that EN Version Beacon Smart Contract should locally maintain
2. Information that Governance Transaction caries
3. for a Governance Transaction arriving at the EN Version Beacon Smart Contract:

   1. The sanity checks that that the Version Beacon Smart Contract should apply to the Governance Transaction to decide whether to accept or reject the Governance Transaction.

      Caution: it is important to consider the possibilities of mistakes, inconsistencies, outdated or repeated governance transactions, as those are prepared and submitted by humans.

   2. for a Governance Transaction that is accepted: how does the transaction update the internal state of the Version Beacon Smart Contract?

   3. the conditions under which an EN Version Beacon Service Event is emitted by the smart contract

   4. Description of how the EN Version Beacon Smart Contract computes the data to be emitted in the EN Version Beacon Service Event.

4. for a EN Version Beacon Service Event (content specified in 3.iv) arriving at the Dynamic Protocol State:

   1. The sanity checks that that the Dynamic Protocol State should apply to the EN Version Beacon Service Event to decide whether to accept or reject the service event.

      Caution: it is important to consider the possibilities of outdated service events, bugs in the smart contract or accidental inconsistent upgrades of the EN Version Beacon Smart Contract. Our sanity checks here don't have to be entirely failsafe, but checking for plausible error cases (especially outdated service events) should be done where possible with reasonable effort.

   2. for an EN Version Beacon Service Event that is accepted: how does the service event update the internal state of the Dynamic Protocol State?

5. simple API sketch of what information should be made available by the Dynamic Protocol State to be queried by the StopControl logic in the Execution Node

Important:
In a nutshell, we (as humans) are providing a directive to the protocol to change Execution Behaviour at some point in the future. These transition points from the "old" to the "new behaviour" must be specified as a view threshold. The behaviour change takes place when the view threshold is reached or exceeded. For more details, please see Flip 298.

Proposal

The focus of this work should be on utilizing the Dynamic Protocol State for versioning the Execution Stack. The specific format on how we express the version is secondary, and probably somewhat easy to change in the future.

Therefore, I would propose to use major.minor for versioning the execution stack, because

• this has the smallest change surface compared to the existing implementation (👉 current VersionBeacon implementation)
• conceptually consistent with out intend to versioning protocol behaviour by removing the patch field (see Flip 298 for further details)

[janezpodhostnik]

Recently I was battling with the programs cache invalidation problem. The TLDR verions of the problem is that the FVM is reading metering settings (execution effort weights, execution memory weights, memory limit) from the state which is then put in the cache. Metering settings are also a dependency for everything else in the cache. Because there was unrelated data in the same registers as the metering settings that kept changing on every block, we had to invalidate the whole cache.

This got me thinking that the metering settings should probably also be in the dynamic protocol state, which would simplify a lot of the code.

The execution parameters that would be in the dynamic protocol state would then look like this.

type ExecutionParameters struct {
  ExecutionVersion string // perhaps instead of string this would be some sort of shortened semver
  ExecutionEffortWeights map[uint]uint64
  MemoryWeights map[uint]uint64
  MemoryLimit uint64
}

Are there any problems with this?

[AlexHentschel]

@janezpodhostnik

metering settings should probably also be in the dynamic protocol state

That makes sense to me. We kind of have a circular dependency if some of the parameters for the execution state are stored in the execution state itself. So putting the parameters in the dynamic protocol state is a nice way to solve this problem, because the dynamic protocol state lives on the foundational level of the protocol.

No problems at all with this. Its going to be a little bit of work, but probably nothing too complicated. I am very happy that the protocol state's new ability to house arbitrary parameters comes in handy.

Tagging @jordanschalm @durkmurder and @tim-barry for visibility here, since we'll need their help to implement that and do a protocol-level upgrade to deploy the changes to mainnet.

[j1010001]

notion doc: Execution State Parameters in the Dynamic Protocol State

[AlexHentschel]

Jordan has created a very early draft PR, illustrating how the execution parameters could be represented in the protocol state: PR #7020