SIMD-0319: Remove Accounts `is_executable` Flag Entirely #319

Lichtso · 2025-07-10T14:39:15Z

No description provided.

jstarry · 2025-07-31T01:31:22Z

proposals/0319-remove-executable-flag-entirely.md

+Program data accounts (owned by loader-v3) must not be filtered by their
+`is_executable` flag anymore.
+
+### Account hash


If is_executable is not part of the accounts hash anymore, we shouldn't be setting the value on deploys in loader v3 or v4, new core programs/precompiles, or when migrating core programs to bpf, right?

It still is: https://github.com/anza-xyz/agave/blob/bca4a36eaa8b5ac64ac5ee998b643d500864b0c0/accounts-db/src/accounts_db.rs#L5110

I don't understand your response. Are we not removing the flag from the account hash? You wrote this below:

The flag must not be added to the input of the hash function anymore.

Ah sorry, misunderstood you initial comment. You were talking about the post SIMD state, not the pre SIMD state.
And yes, we should not be setting that anymore. I can add it to the SIMD.

jstarry · 2025-07-31T01:33:11Z

proposals/0319-remove-executable-flag-entirely.md

+- an account is owned by loader-v1 or loader-v2
+- an account is owned by loader-v3 and contains the program (proxy) state
+- an account is owned by loader-v4 and has the deployed or finalized status


Should have a case for when an account is owned by the native loader as well

Do you mean for the loader programs? Because surely not every zero filled account owned by the native loader should be marked as executable.

I mean I don't really think so either but if we're removing the executable flag we kind of have to, no? Doesn't the same thing apply to loader-v1 and loader-v2? Any account can have those loaders assigned as their owner

Yes, but I really want to avoid having to do ELF loading, parsing and verification for this. As I said in the loader-v4 SIMD, the easiest and most consistent thing to do is to only check the owner and not the contents. If we check the contents, we can only do useful thing in loader-v3 and v4. Also, keep in mind that the only purpose of this here is not to break existing programs on MNB, otherwise I would just serialize the is_executable flag as always false.

I'm not really following your reply. How is adding native loader to this list of checks any different from having loader-v1 and loader-v2 in the list? We don't check the contents for those either.

I can add it, it is just a question of what we want to achieve with this. Like I said, my only goal here was not to break existing programs on MNB. I can re-run the validator replay and see if anything changed, but I think nobody cared about the executable flag of loaders.

I'm typically just coming from the perspective of having consistency. In other places we consider an account to be executable if it's owned by one of the 5 loaders and I noticed you didn't include the native loader in the checks here. I don't see any drawback from including native loader here. Your concern about anyone assigning the owner to the native loader also applies to loader v1 or v2 so I don't think it's actually a concern.

jstarry · 2025-07-31T01:39:51Z

proposals/0319-remove-executable-flag-entirely.md

+
+None.
+
+## Impact


The CU usage for CPIs that pass executable accounts will increase as well, potentially by quite a lot if an executable account size is quite large (ie the token program). Maybe we don't need to charge CU's if direct mapping is enabled? Note that I'm referring to consume_compute_meter in CallerAccount::from_account_info.

There should be no difference, since the is_executable == true path consumes:
https://github.com/anza-xyz/agave/blob/master/syscalls/src/cpi.rs#L809C9-L817C10

the same as the is_executable == false path:
https://github.com/anza-xyz/agave/blob/master/syscalls/src/cpi.rs#L838C17-L847C20
https://github.com/anza-xyz/agave/blob/master/syscalls/src/cpi.rs#L168C13-L173C16
https://github.com/anza-xyz/agave/blob/master/syscalls/src/cpi.rs#L273C9-L279C12

Oh thanks for pointing that out. If we are going to remove the special case for executable accounts in CPI then we should probably remove that is_executable == true cu consumption

Well, that is what removing the special case entails, it follows the normal path. Or were you suggesting we change the special case so that program accounts don't charge CUs?

Ah sorry I misunderstood the code, yeah I think this won't be impactful given that we were already charging CU's for executable accounts. Thanks!

Not necessary but do you mind adding some more detail to the CPI special case section explaining that as before, executable accounts will be charged CU's according to length, but will now have their changes copied into and out of the SVM just like other accounts?

jstarry · 2025-07-31T01:45:57Z

proposals/0319-remove-executable-flag-entirely.md

+
+### Account hash
+
+The flag must not be added to the input of the hash function anymore.


cc @brooksprumo to comment on how this migration will work when the feature for this SIMD is activated at an epoch boundary.

This is actually a very big change. As long as AccountSharedData has the executable field, we need to populate it.

In order to change the algorithm for calculating an account's hash (i.e. removing executable from the account hash), then we need to recalculate the accounts lt hash. There are a few options here:

Don't actually remove executable from the account hash.

Do a one-time re-write of the accounts lt hash at the epoch boundary. This will take 2-5 minutes. I think that's a non-starter.

Add a second accounts lt hash with the new algo, and then at feature activation swap the one used in the bank hash. This is doable, but kind of a pain to do double work; also it's much harder to check/ensure validity of the new accounts lt hash until the feature activation. We did something like this for the first accounts lt hash too, I wouldn't recommend it.

Add a field to AccountSharedData that indicates the hash algo version used to last calculate its hash. This way we don't need to do double hashing. Note, we wouldn't add new fields, we'd just steal a byte from some other now-obsolete field.

Related to (1), update all the accounts currently with executable == true to false. Then once we know all the accounts have false for executable, we can hard code that into the hash calc and remove reading the field, thus allowing us to remove the field afterwards.

My thought is that removing the executable field from AccountSharedData first will make all other changes simpler. However, we do need executable out of AccountHash (now known as AccountLtHash) first. I think (5) may be the simplest option, as (3) and (4) are quite involved. I do think we need (4) eventually though...

jstarry · 2025-07-31T01:46:52Z

proposals/0319-remove-executable-flag-entirely.md

+The flag must not be added to the input of the hash function anymore.
+Note that this is different from hashing it as always `false`.
+
+### VM serialization


Might want to expand this section to cover all potential reads of the executable flag including geyser and rpc

jstarry · 2025-07-31T01:47:33Z

proposals/0319-remove-executable-flag-entirely.md

+### Snapshot minimization special case
+
+Program data accounts (owned by loader-v3) must not be filtered by their
+`is_executable` flag anymore.


IMO this is an implementation detail of agave. Not necessary to include in this SIMD as it's not relevant to other validator implementations.

jstarry · 2025-07-31T01:54:35Z

proposals/0319-remove-executable-flag-entirely.md

+
+None.
+
+## Detailed Design


There's some code that adds recreates precompile accounts at startup and at epoch boundaries if the precompile account doesn't have the executable flag set. This should be detailed too (in add_precompiled_account_with_owner in agave)

jstarry · 2025-07-31T01:56:52Z

proposals/0319-remove-executable-flag-entirely.md

+
+None.
+
+## Detailed Design


There's also some code that verifies that the target bpf program account is executable in the core to bpf migration code path. This should be detailed as well.

Good point. I think it is easiest to make this SIMD dependent on all of the core programs to be migrated first.

Yeah, that's a good point. From chatting with @buffalojoec it sounds like all the builtins that we are planning to migrate have already been migrated. So feel free to leave out the migration code path details.

There are others we hopefully plan to migrate, but their timelines are unknown.

Vote program should be replaced as part of the Alpenglow upgrade

Loader v4 requires a lot of ideating around ELF verification and program cache updates

ZK-El-Gamal is still in active development

brooksprumo · 2025-08-04T13:55:25Z

proposals/0319-remove-executable-flag-entirely.md

+
+See SIMD-0162 for the reasons why the `is_executable` flag is unnecessary
+protocol complexity. That SIMD however only removes all checks of the flag
+which can abort a transaction. There are other influences the flag has on


nit: grammar

Suggested change

which can abort a transaction. There are other influences the flag has on

that can abort a transaction. There are other influences the flag has on

brooksprumo · 2025-08-04T13:56:24Z

proposals/0319-remove-executable-flag-entirely.md

+See SIMD-0162 for the reasons why the `is_executable` flag is unnecessary
+protocol complexity. That SIMD however only removes all checks of the flag
+which can abort a transaction. There are other influences the flag has on
+consensus which shall be removed as well.


nit: grammar

Suggested change

consensus which shall be removed as well.

consensus, which shall be removed as well.

brooksprumo · 2025-08-04T14:00:01Z

proposals/0319-remove-executable-flag-entirely.md

+### Account append vector in snapshot format
+
+When loading existing snapshots the flag must be ignored. When storing
+snapshots the flag should be treated as being always `false`.


Snapshots are a subset of account storage files in general. We would want to make these changes regardless if a node is generating snapshots or not. Similarly, we'd want this applied to all account storage formats, not just append vecs.

Suggested change

### Account append vector in snapshot format

When loading existing snapshots the flag must be ignored. When storing

snapshots the flag should be treated as being always `false`.

### Account storage file format

When loading accounts the flag must be ignored. When storing

accounts the flag should be treated as being always `false`.

Note, as long as Account and AccountSharedData have the executable field, we must always populate it. IOW, on load, we'll also need to set the flag to false.

brooksprumo · 2025-08-04T14:15:36Z

proposals/0319-remove-executable-flag-entirely.md

+
+### Account hash
+
+The flag must not be added to the input of the hash function anymore.


This is actually a very big change. As long as AccountSharedData has the executable field, we need to populate it.

In order to change the algorithm for calculating an account's hash (i.e. removing executable from the account hash), then we need to recalculate the accounts lt hash. There are a few options here:

Don't actually remove executable from the account hash.

Do a one-time re-write of the accounts lt hash at the epoch boundary. This will take 2-5 minutes. I think that's a non-starter.

Add a second accounts lt hash with the new algo, and then at feature activation swap the one used in the bank hash. This is doable, but kind of a pain to do double work; also it's much harder to check/ensure validity of the new accounts lt hash until the feature activation. We did something like this for the first accounts lt hash too, I wouldn't recommend it.

Add a field to AccountSharedData that indicates the hash algo version used to last calculate its hash. This way we don't need to do double hashing. Note, we wouldn't add new fields, we'd just steal a byte from some other now-obsolete field.

Related to (1), update all the accounts currently with executable == true to false. Then once we know all the accounts have false for executable, we can hard code that into the hash calc and remove reading the field, thus allowing us to remove the field afterwards.

My thought is that removing the executable field from AccountSharedData first will make all other changes simpler. However, we do need executable out of AccountHash (now known as AccountLtHash) first. I think (5) may be the simplest option, as (3) and (4) are quite involved. I do think we need (4) eventually though...

LucasSte · 2025-08-11T15:32:25Z

proposals/0319-remove-executable-flag-entirely.md

+Currently CPI ignores changes made by the caller to instruction accounts which
+have the flag set, meaning even requesting write access to a program account
+throws no error. Instead the flag must now be ignored, meaning all changes made
+by the caller to instruction accounts are treated equally, regardless of the
+`is_executable` flag.


What if the callee program modifies the caller program account?

LucasSte · 2025-08-11T15:39:37Z

proposals/0319-remove-executable-flag-entirely.md

+`InstructionError::MissingAccount` anymore but instead not try to update the
+account payload length field (as that is missing if there is no `AccountInfo`).


What do you mean by "not try to update"?

Lichtso force-pushed the remove-executable-flag-entirely branch from 851726a to 01d3214 Compare July 10, 2025 14:39

Lichtso changed the title ~~Remove Accounts is_executable Flag Entirely~~ SIMD-0319: Remove Accounts is_executable Flag Entirely Jul 10, 2025

Lichtso force-pushed the remove-executable-flag-entirely branch from 01d3214 to 715ab14 Compare July 10, 2025 14:43

github-actions bot mentioned this pull request Jul 14, 2025

Upstream Updates - Mon Jul 14 00:18:27 UTC 2025 smartcontractkit/chainlink-solana#1298

Closed

jstarry reviewed Jul 31, 2025

View reviewed changes

brooksprumo reviewed Aug 4, 2025

View reviewed changes

Initial draft.

b8aa4f1

Lichtso force-pushed the remove-executable-flag-entirely branch from 715ab14 to b8aa4f1 Compare August 4, 2025 17:48

LucasSte reviewed Aug 11, 2025

View reviewed changes

brooksprumo mentioned this pull request Oct 20, 2025

SIMD-0329: Track Last Accessed Slot for Accounts #329

Open


		### Account hash

		The flag must not be added to the input of the hash function anymore.

	which can abort a transaction. There are other influences the flag has on
	that can abort a transaction. There are other influences the flag has on

	consensus which shall be removed as well.
	consensus, which shall be removed as well.

		`InstructionError::MissingAccount` anymore but instead not try to update the
		account payload length field (as that is missing if there is no `AccountInfo`).


		None.

		## Impact

SIMD-0319: Remove Accounts is_executable Flag Entirely #319

Are you sure you want to change the base?

SIMD-0319: Remove Accounts is_executable Flag Entirely #319

Uh oh!

Conversation

Lichtso commented Jul 10, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

buffalojoec Aug 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

SIMD-0319: Remove Accounts `is_executable` Flag Entirely #319

SIMD-0319: Remove Accounts `is_executable` Flag Entirely #319

buffalojoec Aug 1, 2025 •

edited

Loading