RPC calls don't work if the requested `version` is the latest finalized block

[zizou0x]

Description

When calling the /protocol_state or /contract_state endpoint for the latest finalized block, it returns 404 Error: Not Found. However, the endpoint works as expected for the blocks immediately before and after the latest finalized block.

Example

Given current finalized block: 21392273

With this request:

curl -X 'POST' \
  'http://0.0.0.0:4242/v1/contract_state' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "chain": "ethereum",
  "pagination": {
    "page": 0,
    "page_size": 2
  },
  "protocol_system": "vm:balancer_v2",
  "version": {
    "block": {
      "chain": "ethereum",
      "number": 21392273
    }
  }
}'

• Behavior:
  • /contract_state for block 21392272 → ✅ Works
  • /contract_state for block 21392274 → ✅ Works
  • /contract_state for block 21392273 → ❌ 404 Error: Not Found

And same behavior for /protocol_state.

On the server side, this log appears:

ERROR get_contract_state: Error while getting contract states. error=Could not find Block with id `21392273`!

---

Expected Behavior

/protocol_state and /contract_state endpoints should return the state for the latest finalized block.

---

Actual Behavior

The endpoint return 404 Error: Not Found specifically for the latest finalized block.

[kayibal]

This is most likely an issue with PendingDeltasBuffer src and or calculate_versions src. Seems like the finalized block is not ingested yet in the database. Potentially we have a lag of 1 block somewhere on accident.

[royvardhan]

@zizou0x Can you guide me on how to reproduce this? Currently setting up everything.

Tried running the commands at README, says Missing authorization token

[zach030]

Hi! I attempted to run the indexer locally using the following command:

cargo run --bin tycho-indexer -- run --substreams-api-token {api_token} \  
--spkg ./tycho-indexer/test/spkg/substreams-ethereum-quickstart-v1.0.0.spkg \  
--module map_block --protocol-type-names test_eth --start-block 12292922 \  
--stop-block 12292932 --rpc-url {rpc_url}  

I used the test spkg, module, start-block, and stop-block parameters provided in the substreams example repository:
https://github.com/streamingfast/substreams-ethereum-quickstart/blob/main/Makefile

Here are the logs after running the command:

2024-12-24T06:03:43.142132Z  INFO Starting Tycho
2024-12-24T06:03:43.864471Z  INFO Upgrading database...
2024-12-24T06:03:44.468168Z  INFO DBCacheWriteExecutor started! name="ethereum"
2024-12-24T06:03:44.468895Z  INFO Building protocol cache
2024-12-24T06:03:44.481219Z  INFO populate: ProtocolCachePopulated n_tokens=0 n_components=0 n_prices=0
2024-12-24T06:03:44.483767Z  WARN No cursor found, starting from the beginning name="test_protocol" chain=Ethereum
2024-12-24T06:03:44.622185Z  INFO Extractor ethereum:test_protocol started!
2024-12-24T06:03:44.622950Z  INFO Starting full server
2024-12-24T06:03:44.628412Z  INFO extractor:subscribe: New subscription subscriber_id=0 extractor_id=ethereum:test_protocol
2024-12-24T06:03:44.636313Z  INFO starting 8 workers
2024-12-24T06:03:44.636439Z  INFO Http and Ws server started server_url="http://0.0.0.0:4242"
2024-12-24T06:03:44.636589Z  INFO Tokio runtime found; starting in existing Tokio runtime
2024-12-24T06:03:48.575082Z  INFO extractor: SubstreamSessionInit session.resolved_start_block=12292922 session.linear_handoff_block=12293000 session.max_parallel_workers=10 session.trace_id="442fcce5e91492a21f79d7d34199d9e6" extractor_id=ethereum:test_protocol sf_trace_id="442fcce5e91492a21f79d7d34199d9e6"
2024-12-24T06:03:48.688227Z  INFO extractor:handle_tick_scoped_data: Received message data.type_url="type.googleapis.com/acme.block_meta.v1.BlockMeta" extractor_id=ethereum:test_protocol sf_trace_id="442fcce5e91492a21f79d7d34199d9e6" block_number=12292922
2024-12-24T06:03:48.688286Z ERROR extractor: Error while processing tick! error=Failed to decode: Unknown message type extractor_id=ethereum:test_protocol sf_trace_id="442fcce5e91492a21f79d7d34199d9e6" block_number=12292922
thread 'main' panicked at tycho-indexer/src/main.rs:83:67:
called `Result::unwrap()` on an `Err` value: DecodeError("Unknown message type")
stack backtrace:
   0: _rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::result::unwrap_failed
   3: tycho_indexer::main
   4: core::ops::function::FnOnce::call_once

this log entry:

2024-12-24T06:03:48.688227Z  INFO extractor:handle_tick_scoped_data: Received message data.type_url="type.googleapis.com/acme.block_meta.v1.BlockMeta" ...  

is generated from a debug print I added in the handle_tick_scoped_data function. It seems that the received data ends with BlockMeta and cannot be processed correctly inside this function.

Relevant code:

tycho-indexer/tycho-indexer/src/extractor/protocol_extractor.rs

Lines 514 to 556 in 9b80145

	let msg = match data.type_url.as_str() {
	url if url.ends_with("BlockChanges") => {
	let raw_msg = pb::tycho::evm::v1::BlockChanges::decode(data.value.as_slice())?;
	trace!(?raw_msg, "Received BlockChanges message");
	BlockChanges::try_from_message((
	raw_msg,
	&self.name,
	self.chain,
	&self.protocol_system,
	&self.protocol_types,
	inp.final_block_height,
	))
	}
	url if url.ends_with("BlockContractChanges") => {
	let raw_msg =
	pb::tycho::evm::v1::BlockContractChanges::decode(data.value.as_slice())?;
	trace!(?raw_msg, "Received BlockContractChanges message");
	BlockContractChanges::try_from_message((
	raw_msg,
	&self.name,
	self.chain,
	self.protocol_system.clone(),
	&self.protocol_types,
	inp.final_block_height,
	))
	.map(Into::into)
	}
	url if url.ends_with("BlockEntityChanges") => {
	let raw_msg =
	pb::tycho::evm::v1::BlockEntityChanges::decode(data.value.as_slice())?;
	trace!(?raw_msg, "Received BlockEntityChanges message");
	BlockEntityChanges::try_from_message((
	raw_msg,
	&self.name,
	self.chain,
	&self.protocol_system,
	&self.protocol_types,
	inp.final_block_height,
	))
	.map(Into::into)
	}
	_ => return Err(ExtractionError::DecodeError("Unknown message type".into())),
	};

I’d like to know if there’s a way to successfully run the indexer locally or if you could provide an alternative spkg file.

[royvardhan]

@zizou0x Can you guide me on how to reproduce this? Currently setting up everything.

Tried running the commands at README, says Missing authorization token

Nvm, got the substreams token from thegraph.

[zizou0x]

Hey @zach030 and @royvardhan,

Thanks for being interested in contributing to Tycho! To keep things simple and avoid doubling up on the work, could one of you take this issue? I’ll be adding some more good first issues soon, so no worries for whoever can't have this one.

[zizou0x]

@zach030

This is because ./tycho-indexer/test/spkg/substreams-ethereum-quickstart-v1.0.0.spkg isn’t one of our substreams module (it can only be used to test the stream but the output isn’t recognised by our extractor). Please use one of our Substreams modules here.

I would also advise you to use the index command instead of run. They both work but the index gives you more flexibility and is the main used one.