Aetheria testing by vaclav-ssvlabs · Pull Request #2627 · ssvlabs/ssv

vaclav-ssvlabs · 2025-12-17T09:23:46Z

just testing PR to run aetheria test workflow with local testnet

codecov · 2025-12-17T09:40:43Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 56.2%. Comparing base (db75988) to head (e5e11c1).

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

ssv-test-bot · 2025-12-18T13:13:53Z

🔴 Test run for PR #2627

Branch: vaclav-test-aetheria-local-testnet
Commit: e998ec6

Status

2025-12-18T13:13:52Z Starting test run
2025-12-18T13:13:54Z Test run submitted successfully for commit e998ec6
2025-12-18T13:13:54Z container image for commit e998ec67cd5500a98f271ca5ac5fe5c8d02170f8 is build and present in container registry
2025-12-18T13:13:56Z allocated node range cluster=1,2,3,4 for test run on network=local_testnet
2025-12-18T13:13:57Z deploying ssv-mini with local testnet, this take aproximatelly 8-12 minutes
2025-12-18T13:26:30Z ssv-mini with local testnet is ready
2025-12-18T13:26:32Z started test executor with tests=(sanity) and network=local_testnet

Click to expand executor logs

═══════════════════════════════════════════════════════════════════════════════════════════════════════
                🧪  WELCOME TO Executor - SSV's Distributed Test Automation Execution Layer  🧪            

                                  ___ __  __ ___  ___  _   _  _____  ___   ___ 
                                 | __|\\ \\/ /| __|/ __|| | | ||_   _|/ _ \\ | _ \\
                                 | _|  >  < | _|| (__ | |_| |  | | | (_) ||   /
                                 |___|/_/\\_\\|___|\\___| \\___/   |_|  \\___/ |_|_\\
                                        

═══════════════════════════════════════════════════════════════════════════════════════════════════════
{"time":"2025-12-18T13:26:39.342198189Z","level":"INFO","msg":"🐳 Docker mode: Using environment variables for config generation"}
{"time":"2025-12-18T13:26:39.354898552Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","endpoint":"http://ssv-mini-pr-2627-commit-e998ec6-orchestrator:8080"}
{"time":"2025-12-18T13:26:39.35501014Z","level":"INFO","msg":"📝 Requesting configuration generation from orchestrator API","network":"local_testnet","clusters":[1,2,3,4],"tests":"(sanity)"}
{"time":"2025-12-18T13:26:39.355299415Z","level":"INFO","msg":"⏳ Waiting for orchestrator to generate config (this may take a while)","timeout":"30m0s"}
{"time":"2025-12-18T13:26:43.417376791Z","level":"INFO","msg":"✅ Configuration generated successfully via API","resource_usage_id":"lt-20251218-9a8b"}
2025/12/18 13:26:43 WARN Threshold not provided in config; using local_testnet default threshold=5000000000000000000
{"time":"2025-12-18T13:26:43.418259332Z","level":"INFO","msg":"🚀 EXECUTOR Initialized Successfully"}
{"time":"2025-12-18T13:26:43.418289399Z","level":"INFO","msg":"⚙️ Test configuration loaded","network":"local_testnet","nodes":[1,2,3,4],"test_type":{"Sanity":{"ShouldRun":true,"E2M":false,"Threshold":false,"BulkAmount":0},"Bulk":{"ShouldRun":false,"E2M":false,"Threshold":false,"BulkAmount":0},"Isolated":{"Tests":null}}}
{"time":"2025-12-18T13:26:43.418529863Z","level":"INFO","msg":"Operator pubkeys file not found, generating with go-ssv-scanner","module":"scanner"}
{"time":"2025-12-18T13:26:43.43658069Z","level":"INFO","msg":"Generated operator pubkeys file","module":"scanner","path":"/app/shared/data/operators/operator-pubkeys-local_testnet.json","count":4}
{"time":"2025-12-18T13:26:43.436664364Z","level":"INFO","msg":"Loaded operator pubkeys into cache","module":"scanner","count":4}
{"time":"2025-12-18T13:26:43.436759406Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"consensus"}
{"time":"2025-12-18T13:26:43.436792506Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"flow-emits"}
{"time":"2025-12-18T13:26:43.43681697Z","level":"INFO","msg":"Validation client initialized","module":"main","loki_enabled":true,"e2m_enabled":false}
{"time":"2025-12-18T13:26:43.436831253Z","level":"INFO","msg":"🔑 KEY STEP: Starting Isolated Tests Concurrently if exists...","module":"main"}
{"time":"2025-12-18T13:26:43.436854436Z","level":"INFO","msg":"🔑 KEY STEP: Starting Sanity Test Suite","module":"main"}
{"time":"2025-12-18T13:26:43.436945779Z","level":"INFO","msg":"No isolated tests configured, skipping","module":"main"}
{"time":"2025-12-18T13:26:43.465126697Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":1}
{"time":"2025-12-18T13:26:43.475772684Z","level":"INFO","msg":"Available slots","test":"sanity","node":1,"slots":500}
{"time":"2025-12-18T13:26:43.475831798Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":2}
{"time":"2025-12-18T13:26:43.481032457Z","level":"INFO","msg":"Available slots","test":"sanity","node":2,"slots":468}
{"time":"2025-12-18T13:26:43.481070169Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":3}
{"time":"2025-12-18T13:26:43.486970089Z","level":"INFO","msg":"Available slots","test":"sanity","node":3,"slots":468}
{"time":"2025-12-18T13:26:43.487017117Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":4}
{"time":"2025-12-18T13:26:43.492608791Z","level":"INFO","msg":"Available slots","test":"sanity","node":4,"slots":468}
{"time":"2025-12-18T13:26:50.160380974Z","level":"INFO","msg":"Extracted keyshares path","test":"sanity","module":"keys","path":"/app/shared/data/keyshares/0x81f9ac9f60825d682d5ab33098117dbdcf3c5245116c8c03a8c0493a5d441ba578a0b3d069d745cfdff70122c65e421a.json"}
{"time":"2025-12-18T13:26:50.160493811Z","level":"INFO","msg":"Key shares file generated successfully","test":"sanity","keyshare_path":"/app/shared/data/keyshares/0x81f9ac9f60825d682d5ab33098117dbdcf3c5245116c8c03a8c0493a5d441ba578a0b3d069d745cfdff70122c65e421a.json"}
{"time":"2025-12-18T13:26:51.25208708Z","level":"INFO","msg":"Allowance is zero","test":"sanity","module":"contract","account":"0xe25583099ba105d9ec0a67f5ae86d90e50036425"}
{"time":"2025-12-18T13:26:51.252303285Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0xe25583099ba105d9ec0a67f5ae86d90e50036425","allowance":"0"}
{"time":"2025-12-18T13:26:51.252398838Z","level":"INFO","msg":"Threshold value","test":"sanity","module":"contract","threshold":"5000000000000000000"}
{"time":"2025-12-18T13:26:51.252455587Z","level":"INFO","msg":"Allowance below threshold","test":"sanity","module":"contract","allowance":"0","threshold":"5000000000000000000"}
{"time":"2025-12-18T13:26:51.28916408Z","level":"INFO","msg":"Allowance approved successfully","test":"sanity","module":"contract","tx_hash":"0xaa4b6beee7f1b197488630691d4f4963249056cccf8a2138d4d74ef047d538fb","amount":"625000000000000000000000000000000000000000000000000000000000000000000000000","spender":"0xBFfF570853d97636b78ebf262af953308924D3D8"}
{"time":"2025-12-18T13:26:51.299199008Z","level":"INFO","msg":"Waiting for transaction receipt, retrying...","test":"sanity","module":"contract","attempt":1,"sleep":10,"tx_hash":"0xaa4b6beee7f1b197488630691d4f4963249056cccf8a2138d4d74ef047d538fb"}
{"time":"2025-12-18T13:27:01.310404489Z","level":"INFO","msg":"Waiting for transaction receipt, retrying...","test":"sanity","module":"contract","attempt":2,"sleep":20,"tx_hash":"0xaa4b6beee7f1b197488630691d4f4963249056cccf8a2138d4d74ef047d538fb"}
{"time":"2025-12-18T13:27:21.316925032Z","level":"INFO","msg":"Transaction receipt received","test":"sanity","module":"contract","tx_hash":"0xaa4b6beee7f1b197488630691d4f4963249056cccf8a2138d4d74ef047d538fb"}
{"time":"2025-12-18T13:27:21.317045648Z","level":"INFO","msg":"Approved additional allowance","test":"sanity","module":"contract","block_hash":"0x0f7c7efa2598d83cd3a476b3968574261d5868cc6a0755574ec99b40fdcae9d1","additional_amount":"625000000000000000000000000000000000000000000000000000000000000000000000000"}
{"time":"2025-12-18T13:27:21.321075355Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0xe25583099ba105d9ec0a67f5ae86d90e50036425","allowance":"625000000000000000000000000000000000000000000000000000000000000000000000000"}
{"time":"2025-12-18T13:27:21.321163182Z","level":"INFO","msg":"Allowance","test":"sanity","module":"contract","allowance":"625000000000000000000000000000000000000000000000000000000000000000000000000"}
{"time":"2025-12-18T13:27:21.321216975Z","level":"INFO","msg":"Calling RegisterValidator with parameters","test":"sanity","module":"contract","validator_pubkey":"81f9ac9f60825d682d5ab33098117dbdcf3c5245116c8c03a8c0493a5d441ba578a0b3d069d745cfdff70122c65e421a","operator_ids":[1,2,3,4],"shares_data_len":1312,"amount":"1500000000000000000","cluster_validator_count":32,"cluster_network_fee_index":0,"cluster_index":1200,"cluster_active":true,"cluster_balance":"1000000000000000000"}
{"time":"2025-12-18T13:27:21.349999319Z","level":"INFO","msg":"Registering validator","test":"sanity","module":"contract","tx_hash":"0xaf5765d75e1373ba72655cc8a903af648880c590d207aa9e0614e4044e5c1756"}
{"time":"2025-12-18T13:27:21.353059984Z","level":"INFO","msg":"Waiting for transaction receipt, retrying...","test":"sanity","module":"contract","attempt":1,"sleep":10,"tx_hash":"0xaf5765d75e1373ba72655cc8a903af648880c590d207aa9e0614e4044e5c1756"}
{"time":"2025-12-18T13:27:31.366395923Z","level":"INFO","msg":"Transaction receipt received","test":"sanity","module":"contract","tx_hash":"0xaf5765d75e1373ba72655cc8a903af648880c590d207aa9e0614e4044e5c1756"}
{"time":"2025-12-18T13:27:31.366486066Z","level":"INFO","msg":"📜 CONTRACT: Contract Register Successful","test":"sanity","txHash":"0xaf5765d75e1373ba72655cc8a903af648880c590d207aa9e0614e4044e5c1756","block_hash":"0xcadab3fd7e560300c06887f642237cfc3a0af2ce4cbb4d858350ce62dff29ce9"}
{"time":"2025-12-18T13:27:31.387784313Z","level":"INFO","msg":"Failed to get data for validator from exporter - retrying...","test":"sanity","module":"helpers","pubkey":"0x81f9ac9f60825d682d5ab33098117dbdcf3c5245116c8c03a8c0493a5d441ba578a0b3d069d745cfdff70122c65e421a","timeToSleep":10,"attempt":1}
{"time":"2025-12-18T13:27:41.393843425Z","level":"INFO","msg":"Failed to get data for validator from exporter - retrying...","test":"sanity","module":"helpers","pubkey":"0x81f9ac9f60825d682d5ab33098117dbdcf3c5245116c8c03a8c0493a5d441ba578a0b3d069d745cfdff70122c65e421a","timeToSleep":20,"attempt":2}
{"time":"2025-12-18T13:28:01.411909207Z","level":"INFO","msg":"Failed to get data for validator from exporter - retrying...","test":"sanity","module":"helpers","pubkey":"0x81f9ac9f60825d682d5ab33098117dbdcf3c5245116c8c03a8c0493a5d441ba578a0b3d069d745cfdff70122c65e421a","timeToSleep":40,"attempt":3}
{"time":"2025-12-18T13:28:41.444676522Z","level":"INFO","msg":"Failed to get data for validator from exporter - retrying...","test":"sanity","module":"helpers","pubkey":"0x81f9ac9f60825d682d5ab33098117dbdcf3c5245116c8c03a8c0493a5d441ba578a0b3d069d745cfdff70122c65e421a","timeToSleep":80,"attempt":4}
{"time":"2025-12-18T13:30:01.462149606Z","level":"INFO","msg":"Successfully retrieved validator index from exporter","test":"sanity","module":"helpers","pubkey":"0x81f9ac9f60825d682d5ab33098117dbdcf3c5245116c8c03a8c0493a5d441ba578a0b3d069d745cfdff70122c65e421a"}
{"time":"2025-12-18T13:30:01.462262618Z","level":"INFO","msg":"Loki validation skipped for local_testnet (ssv-nodes run in Kurtosis)","test":"sanity","indices":["167"],"validate-type":"registration"}
{"time":"2025-12-18T13:30:01.462294455Z","level":"INFO","msg":"✅ SUCCESS: Registration Validation Success","test":"sanity","validate_type":"registration","e2m_enabled":false,"e2m_success":false,"loki_success":true}
{"time":"2025-12-18T13:30:01.47628643Z","level":"INFO","msg":"Calling LiquidateCluster with parameters","test":"sanity","module":"contract","cluster_owner":"0xE25583099BA105D9ec0A67f5Ae86D90e50036425","operator_ids":[1,2,3,4],"cluster_validator_count":33,"cluster_network_fee_index":0,"cluster_index":9600,"cluster_active":true,"cluster_balance":"2499997312000000000"}
{"time":"2025-12-18T13:30:01.504960124Z","level":"INFO","msg":"Liquidating cluster","test":"sanity","module":"contract","tx_hash":"0x87676f623075327e6bc64b23972c6af16944202f862fe848b91d5fe8344b726b"}
{"time":"2025-12-18T13:30:01.511286673Z","level":"INFO","msg":"Waiting for transaction receipt, retrying...","test":"sanity","module":"contract","attempt":1,"sleep":10,"tx_hash":"0x87676f623075327e6bc64b23972c6af16944202f862fe848b91d5fe8344b726b"}
{"time":"2025-12-18T13:30:11.51644956Z","level":"INFO","msg":"Transaction receipt received","test":"sanity","module":"contract","tx_hash":"0x87676f623075327e6bc64b23972c6af16944202f862fe848b91d5fe8344b726b"}
{"time":"2025-12-18T13:30:11.516551248Z","level":"INFO","msg":"📜 CONTRACT: Cluster Liquidation Successful","test":"sanity","txHash":"0x87676f623075327e6bc64b23972c6af16944202f862fe848b91d5fe8344b726b","block_hash":"0x39d56c09f722306dadbca420e5b214106080bc56007b46f573f34daa3c31ee84"}
{"time":"2025-12-18T13:30:11.516586653Z","level":"INFO","msg":"Loki validation skipped for local_testnet (ssv-nodes run in Kurtosis)","test":"sanity","indices":["167"],"validate-type":"liquidation"}
{"time":"2025-12-18T13:30:11.516603561Z","level":"INFO","msg":"✅ SUCCESS: Liquidation Validation Success","test":"sanity","validate_type":"liquidation","e2m_enabled":false,"e2m_success":false,"loki_success":true}
{"time":"2025-12-18T13:30:11.516616803Z","level":"INFO","msg":"=== INVESTIGATION: Cluster state after liquidation ===","test":"sanity"}
{"time":"2025-12-18T13:30:11.519123483Z","level":"INFO","msg":"Cluster data from liquidation event","test":"sanity","source":"liquidation_event","validator_count":33,"network_fee_index":0,"index":0,"active":false,"balance":"0"}
{"time":"2025-12-18T13:30:11.519188071Z","level":"INFO","msg":"Querying scanner for actual contract cluster state after liquidation...","test":"sanity"}
{"time":"2025-12-18T13:30:13.597920691Z","level":"INFO","msg":"Cluster data from scanner (actual contract state)","test":"sanity","source":"scanner_cli","validator_count":33,"network_fee_index":0,"index":0,"active":false,"balance":"0"}
{"time":"2025-12-18T13:30:13.59800496Z","level":"INFO","msg":"Cluster data from before liquidation","test":"sanity","source":"pre_liquidation","validator_count":33,"network_fee_index":0,"index":9600,"active":true,"balance":"2499997312000000000"}
{"time":"2025-12-18T13:30:13.598014656Z","level":"INFO","msg":"=== COMPARISON: Liquidation Event vs Scanner vs Pre-Liquidation ===","test":"sanity"}
{"time":"2025-12-18T13:30:13.598025656Z","level":"INFO","msg":"Index comparison","test":"sanity","event_index":0,"scanner_index":0,"pre_liq_index":9600,"match":true}
{"time":"2025-12-18T13:30:13.598035794Z","level":"INFO","msg":"NetworkFeeIndex comparison","test":"sanity","event_network_fee_index":0,"scanner_network_fee_index":0,"pre_liq_network_fee_index":0,"match":true}
{"time":"2025-12-18T13:30:13.598071128Z","level":"INFO","msg":"Active flag comparison","test":"sanity","event_active":false,"scanner_active":false,"match":true}
{"time":"2025-12-18T13:30:13.598080035Z","level":"INFO","msg":"=== ANALYSIS: Contract behavior detected ===","test":"sanity"}
{"time":"2025-12-18T13:30:13.598088512Z","level":"WARN","msg":"Contract zeroed cluster index during liquidation - will use pre-liquidation index for reactivation","test":"sanity","scanner_index":0,"pre_liq_index":9600}
{"time":"2025-12-18T13:30:13.598108119Z","level":"INFO","msg":"Building reactivation cluster with pre-liquidation index","test":"sanity"}
{"time":"2025-12-18T13:30:13.598133026Z","level":"INFO","msg":"Reactivation cluster snapshot built","test":"sanity","validator_count":33,"network_fee_index":0,"index":9600,"active":false,"balance":"0"}
{"time":"2025-12-18T13:30:13.602031312Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0xe25583099ba105d9ec0a67f5ae86d90e50036425","allowance":"624999999999999999999999999999999999999999999999999999998500000000000000000"}
{"time":"2025-12-18T13:30:13.602094042Z","level":"INFO","msg":"Threshold value","test":"sanity","module":"contract","threshold":"5000000000000000000"}
{"time":"2025-12-18T13:30:13.60210829Z","level":"INFO","msg":"Sufficient allowance","test":"sanity","module":"contract","allowance margin":"625000000000000000000000000000000000000000000000000000000000000000000000000"}
{"time":"2025-12-18T13:30:13.602129298Z","level":"INFO","msg":"Calling ReactivateCluster with parameters","test":"sanity","module":"contract","operator_ids":[1,2,3,4],"amount":"5000000000000000000","cluster_validator_count":33,"cluster_network_fee_index":0,"cluster_index":9600,"cluster_active":false,"cluster_balance":"0"}
{"time":"2025-12-18T13:30:13.619908959Z","level":"ERROR","msg":"CRITICAL: Reactivate transaction failed - simulation reverted","test":"sanity","module":"contract","error":"execution reverted","error_string":"execution reverted","error_data":"0x12e04c87","revert_reason":"execution reverted","error_category":"unknown_error","operator_ids":[1,2,3,4],"amount":"5000000000000000000","cluster_validator_count":33,"cluster_network_fee_index":0,"cluster_index":9600,"cluster_active":false,"cluster_balance":"0"}
{"time":"2025-12-18T13:30:13.620000158Z","level":"ERROR","msg":"❌ FAILURE: Cluster Reactivation Failed","test":"sanity","error":"reactivate failed with 'execution reverted' (category: unknown_error, index=9600, active=false): execution reverted"}
{"time":"2025-12-18T13:30:13.620024031Z","level":"ERROR","msg":"❌ FAILURE: Sanity Test Suite Failed","module":"main","error":"reactivate failed with 'execution reverted' (category: unknown_error, index=9600, active=false): execution reverted"}
{"time":"2025-12-18T13:30:13.620038242Z","level":"INFO","msg":"⏳ PROGRESS: Waiting for all tests to finish","module":"main"}
{"time":"2025-12-18T13:30:13.620100505Z","level":"ERROR","msg":"❌ FAILURE: Test Execution Completed with Failures","module":"main","passed_suites":null,"failed_suites":["sanity"],"skipped_suites":["isolated"]}
{"time":"2025-12-18T13:30:13.62012708Z","level":"ERROR","msg":"❌ FAILURE: Suite Failure Details","module":"main","suite":"sanity","error":"reactivate failed with 'execution reverted' (category: unknown_error, index=9600, active=false): execution reverted"}
{"time":"2025-12-18T13:30:13.620138104Z","level":"INFO","msg":"🔑 KEY STEP: Starting resource teardown...","module":"main","usage_id":"lt-20251218-9a8b"}
{"time":"2025-12-18T13:30:13.625081677Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","module":"main","endpoint":"http://ssv-mini-pr-2627-commit-e998ec6-orchestrator:8080"}
{"time":"2025-12-18T13:30:13.62515556Z","level":"INFO","msg":"🧹 Requesting resource teardown","module":"main","usage_id":"lt-20251218-9a8b"}
{"time":"2025-12-18T13:30:13.659723602Z","level":"INFO","msg":"✅ SUCCESS: Completed resource teardown successfully","module":"main","usage_id":"lt-20251218-9a8b"}

2025-12-18T13:30:17Z The test run failed with error

Result - 🔴 Failure

2025-12-18T13:45:35Z Finished cleaning reserved resources

vaclav-ssvlabs · 2025-12-18T15:25:19Z

/test network=local_testnet

ssv-test-bot · 2025-12-18T15:25:23Z

🔴 Test run for PR #2627

Branch: vaclav-test-aetheria-local-testnet
Commit: e998ec6

Status

2025-12-18T15:25:23Z Starting test run
2025-12-18T15:25:24Z Test run submitted successfully for commit e998ec6
2025-12-18T15:25:25Z container image for commit e998ec67cd5500a98f271ca5ac5fe5c8d02170f8 is build and present in container registry
2025-12-18T15:25:26Z allocated node range cluster=1,2,3,4 for test run on network=local_testnet
2025-12-18T15:25:28Z deploying ssv-mini with local testnet, this take aproximatelly 8-12 minutes
2025-12-18T15:39:01Z ssv-mini with local testnet is ready
2025-12-18T15:39:03Z started test executor with tests=(sanity) and network=local_testnet

Click to expand executor logs

═══════════════════════════════════════════════════════════════════════════════════════════════════════
                🧪  WELCOME TO Executor - SSV's Distributed Test Automation Execution Layer  🧪            

                                  ___ __  __ ___  ___  _   _  _____  ___   ___ 
                                 | __|\\ \\/ /| __|/ __|| | | ||_   _|/ _ \\ | _ \\
                                 | _|  >  < | _|| (__ | |_| |  | | | (_) ||   /
                                 |___|/_/\\_\\|___|\\___| \\___/   |_|  \\___/ |_|_\\
                                        

═══════════════════════════════════════════════════════════════════════════════════════════════════════
{"time":"2025-12-18T15:39:10.95631659Z","level":"INFO","msg":"🐳 Docker mode: Using environment variables for config generation"}
{"time":"2025-12-18T15:39:10.964474707Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","endpoint":"http://ssv-mini-pr-2627-commit-e998ec6-orchestrator:8080"}
{"time":"2025-12-18T15:39:10.964650533Z","level":"INFO","msg":"📝 Requesting configuration generation from orchestrator API","network":"local_testnet","clusters":[1,2,3,4],"tests":"(sanity)"}
{"time":"2025-12-18T15:39:10.965339284Z","level":"INFO","msg":"⏳ Waiting for orchestrator to generate config (this may take a while)","timeout":"30m0s"}
{"time":"2025-12-18T15:39:15.066102312Z","level":"INFO","msg":"✅ Configuration generated successfully via API","resource_usage_id":"lt-20251218-2f1f"}
2025/12/18 15:39:15 WARN Threshold not provided in config; using local_testnet default threshold=5000000000000000000
{"time":"2025-12-18T15:39:15.067657202Z","level":"INFO","msg":"🚀 EXECUTOR Initialized Successfully"}
{"time":"2025-12-18T15:39:15.067741381Z","level":"INFO","msg":"⚙️ Test configuration loaded","network":"local_testnet","nodes":[1,2,3,4],"test_type":{"Sanity":{"ShouldRun":true,"E2M":false,"Threshold":false,"BulkAmount":0},"Bulk":{"ShouldRun":false,"E2M":false,"Threshold":false,"BulkAmount":0},"Isolated":{"Tests":null}}}
{"time":"2025-12-18T15:39:15.068198371Z","level":"INFO","msg":"Operator pubkeys file not found, generating with go-ssv-scanner","module":"scanner"}
{"time":"2025-12-18T15:39:15.094589634Z","level":"INFO","msg":"Generated operator pubkeys file","module":"scanner","path":"/app/shared/data/operators/operator-pubkeys-local_testnet.json","count":4}
{"time":"2025-12-18T15:39:15.094680288Z","level":"INFO","msg":"Loaded operator pubkeys into cache","module":"scanner","count":4}
{"time":"2025-12-18T15:39:15.094765933Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"consensus"}
{"time":"2025-12-18T15:39:15.094812724Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"flow-emits"}
{"time":"2025-12-18T15:39:15.09485143Z","level":"INFO","msg":"Validation client initialized","module":"main","loki_enabled":true,"e2m_enabled":false}
{"time":"2025-12-18T15:39:15.094863551Z","level":"INFO","msg":"🔑 KEY STEP: Starting Isolated Tests Concurrently if exists...","module":"main"}
{"time":"2025-12-18T15:39:15.094887259Z","level":"INFO","msg":"🔑 KEY STEP: Starting Sanity Test Suite","module":"main"}
{"time":"2025-12-18T15:39:15.094997147Z","level":"INFO","msg":"No isolated tests configured, skipping","module":"main"}
{"time":"2025-12-18T15:39:15.141644901Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":1}
{"time":"2025-12-18T15:39:15.151660764Z","level":"INFO","msg":"Available slots","test":"sanity","node":1,"slots":500}
{"time":"2025-12-18T15:39:15.151765527Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":2}
{"time":"2025-12-18T15:39:15.157216186Z","level":"INFO","msg":"Available slots","test":"sanity","node":2,"slots":468}
{"time":"2025-12-18T15:39:15.157307774Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":3}
{"time":"2025-12-18T15:39:15.163151519Z","level":"INFO","msg":"Available slots","test":"sanity","node":3,"slots":468}
{"time":"2025-12-18T15:39:15.16323569Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":4}
{"time":"2025-12-18T15:39:15.169645382Z","level":"INFO","msg":"Available slots","test":"sanity","node":4,"slots":468}
{"time":"2025-12-18T15:39:22.504334669Z","level":"INFO","msg":"Extracted keyshares path","test":"sanity","module":"keys","path":"/app/shared/data/keyshares/0x81f9ac9f60825d682d5ab33098117dbdcf3c5245116c8c03a8c0493a5d441ba578a0b3d069d745cfdff70122c65e421a.json"}
{"time":"2025-12-18T15:39:22.504430818Z","level":"INFO","msg":"Key shares file generated successfully","test":"sanity","keyshare_path":"/app/shared/data/keyshares/0x81f9ac9f60825d682d5ab33098117dbdcf3c5245116c8c03a8c0493a5d441ba578a0b3d069d745cfdff70122c65e421a.json"}
{"time":"2025-12-18T15:39:23.559597514Z","level":"INFO","msg":"Allowance is zero","test":"sanity","module":"contract","account":"0xe25583099ba105d9ec0a67f5ae86d90e50036425"}
{"time":"2025-12-18T15:39:23.559765162Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0xe25583099ba105d9ec0a67f5ae86d90e50036425","allowance":"0"}
{"time":"2025-12-18T15:39:23.559815125Z","level":"INFO","msg":"Threshold value","test":"sanity","module":"contract","threshold":"5000000000000000000"}
{"time":"2025-12-18T15:39:23.559838653Z","level":"INFO","msg":"Allowance below threshold","test":"sanity","module":"contract","allowance":"0","threshold":"5000000000000000000"}
{"time":"2025-12-18T15:39:23.595016203Z","level":"INFO","msg":"Allowance approved successfully","test":"sanity","module":"contract","tx_hash":"0x3e407999fd92a77dbf7e2e9a93993c2bd4f4f3de779b6d11dde3aea3fd9a9798","amount":"625000000000000000000000000000000000000000000000000000000000000000000000000","spender":"0xBFfF570853d97636b78ebf262af953308924D3D8"}
{"time":"2025-12-18T15:39:23.596952401Z","level":"INFO","msg":"Waiting for transaction receipt, retrying...","test":"sanity","module":"contract","attempt":1,"sleep":10,"tx_hash":"0x3e407999fd92a77dbf7e2e9a93993c2bd4f4f3de779b6d11dde3aea3fd9a9798"}
{"time":"2025-12-18T15:39:33.604877721Z","level":"INFO","msg":"Transaction receipt received","test":"sanity","module":"contract","tx_hash":"0x3e407999fd92a77dbf7e2e9a93993c2bd4f4f3de779b6d11dde3aea3fd9a9798"}
{"time":"2025-12-18T15:39:33.60503744Z","level":"INFO","msg":"Approved additional allowance","test":"sanity","module":"contract","block_hash":"0x8c6713ab271669a7ac2c337b271fdbbe909323f187596fee6444601d63944625","additional_amount":"625000000000000000000000000000000000000000000000000000000000000000000000000"}
{"time":"2025-12-18T15:39:33.610876827Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0xe25583099ba105d9ec0a67f5ae86d90e50036425","allowance":"625000000000000000000000000000000000000000000000000000000000000000000000000"}
{"time":"2025-12-18T15:39:33.611037468Z","level":"INFO","msg":"Allowance","test":"sanity","module":"contract","allowance":"625000000000000000000000000000000000000000000000000000000000000000000000000"}
{"time":"2025-12-18T15:39:33.611167395Z","level":"INFO","msg":"Calling RegisterValidator with parameters","test":"sanity","module":"contract","validator_pubkey":"81f9ac9f60825d682d5ab33098117dbdcf3c5245116c8c03a8c0493a5d441ba578a0b3d069d745cfdff70122c65e421a","operator_ids":[1,2,3,4],"shares_data_len":1312,"amount":"1500000000000000000","cluster_validator_count":32,"cluster_network_fee_index":0,"cluster_index":800,"cluster_active":true,"cluster_balance":"1000000000000000000"}
{"time":"2025-12-18T15:39:33.64268928Z","level":"INFO","msg":"Registering validator","test":"sanity","module":"contract","tx_hash":"0x1eb728937fa9a61ed19d7fb7c17821894d1cc1ac462bfb32a09789f20467d816"}
{"time":"2025-12-18T15:39:33.644706567Z","level":"INFO","msg":"Waiting for transaction receipt, retrying...","test":"sanity","module":"contract","attempt":1,"sleep":10,"tx_hash":"0x1eb728937fa9a61ed19d7fb7c17821894d1cc1ac462bfb32a09789f20467d816"}
{"time":"2025-12-18T15:39:43.655605443Z","level":"INFO","msg":"Transaction receipt received","test":"sanity","module":"contract","tx_hash":"0x1eb728937fa9a61ed19d7fb7c17821894d1cc1ac462bfb32a09789f20467d816"}
{"time":"2025-12-18T15:39:43.655680989Z","level":"INFO","msg":"📜 CONTRACT: Contract Register Successful","test":"sanity","txHash":"0x1eb728937fa9a61ed19d7fb7c17821894d1cc1ac462bfb32a09789f20467d816","block_hash":"0xbf53c210f891896ae623c2522b1d3dc2f61bf6e87ac5e00921af7210b6871be0"}
{"time":"2025-12-18T15:39:43.66814794Z","level":"INFO","msg":"Failed to get data for validator from exporter - retrying...","test":"sanity","module":"helpers","pubkey":"0x81f9ac9f60825d682d5ab33098117dbdcf3c5245116c8c03a8c0493a5d441ba578a0b3d069d745cfdff70122c65e421a","timeToSleep":10,"attempt":1}
{"time":"2025-12-18T15:39:53.672333846Z","level":"INFO","msg":"Failed to get data for validator from exporter - retrying...","test":"sanity","module":"helpers","pubkey":"0x81f9ac9f60825d682d5ab33098117dbdcf3c5245116c8c03a8c0493a5d441ba578a0b3d069d745cfdff70122c65e421a","timeToSleep":20,"attempt":2}
{"time":"2025-12-18T15:40:13.691005291Z","level":"INFO","msg":"Failed to get data for validator from exporter - retrying...","test":"sanity","module":"helpers","pubkey":"0x81f9ac9f60825d682d5ab33098117dbdcf3c5245116c8c03a8c0493a5d441ba578a0b3d069d745cfdff70122c65e421a","timeToSleep":40,"attempt":3}
{"time":"2025-12-18T15:40:53.731384739Z","level":"INFO","msg":"Failed to get data for validator from exporter - retrying...","test":"sanity","module":"helpers","pubkey":"0x81f9ac9f60825d682d5ab33098117dbdcf3c5245116c8c03a8c0493a5d441ba578a0b3d069d745cfdff70122c65e421a","timeToSleep":80,"attempt":4}
{"time":"2025-12-18T15:42:13.749947041Z","level":"INFO","msg":"Successfully retrieved validator index from exporter","test":"sanity","module":"helpers","pubkey":"0x81f9ac9f60825d682d5ab33098117dbdcf3c5245116c8c03a8c0493a5d441ba578a0b3d069d745cfdff70122c65e421a"}
{"time":"2025-12-18T15:42:13.750445501Z","level":"INFO","msg":"Loki validation skipped for local_testnet (ssv-nodes run in Kurtosis)","test":"sanity","indices":["167"],"validate-type":"registration"}
{"time":"2025-12-18T15:42:13.750535812Z","level":"INFO","msg":"✅ SUCCESS: Registration Validation Success","test":"sanity","validate_type":"registration","e2m_enabled":false,"e2m_success":false,"loki_success":true}
{"time":"2025-12-18T15:42:13.76656702Z","level":"INFO","msg":"Calling LiquidateCluster with parameters","test":"sanity","module":"contract","cluster_owner":"0xE25583099BA105D9ec0A67f5Ae86D90e50036425","operator_ids":[1,2,3,4],"cluster_validator_count":33,"cluster_network_fee_index":0,"cluster_index":10000,"cluster_active":true,"cluster_balance":"2499997056000000000"}
{"time":"2025-12-18T15:42:13.802941895Z","level":"INFO","msg":"Liquidating cluster","test":"sanity","module":"contract","tx_hash":"0x4982ed09eda7875f9a8b43a271336635919ee14f767a7e5f21749799004ef918"}
{"time":"2025-12-18T15:42:13.806704161Z","level":"INFO","msg":"Waiting for transaction receipt, retrying...","test":"sanity","module":"contract","attempt":1,"sleep":10,"tx_hash":"0x4982ed09eda7875f9a8b43a271336635919ee14f767a7e5f21749799004ef918"}
{"time":"2025-12-18T15:42:23.812566576Z","level":"INFO","msg":"Waiting for transaction receipt, retrying...","test":"sanity","module":"contract","attempt":2,"sleep":20,"tx_hash":"0x4982ed09eda7875f9a8b43a271336635919ee14f767a7e5f21749799004ef918"}
{"time":"2025-12-18T15:42:43.819263714Z","level":"INFO","msg":"Transaction receipt received","test":"sanity","module":"contract","tx_hash":"0x4982ed09eda7875f9a8b43a271336635919ee14f767a7e5f21749799004ef918"}
{"time":"2025-12-18T15:42:43.819327852Z","level":"INFO","msg":"📜 CONTRACT: Cluster Liquidation Successful","test":"sanity","txHash":"0x4982ed09eda7875f9a8b43a271336635919ee14f767a7e5f21749799004ef918","block_hash":"0xbc3fb606a99001b1b10e2e586158f2119595d99f88845a6b8445269f39dacca7"}
{"time":"2025-12-18T15:42:43.819355289Z","level":"INFO","msg":"Loki validation skipped for local_testnet (ssv-nodes run in Kurtosis)","test":"sanity","indices":["167"],"validate-type":"liquidation"}
{"time":"2025-12-18T15:42:43.81937342Z","level":"INFO","msg":"✅ SUCCESS: Liquidation Validation Success","test":"sanity","validate_type":"liquidation","e2m_enabled":false,"e2m_success":false,"loki_success":true}
{"time":"2025-12-18T15:42:43.819390232Z","level":"INFO","msg":"=== INVESTIGATION: Cluster state after liquidation ===","test":"sanity"}
{"time":"2025-12-18T15:42:43.823320466Z","level":"INFO","msg":"Cluster data from liquidation event","test":"sanity","source":"liquidation_event","validator_count":33,"network_fee_index":0,"index":0,"active":false,"balance":"0"}
{"time":"2025-12-18T15:42:43.823386795Z","level":"INFO","msg":"Querying scanner for actual contract cluster state after liquidation...","test":"sanity"}
{"time":"2025-12-18T15:42:45.900557369Z","level":"INFO","msg":"Cluster data from scanner (actual contract state)","test":"sanity","source":"scanner_cli","validator_count":33,"network_fee_index":0,"index":0,"active":false,"balance":"0"}
{"time":"2025-12-18T15:42:45.900652103Z","level":"INFO","msg":"Cluster data from before liquidation","test":"sanity","source":"pre_liquidation","validator_count":33,"network_fee_index":0,"index":10000,"active":true,"balance":"2499997056000000000"}
{"time":"2025-12-18T15:42:45.900661946Z","level":"INFO","msg":"=== COMPARISON: Liquidation Event vs Scanner vs Pre-Liquidation ===","test":"sanity"}
{"time":"2025-12-18T15:42:45.900672547Z","level":"INFO","msg":"Index comparison","test":"sanity","event_index":0,"scanner_index":0,"pre_liq_index":10000,"match":true}
{"time":"2025-12-18T15:42:45.900683896Z","level":"INFO","msg":"NetworkFeeIndex comparison","test":"sanity","event_network_fee_index":0,"scanner_network_fee_index":0,"pre_liq_network_fee_index":0,"match":true}
{"time":"2025-12-18T15:42:45.900713795Z","level":"INFO","msg":"Active flag comparison","test":"sanity","event_active":false,"scanner_active":false,"match":true}
{"time":"2025-12-18T15:42:45.900721783Z","level":"INFO","msg":"=== ANALYSIS: Contract behavior detected ===","test":"sanity"}
{"time":"2025-12-18T15:42:45.900733048Z","level":"WARN","msg":"Contract zeroed cluster index during liquidation - this is EXPECTED","test":"sanity","scanner_index":0,"pre_liq_index":10000}
{"time":"2025-12-18T15:42:45.900742174Z","level":"INFO","msg":"Using scanner data directly for reactivation (index=0 as stored on-chain)","test":"sanity"}
{"time":"2025-12-18T15:42:45.900763995Z","level":"INFO","msg":"DIAGNOSTIC: Exact reactivation cluster snapshot with types","test":"sanity","validator_count":33,"validator_count_type":"uint32","network_fee_index":0,"network_fee_index_type":"uint64","index":0,"index_type":"uint64","active":false,"active_type":"bool","balance":"0","balance_type":"*big.Int","balance_bytes":""}
{"time":"2025-12-18T15:42:45.904167143Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0xe25583099ba105d9ec0a67f5ae86d90e50036425","allowance":"624999999999999999999999999999999999999999999999999999998500000000000000000"}
{"time":"2025-12-18T15:42:45.904296902Z","level":"INFO","msg":"Threshold value","test":"sanity","module":"contract","threshold":"5000000000000000000"}
{"time":"2025-12-18T15:42:45.90435264Z","level":"INFO","msg":"Sufficient allowance","test":"sanity","module":"contract","allowance margin":"625000000000000000000000000000000000000000000000000000000000000000000000000"}
{"time":"2025-12-18T15:42:45.904398157Z","level":"INFO","msg":"Calling ReactivateCluster with parameters","test":"sanity","module":"contract","operator_ids":[1,2,3,4],"amount":"5000000000000000000","cluster_validator_count":33,"cluster_network_fee_index":0,"cluster_index":0,"cluster_active":false,"cluster_balance":"0"}
{"time":"2025-12-18T15:42:45.904442543Z","level":"INFO","msg":"DIAGNOSTIC: Client-side calculated cluster hash (should match on-chain storage)","test":"sanity","module":"contract","expected_cluster_hash":"0xe1ed272b405b3d4fdae16c9b4df44053ed4f6387e5bbd6e77105b2338086d773","hash_components":"validatorCount=33, networkFeeIndex=0, index=0, balance=0, active=false"}
{"time":"2025-12-18T15:42:45.938356036Z","level":"INFO","msg":"Reactivating cluster","test":"sanity","module":"contract","tx_hash":"0x9b72e527e0552ef6e5295354d943d3cd80fc04f65d5d761bfd3c8586a8133ca3"}
{"time":"2025-12-18T15:42:45.940905479Z","level":"INFO","msg":"Waiting for transaction receipt, retrying...","test":"sanity","module":"contract","attempt":1,"sleep":10,"tx_hash":"0x9b72e527e0552ef6e5295354d943d3cd80fc04f65d5d761bfd3c8586a8133ca3"}
{"time":"2025-12-18T15:42:55.946770589Z","level":"INFO","msg":"Transaction receipt received","test":"sanity","module":"contract","tx_hash":"0x9b72e527e0552ef6e5295354d943d3cd80fc04f65d5d761bfd3c8586a8133ca3"}
{"time":"2025-12-18T15:42:55.946901589Z","level":"INFO","msg":"📜 CONTRACT: Cluster Reactivation Successful","test":"sanity","txHash":"0x9b72e527e0552ef6e5295354d943d3cd80fc04f65d5d761bfd3c8586a8133ca3","block_hash":"0x0b747284a1407f47b59c5fbb0f338ebf908bbb56fef0daf27c5ab5b09d43458d"}
{"time":"2025-12-18T15:42:55.946960935Z","level":"INFO","msg":"Loki validation skipped for local_testnet (ssv-nodes run in Kurtosis)","test":"sanity","indices":["167"],"validate-type":"reactivation"}
{"time":"2025-12-18T15:42:55.947002815Z","level":"INFO","msg":"✅ SUCCESS: Reactivation Validation Success","test":"sanity","validate_type":"reactivation","e2m_enabled":false,"e2m_success":false,"loki_success":true}
{"time":"2025-12-18T15:42:55.955048268Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0xe25583099ba105d9ec0a67f5ae86d90e50036425","allowance":"624999999999999999999999999999999999999999999999999999993500000000000000000"}
{"time":"2025-12-18T15:42:55.95511843Z","level":"INFO","msg":"Threshold value","test":"sanity","module":"contract","threshold":"5000000000000000000"}
{"time":"2025-12-18T15:42:55.95515924Z","level":"INFO","msg":"Sufficient allowance","test":"sanity","module":"contract","allowance margin":"625000000000000000000000000000000000000000000000000000000000000000000000000"}
{"time":"2025-12-18T15:42:55.955193568Z","level":"INFO","msg":"Calling RemoveValidator with parameters","test":"sanity","module":"contract","validator_pubkey":"81f9ac9f60825d682d5ab33098117dbdcf3c5245116c8c03a8c0493a5d441ba578a0b3d069d745cfdff70122c65e421a","operator_ids":[1,2,3,4],"cluster_validator_count":33,"cluster_network_fee_index":0,"cluster_index":15200,"cluster_active":true,"cluster_balance":"5000000000000000000"}
{"time":"2025-12-18T15:42:55.979032485Z","level":"INFO","msg":"Removing validator","test":"sanity","module":"contract","tx_hash":"0xa7e9ff190f7a8349c19b7e2ebe61b79cfa0466319f0fde9da74c16b375fd84b0"}
{"time":"2025-12-18T15:42:55.980994134Z","level":"INFO","msg":"Waiting for transaction receipt, retrying...","test":"sanity","module":"contract","attempt":1,"sleep":10,"tx_hash":"0xa7e9ff190f7a8349c19b7e2ebe61b79cfa0466319f0fde9da74c16b375fd84b0"}
{"time":"2025-12-18T15:43:05.992717268Z","level":"INFO","msg":"Waiting for transaction receipt, retrying...","test":"sanity","module":"contract","attempt":2,"sleep":20,"tx_hash":"0xa7e9ff190f7a8349c19b7e2ebe61b79cfa0466319f0fde9da74c16b375fd84b0"}
{"time":"2025-12-18T15:43:26.014738603Z","level":"INFO","msg":"Transaction receipt received","test":"sanity","module":"contract","tx_hash":"0xa7e9ff190f7a8349c19b7e2ebe61b79cfa0466319f0fde9da74c16b375fd84b0"}
{"time":"2025-12-18T15:43:26.014804881Z","level":"ERROR","msg":"CRITICAL: Transaction reverted on-chain","test":"sanity","module":"contract","tx_hash":"0xa7e9ff190f7a8349c19b7e2ebe61b79cfa0466319f0fde9da74c16b375fd84b0","block_hash":"0xddf74b808b663015f47d0a0641ee191564510d76d8b352eb01ee6d57cf3fa4e1","block_number":47,"gas_used":124654,"status":0}
{"time":"2025-12-18T15:43:26.014867708Z","level":"ERROR","msg":"Error checking transaction success","test":"sanity","blockHash":"0xddf74b808b663015f47d0a0641ee191564510d76d8b352eb01ee6d57cf3fa4e1"}
{"time":"2025-12-18T15:43:26.014932685Z","level":"ERROR","msg":"❌ FAILURE: Sanity Test Suite Failed","module":"main","error":"transaction verification failed (tx=0xa7e9ff190f7a8349c19b7e2ebe61b79cfa0466319f0fde9da74c16b375fd84b0, block=0xddf74b808b663015f47d0a0641ee191564510d76d8b352eb01ee6d57cf3fa4e1, gas=124654)"}
{"time":"2025-12-18T15:43:26.014958023Z","level":"INFO","msg":"⏳ PROGRESS: Waiting for all tests to finish","module":"main"}
{"time":"2025-12-18T15:43:26.015030278Z","level":"ERROR","msg":"❌ FAILURE: Test Execution Completed with Failures","module":"main","passed_suites":null,"failed_suites":["sanity"],"skipped_suites":["isolated"]}
{"time":"2025-12-18T15:43:26.01505726Z","level":"ERROR","msg":"❌ FAILURE: Suite Failure Details","module":"main","suite":"sanity","error":"transaction verification failed (tx=0xa7e9ff190f7a8349c19b7e2ebe61b79cfa0466319f0fde9da74c16b375fd84b0, block=0xddf74b808b663015f47d0a0641ee191564510d76d8b352eb01ee6d57cf3fa4e1, gas=124654)"}
{"time":"2025-12-18T15:43:26.015068294Z","level":"INFO","msg":"🔑 KEY STEP: Starting resource teardown...","module":"main","usage_id":"lt-20251218-2f1f"}
{"time":"2025-12-18T15:43:26.023414589Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","module":"main","endpoint":"http://ssv-mini-pr-2627-commit-e998ec6-orchestrator:8080"}
{"time":"2025-12-18T15:43:26.02348863Z","level":"INFO","msg":"🧹 Requesting resource teardown","module":"main","usage_id":"lt-20251218-2f1f"}
{"time":"2025-12-18T15:43:26.040176199Z","level":"INFO","msg":"✅ SUCCESS: Completed resource teardown successfully","module":"main","usage_id":"lt-20251218-2f1f"}

2025-12-18T15:43:29Z The test run failed with error

Result - 🔴 Failure

2025-12-18T15:58:33Z Finished cleaning reserved resources

vaclav-ssvlabs · 2025-12-18T17:57:52Z

/test network=local_testnet

ssv-test-bot · 2025-12-18T17:57:55Z

🔴 Test run for PR #2627

Branch: vaclav-test-aetheria-local-testnet
Commit: e998ec6

Status

2025-12-18T17:57:55Z Starting test run
2025-12-18T17:57:56Z Test run submitted successfully for commit e998ec6
2025-12-18T17:57:57Z container image for commit e998ec67cd5500a98f271ca5ac5fe5c8d02170f8 is build and present in container registry
2025-12-18T17:57:59Z allocated node range cluster=1,2,3,4 for test run on network=local_testnet
2025-12-18T17:58:00Z deploying ssv-mini with local testnet, this take aproximatelly 8-12 minutes
2025-12-18T18:09:33Z ssv-mini with local testnet is ready
2025-12-18T18:09:35Z started test executor with tests=(sanity) and network=local_testnet

Click to expand executor logs

═══════════════════════════════════════════════════════════════════════════════════════════════════════
                🧪  WELCOME TO Executor - SSV's Distributed Test Automation Execution Layer  🧪            

                                  ___ __  __ ___  ___  _   _  _____  ___   ___ 
                                 | __|\\ \\/ /| __|/ __|| | | ||_   _|/ _ \\ | _ \\
                                 | _|  >  < | _|| (__ | |_| |  | | | (_) ||   /
                                 |___|/_/\\_\\|___|\\___| \\___/   |_|  \\___/ |_|_\\
                                        

═══════════════════════════════════════════════════════════════════════════════════════════════════════
{"time":"2025-12-18T18:10:10.860532152Z","level":"INFO","msg":"🐳 Docker mode: Using environment variables for config generation"}
{"time":"2025-12-18T18:10:10.874190116Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","endpoint":"http://ssv-mini-pr-2627-commit-e998ec6-orchestrator:8080"}
{"time":"2025-12-18T18:10:10.87437241Z","level":"INFO","msg":"📝 Requesting configuration generation from orchestrator API","network":"local_testnet","clusters":[1,2,3,4],"tests":"(sanity)"}
{"time":"2025-12-18T18:10:10.874611917Z","level":"INFO","msg":"⏳ Waiting for orchestrator to generate config (this may take a while)","timeout":"30m0s"}
{"time":"2025-12-18T18:10:14.994948232Z","level":"INFO","msg":"✅ Configuration generated successfully via API","resource_usage_id":"lt-20251218-fe31"}
2025/12/18 18:10:14 WARN Threshold not provided in config; using local_testnet default threshold=5000000000000000000
{"time":"2025-12-18T18:10:14.996232882Z","level":"INFO","msg":"🚀 EXECUTOR Initialized Successfully"}
{"time":"2025-12-18T18:10:14.996299338Z","level":"INFO","msg":"⚙️ Test configuration loaded","network":"local_testnet","nodes":[1,2,3,4],"test_type":{"Sanity":{"ShouldRun":true,"E2M":false,"Threshold":false,"BulkAmount":0},"Bulk":{"ShouldRun":false,"E2M":false,"Threshold":false,"BulkAmount":0},"Isolated":{"Tests":null}}}
{"time":"2025-12-18T18:10:14.996676073Z","level":"INFO","msg":"Operator pubkeys file not found, generating with go-ssv-scanner","module":"scanner"}
{"time":"2025-12-18T18:10:15.028670409Z","level":"INFO","msg":"Generated operator pubkeys file","module":"scanner","path":"/app/shared/data/operators/operator-pubkeys-local_testnet.json","count":4}
{"time":"2025-12-18T18:10:15.028940188Z","level":"INFO","msg":"Loaded operator pubkeys into cache","module":"scanner","count":4}
{"time":"2025-12-18T18:10:15.029081059Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"consensus"}
{"time":"2025-12-18T18:10:15.029130651Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"flow-emits"}
{"time":"2025-12-18T18:10:15.029177381Z","level":"INFO","msg":"Validation client initialized","module":"main","loki_enabled":true,"e2m_enabled":false}
{"time":"2025-12-18T18:10:15.029211838Z","level":"INFO","msg":"🔑 KEY STEP: Starting Isolated Tests Concurrently if exists...","module":"main"}
{"time":"2025-12-18T18:10:15.029279666Z","level":"INFO","msg":"🔑 KEY STEP: Starting Sanity Test Suite","module":"main"}
{"time":"2025-12-18T18:10:15.029496551Z","level":"INFO","msg":"No isolated tests configured, skipping","module":"main"}
{"time":"2025-12-18T18:10:15.07756015Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":1}
{"time":"2025-12-18T18:10:15.087892489Z","level":"INFO","msg":"Available slots","test":"sanity","node":1,"slots":500}
{"time":"2025-12-18T18:10:15.087968516Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":2}
{"time":"2025-12-18T18:10:15.094798301Z","level":"INFO","msg":"Available slots","test":"sanity","node":2,"slots":468}
{"time":"2025-12-18T18:10:15.094873734Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":3}
{"time":"2025-12-18T18:10:15.101716206Z","level":"INFO","msg":"Available slots","test":"sanity","node":3,"slots":468}
{"time":"2025-12-18T18:10:15.101793911Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":4}
{"time":"2025-12-18T18:10:15.108662294Z","level":"INFO","msg":"Available slots","test":"sanity","node":4,"slots":468}
{"time":"2025-12-18T18:10:21.724862095Z","level":"INFO","msg":"Extracted keyshares path","test":"sanity","module":"keys","path":"/app/shared/data/keyshares/0x81f9ac9f60825d682d5ab33098117dbdcf3c5245116c8c03a8c0493a5d441ba578a0b3d069d745cfdff70122c65e421a.json"}
{"time":"2025-12-18T18:10:21.724921475Z","level":"INFO","msg":"Key shares file generated successfully","test":"sanity","keyshare_path":"/app/shared/data/keyshares/0x81f9ac9f60825d682d5ab33098117dbdcf3c5245116c8c03a8c0493a5d441ba578a0b3d069d745cfdff70122c65e421a.json"}
{"time":"2025-12-18T18:10:22.805147621Z","level":"INFO","msg":"Allowance is zero","test":"sanity","module":"contract","account":"0xe25583099ba105d9ec0a67f5ae86d90e50036425"}
{"time":"2025-12-18T18:10:22.80528543Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0xe25583099ba105d9ec0a67f5ae86d90e50036425","allowance":"0"}
{"time":"2025-12-18T18:10:22.805318871Z","level":"INFO","msg":"Threshold value","test":"sanity","module":"contract","threshold":"5000000000000000000"}
{"time":"2025-12-18T18:10:22.805350618Z","level":"INFO","msg":"Allowance below threshold","test":"sanity","module":"contract","allowance":"0","threshold":"5000000000000000000"}
{"time":"2025-12-18T18:10:22.833158182Z","level":"INFO","msg":"Allowance approved successfully","test":"sanity","module":"contract","tx_hash":"0x6beba6e94eba0cf291c4221c96732dbb84972e050623749b00d1e8dab518f0bf","amount":"625000000000000000000000000000000000000000000000000000000000000000000000000","spender":"0xBFfF570853d97636b78ebf262af953308924D3D8"}
{"time":"2025-12-18T18:10:22.839161945Z","level":"INFO","msg":"Waiting for transaction receipt, retrying...","test":"sanity","module":"contract","attempt":1,"sleep":10,"tx_hash":"0x6beba6e94eba0cf291c4221c96732dbb84972e050623749b00d1e8dab518f0bf"}
{"time":"2025-12-18T18:10:32.851700234Z","level":"INFO","msg":"Transaction receipt received","test":"sanity","module":"contract","tx_hash":"0x6beba6e94eba0cf291c4221c96732dbb84972e050623749b00d1e8dab518f0bf"}
{"time":"2025-12-18T18:10:32.851872932Z","level":"INFO","msg":"Approved additional allowance","test":"sanity","module":"contract","block_hash":"0xa2b26d15ef51d85b0ae84c8b38c690d42273421d4eaccd4e4237c71b63af44ad","additional_amount":"625000000000000000000000000000000000000000000000000000000000000000000000000"}
{"time":"2025-12-18T18:10:32.856801101Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0xe25583099ba105d9ec0a67f5ae86d90e50036425","allowance":"625000000000000000000000000000000000000000000000000000000000000000000000000"}
{"time":"2025-12-18T18:10:32.856882625Z","level":"INFO","msg":"Allowance","test":"sanity","module":"contract","allowance":"625000000000000000000000000000000000000000000000000000000000000000000000000"}
{"time":"2025-12-18T18:10:32.856941207Z","level":"INFO","msg":"Calling RegisterValidator with parameters","test":"sanity","module":"contract","validator_pubkey":"81f9ac9f60825d682d5ab33098117dbdcf3c5245116c8c03a8c0493a5d441ba578a0b3d069d745cfdff70122c65e421a","operator_ids":[1,2,3,4],"shares_data_len":1312,"amount":"1500000000000000000","cluster_validator_count":32,"cluster_network_fee_index":0,"cluster_index":1200,"cluster_active":true,"cluster_balance":"1000000000000000000"}
{"time":"2025-12-18T18:10:32.88671363Z","level":"INFO","msg":"Registering validator","test":"sanity","module":"contract","tx_hash":"0xad4495fbc3db0eb91e79531a68cf58103289377677ae6fb056c6011410f7a7dc"}
{"time":"2025-12-18T18:10:32.889315697Z","level":"INFO","msg":"Waiting for transaction receipt, retrying...","test":"sanity","module":"contract","attempt":1,"sleep":10,"tx_hash":"0xad4495fbc3db0eb91e79531a68cf58103289377677ae6fb056c6011410f7a7dc"}
{"time":"2025-12-18T18:10:42.900215549Z","level":"INFO","msg":"Transaction receipt received","test":"sanity","module":"contract","tx_hash":"0xad4495fbc3db0eb91e79531a68cf58103289377677ae6fb056c6011410f7a7dc"}
{"time":"2025-12-18T18:10:42.900346092Z","level":"INFO","msg":"📜 CONTRACT: Contract Register Successful","test":"sanity","txHash":"0xad4495fbc3db0eb91e79531a68cf58103289377677ae6fb056c6011410f7a7dc","block_hash":"0xff8a2bc41d3f18b7c0bbd4b6cf2a879bbd146ffa9faa9938daef0d693499db5d"}
{"time":"2025-12-18T18:10:42.918304184Z","level":"INFO","msg":"Failed to get data for validator from exporter - retrying...","test":"sanity","module":"helpers","pubkey":"0x81f9ac9f60825d682d5ab33098117dbdcf3c5245116c8c03a8c0493a5d441ba578a0b3d069d745cfdff70122c65e421a","timeToSleep":10,"attempt":1}
{"time":"2025-12-18T18:10:52.927951471Z","level":"INFO","msg":"Failed to get data for validator from exporter - retrying...","test":"sanity","module":"helpers","pubkey":"0x81f9ac9f60825d682d5ab33098117dbdcf3c5245116c8c03a8c0493a5d441ba578a0b3d069d745cfdff70122c65e421a","timeToSleep":20,"attempt":2}
{"time":"2025-12-18T18:11:12.947117488Z","level":"INFO","msg":"Failed to get data for validator from exporter - retrying...","test":"sanity","module":"helpers","pubkey":"0x81f9ac9f60825d682d5ab33098117dbdcf3c5245116c8c03a8c0493a5d441ba578a0b3d069d745cfdff70122c65e421a","timeToSleep":40,"attempt":3}
{"time":"2025-12-18T18:11:52.971926995Z","level":"INFO","msg":"Failed to get data for validator from exporter - retrying...","test":"sanity","module":"helpers","pubkey":"0x81f9ac9f60825d682d5ab33098117dbdcf3c5245116c8c03a8c0493a5d441ba578a0b3d069d745cfdff70122c65e421a","timeToSleep":80,"attempt":4}
{"time":"2025-12-18T18:13:12.993449812Z","level":"INFO","msg":"Successfully retrieved validator index from exporter","test":"sanity","module":"helpers","pubkey":"0x81f9ac9f60825d682d5ab33098117dbdcf3c5245116c8c03a8c0493a5d441ba578a0b3d069d745cfdff70122c65e421a"}
{"time":"2025-12-18T18:13:12.993727993Z","level":"INFO","msg":"Loki validation skipped for local_testnet (ssv-nodes run in Kurtosis)","test":"sanity","indices":["167"],"validate-type":"registration"}
{"time":"2025-12-18T18:13:12.993828671Z","level":"INFO","msg":"✅ SUCCESS: Registration Validation Success","test":"sanity","validate_type":"registration","e2m_enabled":false,"e2m_success":false,"loki_success":true}
{"time":"2025-12-18T18:13:13.014757401Z","level":"INFO","msg":"Calling LiquidateCluster with parameters","test":"sanity","module":"contract","cluster_owner":"0xE25583099BA105D9ec0A67f5Ae86D90e50036425","operator_ids":[1,2,3,4],"cluster_validator_count":33,"cluster_network_fee_index":0,"cluster_index":10800,"cluster_active":true,"cluster_balance":"2499996928000000000"}
{"time":"2025-12-18T18:13:13.038674121Z","level":"INFO","msg":"Liquidating cluster","test":"sanity","module":"contract","tx_hash":"0xdc00ceab68bf1117cd6d5cf45dc7c0a879f550cd9dbc03e752ae012deb4edfbf"}
{"time":"2025-12-18T18:13:13.040676104Z","level":"INFO","msg":"Waiting for transaction receipt, retrying...","test":"sanity","module":"contract","attempt":1,"sleep":10,"tx_hash":"0xdc00ceab68bf1117cd6d5cf45dc7c0a879f550cd9dbc03e752ae012deb4edfbf"}
{"time":"2025-12-18T18:13:23.051737963Z","level":"INFO","msg":"Transaction receipt received","test":"sanity","module":"contract","tx_hash":"0xdc00ceab68bf1117cd6d5cf45dc7c0a879f550cd9dbc03e752ae012deb4edfbf"}
{"time":"2025-12-18T18:13:23.051864969Z","level":"INFO","msg":"📜 CONTRACT: Cluster Liquidation Successful","test":"sanity","txHash":"0xdc00ceab68bf1117cd6d5cf45dc7c0a879f550cd9dbc03e752ae012deb4edfbf","block_hash":"0x4d81c5691e958082616ffcf2bb112dbf119c3310f3b6d21508dd4671bfe0ac3e"}
{"time":"2025-12-18T18:13:23.051916933Z","level":"INFO","msg":"Loki validation skipped for local_testnet (ssv-nodes run in Kurtosis)","test":"sanity","indices":["167"],"validate-type":"liquidation"}
{"time":"2025-12-18T18:13:23.051950493Z","level":"INFO","msg":"✅ SUCCESS: Liquidation Validation Success","test":"sanity","validate_type":"liquidation","e2m_enabled":false,"e2m_success":false,"loki_success":true}
{"time":"2025-12-18T18:13:23.052058947Z","level":"INFO","msg":"=== INVESTIGATION: Cluster state after liquidation ===","test":"sanity"}
{"time":"2025-12-18T18:13:23.054838336Z","level":"INFO","msg":"Cluster data from liquidation event","test":"sanity","source":"liquidation_event","validator_count":33,"network_fee_index":0,"index":0,"active":false,"balance":"0"}
{"time":"2025-12-18T18:13:23.054922667Z","level":"INFO","msg":"Querying scanner for actual contract cluster state after liquidation...","test":"sanity"}
{"time":"2025-12-18T18:13:25.128246595Z","level":"INFO","msg":"Cluster data from scanner (actual contract state)","test":"sanity","source":"scanner_cli","validator_count":33,"network_fee_index":0,"index":0,"active":false,"balance":"0"}
{"time":"2025-12-18T18:13:25.128318386Z","level":"INFO","msg":"Cluster data from before liquidation","test":"sanity","source":"pre_liquidation","validator_count":33,"network_fee_index":0,"index":10800,"active":true,"balance":"2499996928000000000"}
{"time":"2025-12-18T18:13:25.128327802Z","level":"INFO","msg":"=== COMPARISON: Liquidation Event vs Scanner vs Pre-Liquidation ===","test":"sanity"}
{"time":"2025-12-18T18:13:25.128338867Z","level":"INFO","msg":"Index comparison","test":"sanity","event_index":0,"scanner_index":0,"pre_liq_index":10800,"match":true}
{"time":"2025-12-18T18:13:25.128349813Z","level":"INFO","msg":"NetworkFeeIndex comparison","test":"sanity","event_network_fee_index":0,"scanner_network_fee_index":0,"pre_liq_network_fee_index":0,"match":true}
{"time":"2025-12-18T18:13:25.128361488Z","level":"INFO","msg":"Active flag comparison","test":"sanity","event_active":false,"scanner_active":false,"match":true}
{"time":"2025-12-18T18:13:25.128373021Z","level":"INFO","msg":"=== ANALYSIS: Contract behavior detected ===","test":"sanity"}
{"time":"2025-12-18T18:13:25.128396846Z","level":"WARN","msg":"Contract zeroed cluster index during liquidation - this is EXPECTED","test":"sanity","scanner_index":0,"pre_liq_index":10800}
{"time":"2025-12-18T18:13:25.128409359Z","level":"INFO","msg":"Using scanner data directly for reactivation (index=0 as stored on-chain)","test":"sanity"}
{"time":"2025-12-18T18:13:25.128427991Z","level":"INFO","msg":"DIAGNOSTIC: Exact reactivation cluster snapshot with types","test":"sanity","validator_count":33,"validator_count_type":"uint32","network_fee_index":0,"network_fee_index_type":"uint64","index":0,"index_type":"uint64","active":false,"active_type":"bool","balance":"0","balance_type":"*big.Int","balance_bytes":""}
{"time":"2025-12-18T18:13:25.131090125Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0xe25583099ba105d9ec0a67f5ae86d90e50036425","allowance":"624999999999999999999999999999999999999999999999999999998500000000000000000"}
{"time":"2025-12-18T18:13:25.131124401Z","level":"INFO","msg":"Threshold value","test":"sanity","module":"contract","threshold":"5000000000000000000"}
{"time":"2025-12-18T18:13:25.131140024Z","level":"INFO","msg":"Sufficient allowance","test":"sanity","module":"contract","allowance margin":"625000000000000000000000000000000000000000000000000000000000000000000000000"}
{"time":"2025-12-18T18:13:25.131159114Z","level":"INFO","msg":"Calling ReactivateCluster with parameters","test":"sanity","module":"contract","operator_ids":[1,2,3,4],"amount":"5000000000000000000","cluster_validator_count":33,"cluster_network_fee_index":0,"cluster_index":0,"cluster_active":false,"cluster_balance":"0"}
{"time":"2025-12-18T18:13:25.131194021Z","level":"INFO","msg":"DIAGNOSTIC: Client-side calculated cluster hash (should match on-chain storage)","test":"sanity","module":"contract","expected_cluster_hash":"0xe1ed272b405b3d4fdae16c9b4df44053ed4f6387e5bbd6e77105b2338086d773","hash_components":"validatorCount=33, networkFeeIndex=0, index=0, balance=0, active=false"}
{"time":"2025-12-18T18:13:25.155797565Z","level":"INFO","msg":"Reactivating cluster","test":"sanity","module":"contract","tx_hash":"0x52317af6d5f60a6f45b3649b19ffb1854792fbdd509fc0048c7719bb0e9c3fe8"}
{"time":"2025-12-18T18:13:25.158181137Z","level":"INFO","msg":"Waiting for transaction receipt, retrying...","test":"sanity","module":"contract","attempt":1,"sleep":10,"tx_hash":"0x52317af6d5f60a6f45b3649b19ffb1854792fbdd509fc0048c7719bb0e9c3fe8"}
{"time":"2025-12-18T18:13:35.165483571Z","level":"INFO","msg":"Waiting for transaction receipt, retrying...","test":"sanity","module":"contract","attempt":2,"sleep":20,"tx_hash":"0x52317af6d5f60a6f45b3649b19ffb1854792fbdd509fc0048c7719bb0e9c3fe8"}
{"time":"2025-12-18T18:13:55.180746542Z","level":"INFO","msg":"Transaction receipt received","test":"sanity","module":"contract","tx_hash":"0x52317af6d5f60a6f45b3649b19ffb1854792fbdd509fc0048c7719bb0e9c3fe8"}
{"time":"2025-12-18T18:13:55.18087113Z","level":"ERROR","msg":"CRITICAL: Transaction reverted on-chain","test":"sanity","module":"contract","tx_hash":"0x52317af6d5f60a6f45b3649b19ffb1854792fbdd509fc0048c7719bb0e9c3fe8","block_hash":"0x0b0c7c81a3548cbf5587a27b14338cb3ddfcb65c2040e0590d347a43745590fb","block_number":48,"gas_used":143056,"status":0}
{"time":"2025-12-18T18:13:55.180959748Z","level":"ERROR","msg":"Error checking transaction success","test":"sanity","blockHash":"0x0b0c7c81a3548cbf5587a27b14338cb3ddfcb65c2040e0590d347a43745590fb"}
{"time":"2025-12-18T18:13:55.181010468Z","level":"ERROR","msg":"❌ FAILURE: Sanity Test Suite Failed","module":"main","error":"transaction verification failed (tx=0x52317af6d5f60a6f45b3649b19ffb1854792fbdd509fc0048c7719bb0e9c3fe8, block=0x0b0c7c81a3548cbf5587a27b14338cb3ddfcb65c2040e0590d347a43745590fb, gas=143056)"}
{"time":"2025-12-18T18:13:55.181041387Z","level":"INFO","msg":"⏳ PROGRESS: Waiting for all tests to finish","module":"main"}
{"time":"2025-12-18T18:13:55.181146126Z","level":"ERROR","msg":"❌ FAILURE: Test Execution Completed with Failures","module":"main","passed_suites":null,"failed_suites":["sanity"],"skipped_suites":["isolated"]}
{"time":"2025-12-18T18:13:55.181233154Z","level":"ERROR","msg":"❌ FAILURE: Suite Failure Details","module":"main","suite":"sanity","error":"transaction verification failed (tx=0x52317af6d5f60a6f45b3649b19ffb1854792fbdd509fc0048c7719bb0e9c3fe8, block=0x0b0c7c81a3548cbf5587a27b14338cb3ddfcb65c2040e0590d347a43745590fb, gas=143056)"}
{"time":"2025-12-18T18:13:55.181269932Z","level":"INFO","msg":"🔑 KEY STEP: Starting resource teardown...","module":"main","usage_id":"lt-20251218-fe31"}
{"time":"2025-12-18T18:13:55.188859912Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","module":"main","endpoint":"http://ssv-mini-pr-2627-commit-e998ec6-orchestrator:8080"}
{"time":"2025-12-18T18:13:55.189027951Z","level":"INFO","msg":"🧹 Requesting resource teardown","module":"main","usage_id":"lt-20251218-fe31"}
{"time":"2025-12-18T18:13:55.20943121Z","level":"INFO","msg":"✅ SUCCESS: Completed resource teardown successfully","module":"main","usage_id":"lt-20251218-fe31"}

2025-12-18T18:13:58Z The test run failed with error

Result - 🔴 Failure

2025-12-18T18:29:02Z Finished cleaning reserved resources

vaclav-ssvlabs · 2025-12-18T22:26:20Z

/test network=local_testnet

ssv-test-bot · 2025-12-18T22:26:23Z

🟡 Test run for PR #2627

Branch: vaclav-test-aetheria-local-testnet
Commit: e998ec6

Status

2025-12-18T22:26:22Z Starting test run
2025-12-18T22:26:24Z Test run submitted successfully for commit e998ec6
2025-12-18T22:26:25Z container image for commit e998ec67cd5500a98f271ca5ac5fe5c8d02170f8 is build and present in container registry
2025-12-18T22:26:27Z allocated node range cluster=1,2,3,4 for test run on network=local_testnet
2025-12-18T22:26:28Z deploying ssv-mini with local testnet, this take aproximatelly 8-12 minutes

vaclav-ssvlabs · 2025-12-18T23:20:08Z

/test network=local_testnet

ssv-test-bot · 2025-12-18T23:20:11Z

🔴 Test run for PR #2627

Result - 🔴 Failure

2025-12-21T01:10:30Z Finished cleaning reserved resources

vaclav-ssvlabs · 2025-12-21T06:35:51Z

/test network=local_testnet

vaclav-ssvlabs · 2026-01-29T15:03:00Z

/test

ssv-test-bot · 2026-01-29T15:03:03Z

🔴 Test run for PR #2627

Branch: vaclav-test-aetheria-local-testnet
Commit: e5e11c1

Status

2026-01-29T15:03:03Z Starting test run
2026-01-29T15:03:05Z Test run submitted successfully for commit e5e11c1
2026-01-29T15:03:06Z container image for commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is build and present in container registry
2026-01-29T15:03:08Z automatically allocated node range cluster=300,301,302,303 for test run on network=hoodi_stage
2026-01-29T15:03:09Z deploying commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 to ssv nodes 300,301,302,303
2026-01-29T15:04:26Z commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is deployed to ssv nodes 300,301,302,303
2026-01-29T15:04:28Z started test executor with tests=(sanity) and network=hoodi_stage
2026-01-29T15:04:39Z The test run failed with error

Result - 🔴 Failure

Click to expand executor logs

═══════════════════════════════════════════════════════════════════════════════════════════════════════
                🧪  WELCOME TO Executor - SSV's Distributed Test Automation Execution Layer  🧪            

                                  ___ __  __ ___  ___  _   _  _____  ___   ___ 
                                 | __|\ \/ /| __|/ __|| | | ||_   _|/ _ \ | _ \
                                 | _|  >  < | _|| (__ | |_| |  | | | (_) ||   /
                                 |___|/_/\_\|___|\___| \___/   |_|  \___/ |_|_\
                                        

═══════════════════════════════════════════════════════════════════════════════════════════════════════
{"time":"2026-01-29T15:04:35.457202985Z","level":"INFO","msg":"🐳 Docker mode: Using environment variables for config generation"}
{"time":"2026-01-29T15:04:35.466466932Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","endpoint":"http://aetheria-orchestrator.aetheria.svc:8080"}
{"time":"2026-01-29T15:04:35.466610415Z","level":"INFO","msg":"📝 Requesting configuration generation from orchestrator API","network":"hoodi_stage","clusters":[300,301,302,303],"tests":"(sanity)"}
{"time":"2026-01-29T15:04:35.467037626Z","level":"INFO","msg":"⏳ Waiting for orchestrator to generate config (this may take a while)","timeout":"30m0s"}
{"time":"2026-01-29T15:04:35.501636159Z","level":"ERROR","msg":"❌ Failed to generate config via API in Docker mode","error":"failed to generate config from API: orchestrator API returned status code 500: {\"success\":false,\"error\":\"Config generation failed: no available cluster found for network hoodi_stage with cluster numbers [300 301 302 303]\"}\n"}
panic: Config generation failed in Docker mode - cannot fallback to file when environment variables are provided

goroutine 1 [running]:
github.com/ssvlabs/executor/bts.Init({0xade0f0, 0xe7f4a0})
	/app/bts/bootstrap.go:59 +0xbe5
main.main()
	/app/cmd/main.go:21 +0x30

vaclav-ssvlabs · 2026-01-29T15:09:34Z

/test

ssv-test-bot · 2026-01-29T15:09:39Z

🔴 Test run for PR #2627

Branch: vaclav-test-aetheria-local-testnet
Commit: e5e11c1

Status

2026-01-29T15:09:38Z Starting test run
2026-01-29T15:09:40Z Test run submitted successfully for commit e5e11c1
2026-01-29T15:09:41Z container image for commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is build and present in container registry
2026-01-29T15:09:42Z automatically allocated node range cluster=300,301,302,303 for test run on network=hoodi_stage
2026-01-29T15:09:44Z automatically allocated node range cluster=300,301,302,303 for test run on network=hoodi_stage
2026-01-29T15:09:45Z deploying commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 to ssv nodes 300,301,302,303
2026-01-29T15:09:47Z commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is deployed to ssv nodes 300,301,302,303
2026-01-29T15:09:49Z started test executor with tests=(sanity) and network=hoodi_stage
2026-01-29T15:10:10Z The test run failed with error

Result - 🔴 Failure

Click to expand executor logs

═══════════════════════════════════════════════════════════════════════════════════════════════════════
                🧪  WELCOME TO Executor - SSV's Distributed Test Automation Execution Layer  🧪            

                                  ___ __  __ ___  ___  _   _  _____  ___   ___ 
                                 | __|\ \/ /| __|/ __|| | | ||_   _|/ _ \ | _ \
                                 | _|  >  < | _|| (__ | |_| |  | | | (_) ||   /
                                 |___|/_/\_\|___|\___| \___/   |_|  \___/ |_|_\
                                        

═══════════════════════════════════════════════════════════════════════════════════════════════════════
{"time":"2026-01-29T15:09:56.240675454Z","level":"INFO","msg":"🐳 Docker mode: Using environment variables for config generation"}
{"time":"2026-01-29T15:09:56.252667867Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","endpoint":"http://aetheria-orchestrator.aetheria.svc:8080"}
{"time":"2026-01-29T15:09:56.252769409Z","level":"INFO","msg":"📝 Requesting configuration generation from orchestrator API","network":"hoodi_stage","clusters":[300,301,302,303],"tests":"(sanity)"}
{"time":"2026-01-29T15:09:56.252969037Z","level":"INFO","msg":"⏳ Waiting for orchestrator to generate config (this may take a while)","timeout":"30m0s"}
{"time":"2026-01-29T15:09:56.316772804Z","level":"INFO","msg":"✅ Configuration generated successfully via API","resource_usage_id":"f642adc5f0ef3d5ad520be40b723d1d9"}
{"time":"2026-01-29T15:09:56.318144201Z","level":"INFO","msg":"🚀 EXECUTOR Initialized Successfully"}
{"time":"2026-01-29T15:09:56.318201034Z","level":"INFO","msg":"⚙️ Test configuration loaded","network":"hoodi_stage","nodes":[300,301,302,303],"test_type":{"Sanity":{"ShouldRun":true,"E2M":false,"Threshold":false,"BulkAmount":0},"Bulk":{"ShouldRun":false,"E2M":false,"Threshold":false,"BulkAmount":0},"Isolated":{"Tests":null}}}
{"time":"2026-01-29T15:09:56.318514429Z","level":"INFO","msg":"Operator pubkeys file not found, generating with go-ssv-scanner","module":"scanner"}
{"time":"2026-01-29T15:09:56.716360124Z","level":"INFO","msg":"Generated operator pubkeys file","module":"scanner","path":"/app/shared/data/operators/operator-pubkeys-hoodi_stage.json","count":501}
{"time":"2026-01-29T15:09:56.726671278Z","level":"INFO","msg":"Loaded operator pubkeys into cache","module":"scanner","count":501}
{"time":"2026-01-29T15:09:56.727074399Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"consensus"}
{"time":"2026-01-29T15:09:56.72716101Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"flow-emits"}
{"time":"2026-01-29T15:09:56.72723687Z","level":"INFO","msg":"Validation client initialized","module":"main","loki_enabled":true,"e2m_enabled":false}
{"time":"2026-01-29T15:09:56.727294714Z","level":"INFO","msg":"🔑 KEY STEP: Starting Isolated Tests Concurrently if exists...","module":"main"}
{"time":"2026-01-29T15:09:56.727372123Z","level":"INFO","msg":"🔑 KEY STEP: Starting Sanity Test Suite","module":"main"}
{"time":"2026-01-29T15:09:56.727583264Z","level":"INFO","msg":"No isolated tests configured, skipping","module":"main"}
{"time":"2026-01-29T15:09:56.773464358Z","level":"INFO","msg":"EXECUTOR WALLET ADDRESS - Cluster contract initialized","test":"sanity","module":"contract","executor_wallet_address":"0x91E32eFb8139cd88caE0Df30d2Bf471294c6ed27","network_contract":"0x0AaACE4e8AFFC47C6834171c88D342A4aBD8f105","views_contract":"0x9143b8C25EfA53F28de4cBeFd0b6dFd66D43FeA6","token_contract":"0x746C33ccC28b1363c35c09baDAF41b2FFa7E6D56"}
{"time":"2026-01-29T15:09:56.776739325Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":300}
{"time":"2026-01-29T15:09:56.787760364Z","level":"INFO","msg":"Available slots","test":"sanity","node":300,"slots":2999}
{"time":"2026-01-29T15:09:56.787872463Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":301}
{"time":"2026-01-29T15:09:56.795650817Z","level":"INFO","msg":"Available slots","test":"sanity","node":301,"slots":2999}
{"time":"2026-01-29T15:09:56.795685408Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":302}
{"time":"2026-01-29T15:09:56.802897873Z","level":"INFO","msg":"Available slots","test":"sanity","node":302,"slots":2999}
{"time":"2026-01-29T15:09:56.802924423Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":303}
{"time":"2026-01-29T15:09:56.8103426Z","level":"INFO","msg":"Available slots","test":"sanity","node":303,"slots":2999}
{"time":"2026-01-29T15:10:05.567984527Z","level":"INFO","msg":"Extracted keyshares path","test":"sanity","module":"keys","path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json"}
{"time":"2026-01-29T15:10:05.568065499Z","level":"INFO","msg":"Key shares file generated successfully","test":"sanity","keyshare_path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json"}
{"time":"2026-01-29T15:10:06.622553703Z","level":"WARN","msg":"Validator already registered; removing before registration","test":"sanity","validator_pubkey":"0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27"}
{"time":"2026-01-29T15:10:06.636244043Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0x91e32efb8139cd88cae0df30d2bf471294c6ed27","allowance":"99051281939116557600000000"}
{"time":"2026-01-29T15:10:06.636398317Z","level":"INFO","msg":"Threshold value","test":"sanity","module":"contract","threshold":"9000000000000000000000"}
{"time":"2026-01-29T15:10:06.636520523Z","level":"INFO","msg":"Sufficient allowance","test":"sanity","module":"contract","allowance margin":"6561000000000000000000000000000000000000000000000000000000000000000000000000000000000000"}
{"time":"2026-01-29T15:10:06.697970403Z","level":"ERROR","msg":"Failed to remove validator","test":"sanity","module":"contract","error":"execution reverted"}
{"time":"2026-01-29T15:10:06.698104742Z","level":"ERROR","msg":"❌ FAILURE: Sanity Test Suite Failed","module":"main","error":"execution reverted"}
{"time":"2026-01-29T15:10:06.698127639Z","level":"INFO","msg":"⏳ PROGRESS: Waiting for all tests to finish","module":"main"}
{"time":"2026-01-29T15:10:06.698195839Z","level":"ERROR","msg":"❌ FAILURE: Test Execution Completed with Failures","module":"main","passed_suites":null,"failed_suites":["sanity"],"skipped_suites":["isolated"]}
{"time":"2026-01-29T15:10:06.698244348Z","level":"ERROR","msg":"❌ FAILURE: Suite Failure Details","module":"main","suite":"sanity","error":"execution reverted"}
{"time":"2026-01-29T15:10:06.698267163Z","level":"INFO","msg":"🔑 KEY STEP: Starting resource teardown...","module":"main","usage_id":"f642adc5f0ef3d5ad520be40b723d1d9"}
{"time":"2026-01-29T15:10:06.699720402Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","module":"main","endpoint":"http://aetheria-orchestrator.aetheria.svc:8080"}
{"time":"2026-01-29T15:10:06.699863475Z","level":"INFO","msg":"🧹 Requesting resource teardown","module":"main","usage_id":"f642adc5f0ef3d5ad520be40b723d1d9"}
{"time":"2026-01-29T15:10:06.767844969Z","level":"INFO","msg":"✅ SUCCESS: Completed resource teardown successfully","module":"main","usage_id":"f642adc5f0ef3d5ad520be40b723d1d9"}

2026-01-29T15:25:14Z Finished cleaning reserved resources

vaclav-ssvlabs · 2026-01-29T16:00:29Z

/test

ssv-test-bot · 2026-01-29T16:00:33Z

🔴 Test run for PR #2627

Branch: vaclav-test-aetheria-local-testnet
Commit: e5e11c1

Status

2026-01-29T16:00:33Z Starting test run
2026-01-29T16:00:34Z Test run submitted successfully for commit e5e11c1
2026-01-29T16:00:35Z container image for commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is build and present in container registry
2026-01-29T16:00:37Z automatically allocated node range cluster=300,301,302,303 for test run on network=hoodi_stage
2026-01-29T16:00:38Z deploying commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 to ssv nodes 300,301,302,303
2026-01-29T16:01:25Z commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is deployed to ssv nodes 300,301,302,303
2026-01-29T16:01:27Z started test executor with tests=(sanity) and network=hoodi_stage
2026-01-29T16:01:39Z The test run failed with error

Result - 🔴 Failure

Click to expand executor logs

{"time":"2026-01-29T16:01:35.07244459Z","level":"INFO","msg":"🐳 Docker mode: Using environment variables for config generation"}
═══════════════════════════════════════════════════════════════════════════════════════════════════════
                🧪  WELCOME TO Executor - SSV's Distributed Test Automation Execution Layer  🧪            

                                  ___ __  __ ___  ___  _   _  _____  ___   ___ 
                                 | __|\ \/ /| __|/ __|| | | ||_   _|/ _ \ | _ \
                                 | _|  >  < | _|| (__ | |_| |  | | | (_) ||   /
                                 |___|/_/\_\|___|\___| \___/   |_|  \___/ |_|_\
                                        

═══════════════════════════════════════════════════════════════════════════════════════════════════════
{"time":"2026-01-29T16:01:35.082786777Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","endpoint":"http://aetheria-orchestrator.aetheria.svc:8080"}
{"time":"2026-01-29T16:01:35.082835702Z","level":"INFO","msg":"📝 Requesting configuration generation from orchestrator API","network":"hoodi_stage","clusters":[300,301,302,303],"tests":"(sanity)"}
{"time":"2026-01-29T16:01:35.083089498Z","level":"INFO","msg":"⏳ Waiting for orchestrator to generate config (this may take a while)","timeout":"30m0s"}
{"time":"2026-01-29T16:01:35.114550149Z","level":"ERROR","msg":"❌ Failed to generate config via API in Docker mode","error":"failed to generate config from API: orchestrator API returned status code 500: {\"success\":false,\"error\":\"Config generation failed: no available cluster found for network hoodi_stage with cluster numbers [300 301 302 303]\"}\n"}
panic: Config generation failed in Docker mode - cannot fallback to file when environment variables are provided

goroutine 1 [running]:
github.com/ssvlabs/executor/bts.Init({0xadf2f0, 0xe814a0})
	/app/bts/bootstrap.go:59 +0xbe5
main.main()
	/app/cmd/main.go:21 +0x30

vaclav-ssvlabs · 2026-01-29T16:03:39Z

/test

ssv-test-bot · 2026-01-29T16:03:42Z

❌ Failed to start test run:

test run stage-executor-ssv-pr-2627-commit-e5e11c1 is currently running (sync: Synced, health: Degraded): a test is already running or cleaning up for this pull request: Please wait for the existing test to complete or be cleaned up before starting a new one.

vaclav-ssvlabs · 2026-01-29T16:58:29Z

/test

ssv-test-bot · 2026-01-29T16:58:32Z

🔴 Test run for PR #2627

Branch: vaclav-test-aetheria-local-testnet
Commit: e5e11c1

Status

2026-01-29T16:58:32Z Starting test run
2026-01-29T16:58:33Z Test run submitted successfully for commit e5e11c1
2026-01-29T16:58:34Z container image for commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is build and present in container registry
2026-01-29T16:58:36Z automatically allocated node range cluster=300,301,302,303 for test run on network=hoodi_stage
2026-01-29T16:58:38Z deploying commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 to ssv nodes 300,301,302,303
2026-01-29T16:59:40Z commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is deployed to ssv nodes 300,301,302,303
2026-01-29T16:59:41Z started test executor with tests=(sanity) and network=hoodi_stage
2026-01-29T16:59:52Z The test run failed with error

Result - 🔴 Failure

Click to expand executor logs

═══════════════════════════════════════════════════════════════════════════════════════════════════════
                🧪  WELCOME TO Executor - SSV's Distributed Test Automation Execution Layer  🧪            

                                  ___ __  __ ___  ___  _   _  _____  ___   ___ 
                                 | __|\ \/ /| __|/ __|| | | ||_   _|/ _ \ | _ \
                                 | _|  >  < | _|| (__ | |_| |  | | | (_) ||   /
                                 |___|/_/\_\|___|\___| \___/   |_|  \___/ |_|_\
                                        

═══════════════════════════════════════════════════════════════════════════════════════════════════════
{"time":"2026-01-29T16:59:48.565341263Z","level":"INFO","msg":"🐳 Docker mode: Using environment variables for config generation"}
{"time":"2026-01-29T16:59:48.573538442Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","endpoint":"http://aetheria-orchestrator.aetheria.svc:8080"}
{"time":"2026-01-29T16:59:48.573654411Z","level":"INFO","msg":"📝 Requesting configuration generation from orchestrator API","network":"hoodi_stage","clusters":[300,301,302,303],"tests":"(sanity)"}
{"time":"2026-01-29T16:59:48.573861277Z","level":"INFO","msg":"⏳ Waiting for orchestrator to generate config (this may take a while)","timeout":"30m0s"}
{"time":"2026-01-29T16:59:48.689173452Z","level":"INFO","msg":"✅ Configuration generated successfully via API","resource_usage_id":"0951af950c71b04577f4a4c00bbb3e3d"}
{"time":"2026-01-29T16:59:48.690783304Z","level":"INFO","msg":"🚀 EXECUTOR Initialized Successfully"}
{"time":"2026-01-29T16:59:48.690863472Z","level":"INFO","msg":"⚙️ Test configuration loaded","network":"hoodi_stage","nodes":[300,301,302,303],"test_type":{"Sanity":{"ShouldRun":true,"E2M":false,"Threshold":false,"BulkAmount":0},"Bulk":{"ShouldRun":false,"E2M":false,"Threshold":false,"BulkAmount":0},"Isolated":{"Tests":null}}}
{"time":"2026-01-29T16:59:48.691409058Z","level":"INFO","msg":"Operator pubkeys file not found, generating with go-ssv-scanner","module":"scanner"}
{"time":"2026-01-29T16:59:49.175020507Z","level":"INFO","msg":"Generated operator pubkeys file","module":"scanner","path":"/app/shared/data/operators/operator-pubkeys-hoodi_stage.json","count":504}
{"time":"2026-01-29T16:59:49.181897286Z","level":"INFO","msg":"Loaded operator pubkeys into cache","module":"scanner","count":504}
{"time":"2026-01-29T16:59:49.182025788Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"consensus"}
{"time":"2026-01-29T16:59:49.182049604Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"flow-emits"}
{"time":"2026-01-29T16:59:49.182081139Z","level":"INFO","msg":"Validation client initialized","module":"main","loki_enabled":true,"e2m_enabled":false}
{"time":"2026-01-29T16:59:49.182108004Z","level":"INFO","msg":"🔑 KEY STEP: Starting Isolated Tests Concurrently if exists...","module":"main"}
{"time":"2026-01-29T16:59:49.182147308Z","level":"INFO","msg":"🔑 KEY STEP: Starting Sanity Test Suite","module":"main"}
{"time":"2026-01-29T16:59:49.182292183Z","level":"INFO","msg":"No isolated tests configured, skipping","module":"main"}
{"time":"2026-01-29T16:59:49.218814207Z","level":"INFO","msg":"EXECUTOR WALLET ADDRESS - Cluster contract initialized","test":"sanity","module":"contract","executor_wallet_address":"0x91E32eFb8139cd88caE0Df30d2Bf471294c6ed27","network_contract":"0x0AaACE4e8AFFC47C6834171c88D342A4aBD8f105","views_contract":"0x9143b8C25EfA53F28de4cBeFd0b6dFd66D43FeA6","token_contract":"0x746C33ccC28b1363c35c09baDAF41b2FFa7E6D56"}
{"time":"2026-01-29T16:59:49.219765797Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":300}
{"time":"2026-01-29T16:59:49.228217451Z","level":"ERROR","msg":"❌ FAILURE: Sanity Test Suite Failed","module":"main","error":"operator 300 is not active on-chain"}
{"time":"2026-01-29T16:59:49.228322969Z","level":"INFO","msg":"⏳ PROGRESS: Waiting for all tests to finish","module":"main"}
{"time":"2026-01-29T16:59:49.228420783Z","level":"ERROR","msg":"❌ FAILURE: Test Execution Completed with Failures","module":"main","passed_suites":null,"failed_suites":["sanity"],"skipped_suites":["isolated"]}
{"time":"2026-01-29T16:59:49.228501706Z","level":"ERROR","msg":"❌ FAILURE: Suite Failure Details","module":"main","suite":"sanity","error":"operator 300 is not active on-chain"}
{"time":"2026-01-29T16:59:49.228554302Z","level":"INFO","msg":"🔑 KEY STEP: Starting resource teardown...","module":"main","usage_id":"0951af950c71b04577f4a4c00bbb3e3d"}
{"time":"2026-01-29T16:59:49.229721905Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","module":"main","endpoint":"http://aetheria-orchestrator.aetheria.svc:8080"}
{"time":"2026-01-29T16:59:49.229795186Z","level":"INFO","msg":"🧹 Requesting resource teardown","module":"main","usage_id":"0951af950c71b04577f4a4c00bbb3e3d"}
{"time":"2026-01-29T16:59:49.275705965Z","level":"INFO","msg":"✅ SUCCESS: Completed resource teardown successfully","module":"main","usage_id":"0951af950c71b04577f4a4c00bbb3e3d"}

2026-01-29T17:14:57Z Finished cleaning reserved resources

vaclav-ssvlabs · 2026-01-30T09:08:33Z

/test cluster=304,305,306,307

ssv-test-bot · 2026-01-30T09:08:36Z

🔴 Test run for PR #2627

Branch: vaclav-test-aetheria-local-testnet
Commit: e5e11c1

Status

2026-01-30T09:08:36Z Starting test run
2026-01-30T09:08:38Z Test run submitted successfully for commit e5e11c1
2026-01-30T09:08:39Z container image for commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is build and present in container registry
2026-01-30T09:08:40Z using user defined node range cluster=304,305,306,307 on network=hoodi_stage
2026-01-30T09:08:41Z deploying commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 to ssv nodes 304,305,306,307
2026-01-30T09:09:43Z commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is deployed to ssv nodes 304,305,306,307
2026-01-30T09:09:44Z started test executor with tests=(sanity) and network=hoodi_stage

Click to expand executor logs

═══════════════════════════════════════════════════════════════════════════════════════════════════════
                🧪  WELCOME TO Executor - SSV's Distributed Test Automation Execution Layer  🧪            

                                  ___ __  __ ___  ___  _   _  _____  ___   ___ 
                                 | __|\\ \\/ /| __|/ __|| | | ||_   _|/ _ \\ | _ \\
                                 | _|  >  < | _|| (__ | |_| |  | | | (_) ||   /
                                 |___|/_/\\_\\|___|\\___| \\___/   |_|  \\___/ |_|_\\
                                        

═══════════════════════════════════════════════════════════════════════════════════════════════════════
{"time":"2026-01-30T09:10:16.69959884Z","level":"INFO","msg":"🐳 Docker mode: Using environment variables for config generation"}
{"time":"2026-01-30T09:10:16.708569847Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","endpoint":"http://aetheria-orchestrator.aetheria.svc:8080"}
{"time":"2026-01-30T09:10:16.708647429Z","level":"INFO","msg":"📝 Requesting configuration generation from orchestrator API","network":"hoodi_stage","clusters":[304,305,306,307],"tests":"(sanity)"}
{"time":"2026-01-30T09:10:16.814728091Z","level":"INFO","msg":"✅ Configuration generated successfully via API","resource_usage_id":"53b835beb91b604db7757b9a7bfe46ae"}
{"time":"2026-01-30T09:10:16.817308238Z","level":"INFO","msg":"🚀 EXECUTOR Initialized Successfully"}
{"time":"2026-01-30T09:10:16.817390711Z","level":"INFO","msg":"⚙️ Test configuration loaded","network":"hoodi_stage","nodes":[304,305,306,307],"test_type":{"Sanity":{"ShouldRun":true,"E2M":false,"Threshold":false,"BulkAmount":0},"Bulk":{"ShouldRun":false,"E2M":false,"Threshold":false,"BulkAmount":0},"Isolated":{"Tests":null}}}
{"time":"2026-01-30T09:10:21.580433512Z","level":"INFO","msg":"Generated operator pubkeys file","module":"scanner","path":"/app/shared/data/operators/operator-pubkeys-hoodi_stage.json"}
{"time":"2026-01-30T09:10:21.580607829Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"consensus"}
{"time":"2026-01-30T09:10:21.580639607Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"flow-emits"}
{"time":"2026-01-30T09:10:21.580703314Z","level":"INFO","msg":"Validation client initialized","module":"main","loki_enabled":true,"e2m_enabled":false}
{"time":"2026-01-30T09:10:21.580739879Z","level":"INFO","msg":"🔑 KEY STEP: Starting Isolated Tests Concurrently if exists...","module":"main"}
{"time":"2026-01-30T09:10:21.580816865Z","level":"INFO","msg":"🔑 KEY STEP: Starting Sanity Test Suite","module":"main"}
{"time":"2026-01-30T09:10:21.581019332Z","level":"INFO","msg":"No isolated tests configured, skipping","module":"main"}
{"time":"2026-01-30T09:10:21.644958202Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":304}
{"time":"2026-01-30T09:10:21.655403309Z","level":"INFO","msg":"Available slots","test":"sanity","node":304,"slots":3000}
{"time":"2026-01-30T09:10:21.655486062Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":305}
{"time":"2026-01-30T09:10:21.662708034Z","level":"INFO","msg":"Available slots","test":"sanity","node":305,"slots":3000}
{"time":"2026-01-30T09:10:21.662803472Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":306}
{"time":"2026-01-30T09:10:21.669579032Z","level":"INFO","msg":"Available slots","test":"sanity","node":306,"slots":3000}
{"time":"2026-01-30T09:10:21.669646218Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":307}
{"time":"2026-01-30T09:10:21.67679178Z","level":"INFO","msg":"Available slots","test":"sanity","node":307,"slots":3000}
{"time":"2026-01-30T09:14:27.588414406Z","level":"INFO","msg":"Extracted keyshares path","test":"sanity","module":"keys","path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json"}
{"time":"2026-01-30T09:14:27.588538302Z","level":"INFO","msg":"Key shares file generated successfully","test":"sanity","keyshare_path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json"}
{"time":"2026-01-30T09:14:34.628638254Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0x5cc0dde14e7256340cc820415a6022a7d1c93a35","allowance":"9999999999999999999999473850789666515400000000"}
{"time":"2026-01-30T09:14:34.628756817Z","level":"INFO","msg":"Threshold value","test":"sanity","module":"contract","threshold":"9000000000000000000000"}
{"time":"2026-01-30T09:14:34.628800193Z","level":"INFO","msg":"Sufficient allowance","test":"sanity","module":"contract","allowance margin":"6561000000000000000000000000000000000000000000000000000000000000000000000000000000000000"}
{"time":"2026-01-30T09:14:34.632668548Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0x5cc0dde14e7256340cc820415a6022a7d1c93a35","allowance":"9999999999999999999999473850789666515400000000"}
{"time":"2026-01-30T09:14:34.632742031Z","level":"INFO","msg":"Allowance","test":"sanity","module":"contract","allowance":"9999999999999999999999473850789666515400000000"}
{"time":"2026-01-30T09:14:34.687469191Z","level":"ERROR","msg":"Failed to register validator","test":"sanity","module":"contract","error":"execution reverted"}
{"time":"2026-01-30T09:14:34.687624664Z","level":"ERROR","msg":"❌ FAILURE: Sanity Test Suite Failed","module":"main","error":"execution reverted"}
{"time":"2026-01-30T09:14:34.687647605Z","level":"INFO","msg":"⏳ PROGRESS: Waiting for all tests to finish","module":"main"}
{"time":"2026-01-30T09:14:34.687868731Z","level":"ERROR","msg":"❌ FAILURE: Test Execution Completed with Failures","module":"main","passed_suites":null,"failed_suites":["sanity"],"skipped_suites":["isolated"]}
{"time":"2026-01-30T09:14:34.687961341Z","level":"ERROR","msg":"❌ FAILURE: Suite Failure Details","module":"main","suite":"sanity","error":"execution reverted"}
{"time":"2026-01-30T09:14:34.687995898Z","level":"INFO","msg":"🔑 KEY STEP: Starting resource teardown...","module":"main","usage_id":"53b835beb91b604db7757b9a7bfe46ae"}
{"time":"2026-01-30T09:14:34.697026785Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","module":"main","endpoint":"http://aetheria-orchestrator.aetheria.svc:8080"}
{"time":"2026-01-30T09:14:34.697134323Z","level":"INFO","msg":"🧹 Requesting resource teardown","module":"main","usage_id":"53b835beb91b604db7757b9a7bfe46ae"}
{"time":"2026-01-30T09:14:34.731745618Z","level":"INFO","msg":"✅ SUCCESS: Completed resource teardown successfully","module":"main","usage_id":"53b835beb91b604db7757b9a7bfe46ae"}

2026-01-30T09:14:38Z The test run failed with error

Result - 🔴 Failure

2026-01-30T09:29:42Z Finished cleaning reserved resources

vaclav-ssvlabs · 2026-01-30T10:35:32Z

/test cluster=304,305,306,307

ssv-test-bot · 2026-01-30T10:35:36Z

🔴 Test run for PR #2627

Branch: vaclav-test-aetheria-local-testnet
Commit: e5e11c1

Status

2026-01-30T10:35:35Z Starting test run
2026-01-30T10:35:36Z Test run submitted successfully for commit e5e11c1
2026-01-30T10:35:37Z container image for commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is build and present in container registry
2026-01-30T10:35:39Z using user defined node range cluster=304,305,306,307 on network=hoodi_stage
2026-01-30T10:35:40Z deploying commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 to ssv nodes 304,305,306,307
2026-01-30T10:36:42Z commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is deployed to ssv nodes 304,305,306,307
2026-01-30T10:36:43Z started test executor with tests=(sanity) and network=hoodi_stage

Click to expand executor logs

{"time":"2026-01-30T10:37:16.754790639Z","level":"INFO","msg":"🐳 Docker mode: Using environment variables for config generation"}
═══════════════════════════════════════════════════════════════════════════════════════════════════════
                🧪  WELCOME TO Executor - SSV's Distributed Test Automation Execution Layer  🧪            

                                  ___ __  __ ___  ___  _   _  _____  ___   ___ 
                                 | __|\\ \\/ /| __|/ __|| | | ||_   _|/ _ \\ | _ \\
                                 | _|  >  < | _|| (__ | |_| |  | | | (_) ||   /
                                 |___|/_/\\_\\|___|\\___| \\___/   |_|  \\___/ |_|_\\
                                        

═══════════════════════════════════════════════════════════════════════════════════════════════════════
{"time":"2026-01-30T10:37:16.772926425Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","endpoint":"http://aetheria-orchestrator.aetheria.svc:8080"}
{"time":"2026-01-30T10:37:16.773019369Z","level":"INFO","msg":"📝 Requesting configuration generation from orchestrator API","network":"hoodi_stage","clusters":[304,305,306,307],"tests":"(sanity)"}
{"time":"2026-01-30T10:37:16.850614969Z","level":"INFO","msg":"✅ Configuration generated successfully via API","resource_usage_id":"00b67332290b9ec56b44eb42463f686f"}
{"time":"2026-01-30T10:37:16.852431941Z","level":"INFO","msg":"🚀 EXECUTOR Initialized Successfully"}
{"time":"2026-01-30T10:37:16.852516169Z","level":"INFO","msg":"⚙️ Test configuration loaded","network":"hoodi_stage","nodes":[304,305,306,307],"test_type":{"Sanity":{"ShouldRun":true,"E2M":false,"Threshold":false,"BulkAmount":0},"Bulk":{"ShouldRun":false,"E2M":false,"Threshold":false,"BulkAmount":0},"Isolated":{"Tests":null}}}
{"time":"2026-01-30T10:37:21.475821877Z","level":"INFO","msg":"Generated operator pubkeys file","module":"scanner","path":"/app/shared/data/operators/operator-pubkeys-hoodi_stage.json"}
{"time":"2026-01-30T10:37:21.478061757Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"consensus"}
{"time":"2026-01-30T10:37:21.478116027Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"flow-emits"}
{"time":"2026-01-30T10:37:21.478141452Z","level":"INFO","msg":"Validation client initialized","module":"main","loki_enabled":true,"e2m_enabled":false}
{"time":"2026-01-30T10:37:21.478190504Z","level":"INFO","msg":"🔑 KEY STEP: Starting Isolated Tests Concurrently if exists...","module":"main"}
{"time":"2026-01-30T10:37:21.478249558Z","level":"INFO","msg":"🔑 KEY STEP: Starting Sanity Test Suite","module":"main"}
{"time":"2026-01-30T10:37:21.478487885Z","level":"INFO","msg":"No isolated tests configured, skipping","module":"main"}
{"time":"2026-01-30T10:37:21.541793931Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":304}
{"time":"2026-01-30T10:37:21.558784959Z","level":"INFO","msg":"Available slots","test":"sanity","node":304,"slots":3000}
{"time":"2026-01-30T10:37:21.558892147Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":305}
{"time":"2026-01-30T10:37:21.567359091Z","level":"INFO","msg":"Available slots","test":"sanity","node":305,"slots":3000}
{"time":"2026-01-30T10:37:21.567439702Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":306}
{"time":"2026-01-30T10:37:21.575473276Z","level":"INFO","msg":"Available slots","test":"sanity","node":306,"slots":3000}
{"time":"2026-01-30T10:37:21.575579387Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":307}
{"time":"2026-01-30T10:37:21.584628534Z","level":"INFO","msg":"Available slots","test":"sanity","node":307,"slots":3000}
{"time":"2026-01-30T10:42:16.945780866Z","level":"INFO","msg":"Extracted keyshares path","test":"sanity","module":"keys","path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json"}
{"time":"2026-01-30T10:42:16.9459329Z","level":"INFO","msg":"Key shares file generated successfully","test":"sanity","keyshare_path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json"}
{"time":"2026-01-30T10:42:23.286012136Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0x5cc0dde14e7256340cc820415a6022a7d1c93a35","allowance":"9999999999999999999999498323694910796200000000"}
{"time":"2026-01-30T10:42:23.286200717Z","level":"INFO","msg":"Threshold value","test":"sanity","module":"contract","threshold":"9000000000000000000000"}
{"time":"2026-01-30T10:42:23.286246554Z","level":"INFO","msg":"Sufficient allowance","test":"sanity","module":"contract","allowance margin":"6561000000000000000000000000000000000000000000000000000000000000000000000000000000000000"}
{"time":"2026-01-30T10:42:23.29067619Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0x5cc0dde14e7256340cc820415a6022a7d1c93a35","allowance":"9999999999999999999999498323694910796200000000"}
{"time":"2026-01-30T10:42:23.290787403Z","level":"INFO","msg":"Allowance","test":"sanity","module":"contract","allowance":"9999999999999999999999498323694910796200000000"}
{"time":"2026-01-30T10:42:23.417932441Z","level":"ERROR","msg":"Failed to register validator","test":"sanity","module":"contract","error":"execution reverted"}
{"time":"2026-01-30T10:42:23.418174381Z","level":"ERROR","msg":"❌ FAILURE: Sanity Test Suite Failed","module":"main","error":"execution reverted"}
{"time":"2026-01-30T10:42:23.418239499Z","level":"INFO","msg":"⏳ PROGRESS: Waiting for all tests to finish","module":"main"}
{"time":"2026-01-30T10:42:23.418337509Z","level":"ERROR","msg":"❌ FAILURE: Test Execution Completed with Failures","module":"main","passed_suites":null,"failed_suites":["sanity"],"skipped_suites":["isolated"]}
{"time":"2026-01-30T10:42:23.418400077Z","level":"ERROR","msg":"❌ FAILURE: Suite Failure Details","module":"main","suite":"sanity","error":"execution reverted"}
{"time":"2026-01-30T10:42:23.418447038Z","level":"INFO","msg":"🔑 KEY STEP: Starting resource teardown...","module":"main","usage_id":"00b67332290b9ec56b44eb42463f686f"}
{"time":"2026-01-30T10:42:23.433398255Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","module":"main","endpoint":"http://aetheria-orchestrator.aetheria.svc:8080"}
{"time":"2026-01-30T10:42:23.433609323Z","level":"INFO","msg":"🧹 Requesting resource teardown","module":"main","usage_id":"00b67332290b9ec56b44eb42463f686f"}
{"time":"2026-01-30T10:42:23.486159371Z","level":"INFO","msg":"✅ SUCCESS: Completed resource teardown successfully","module":"main","usage_id":"00b67332290b9ec56b44eb42463f686f"}

2026-01-30T10:42:27Z The test run failed with error

Result - 🔴 Failure

vaclav-ssvlabs · 2026-01-30T10:45:33Z

/test

ssv-test-bot · 2026-01-30T10:45:36Z

❌ Failed to start test run:

test run stage-executor-ssv-pr-2627-commit-e5e11c1 is currently running (sync: Synced, health: Degraded): a test is already running or cleaning up for this pull request: Please wait for the existing test to complete or be cleaned up before starting a new one.

vaclav-ssvlabs · 2026-01-30T11:54:19Z

/test cluster=304,305,306,307

ssv-test-bot · 2026-01-30T11:54:23Z

🔴 Test run for PR #2627

Branch: vaclav-test-aetheria-local-testnet
Commit: e5e11c1

Status

2026-01-30T11:54:22Z Starting test run
2026-01-30T11:54:24Z Test run submitted successfully for commit e5e11c1
2026-01-30T11:54:25Z container image for commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is build and present in container registry
2026-01-30T11:54:26Z using user defined node range cluster=304,305,306,307 on network=hoodi_stage
2026-01-30T11:54:27Z deploying commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 to ssv nodes 304,305,306,307
2026-01-30T11:55:29Z commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is deployed to ssv nodes 304,305,306,307
2026-01-30T11:55:31Z started test executor with tests=(sanity) and network=hoodi_stage

Click to expand executor logs

═══════════════════════════════════════════════════════════════════════════════════════════════════════
                🧪  WELCOME TO Executor - SSV's Distributed Test Automation Execution Layer  🧪            

                                  ___ __  __ ___  ___  _   _  _____  ___   ___ 
                                 | __|\\ \\/ /| __|/ __|| | | ||_   _|/ _ \\ | _ \\
                                 | _|  >  < | _|| (__ | |_| |  | | | (_) ||   /
                                 |___|/_/\\_\\|___|\\___| \\___/   |_|  \\___/ |_|_\\
                                        

═══════════════════════════════════════════════════════════════════════════════════════════════════════
{"time":"2026-01-30T11:55:38.212399454Z","level":"INFO","msg":"🐳 Docker mode: Using environment variables for config generation"}
{"time":"2026-01-30T11:55:38.224784171Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","endpoint":"http://aetheria-orchestrator.aetheria.svc:8080"}
{"time":"2026-01-30T11:55:38.225004568Z","level":"INFO","msg":"📝 Requesting configuration generation from orchestrator API","network":"hoodi_stage","clusters":[304,305,306,307],"tests":"(sanity)"}
{"time":"2026-01-30T11:55:38.332057379Z","level":"INFO","msg":"✅ Configuration generated successfully via API","resource_usage_id":"96dc396adcfaeb31d5e42d2754ebdf35"}
{"time":"2026-01-30T11:55:38.333692931Z","level":"INFO","msg":"🚀 EXECUTOR Initialized Successfully"}
{"time":"2026-01-30T11:55:38.333778593Z","level":"INFO","msg":"⚙️ Test configuration loaded","network":"hoodi_stage","nodes":[304,305,306,307],"test_type":{"Sanity":{"ShouldRun":true,"E2M":false,"Threshold":false,"BulkAmount":0},"Bulk":{"ShouldRun":false,"E2M":false,"Threshold":false,"BulkAmount":0},"Isolated":{"Tests":null}}}
{"time":"2026-01-30T11:55:43.118123086Z","level":"INFO","msg":"Generated operator pubkeys file","module":"scanner","path":"/app/shared/data/operators/operator-pubkeys-hoodi_stage.json"}
{"time":"2026-01-30T11:55:43.118426113Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"consensus"}
{"time":"2026-01-30T11:55:43.118460919Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"flow-emits"}
{"time":"2026-01-30T11:55:43.118505648Z","level":"INFO","msg":"Validation client initialized","module":"main","loki_enabled":true,"e2m_enabled":false}
{"time":"2026-01-30T11:55:43.118627166Z","level":"INFO","msg":"🔑 KEY STEP: Starting Isolated Tests Concurrently if exists...","module":"main"}
{"time":"2026-01-30T11:55:43.118709227Z","level":"INFO","msg":"🔑 KEY STEP: Starting Sanity Test Suite","module":"main"}
{"time":"2026-01-30T11:55:43.118887286Z","level":"INFO","msg":"No isolated tests configured, skipping","module":"main"}
{"time":"2026-01-30T11:55:43.185323248Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":304}
{"time":"2026-01-30T11:55:43.196755068Z","level":"INFO","msg":"Available slots","test":"sanity","node":304,"slots":3000}
{"time":"2026-01-30T11:55:43.196837859Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":305}
{"time":"2026-01-30T11:55:43.207503763Z","level":"INFO","msg":"Available slots","test":"sanity","node":305,"slots":3000}
{"time":"2026-01-30T11:55:43.207571692Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":306}
{"time":"2026-01-30T11:55:43.213453975Z","level":"INFO","msg":"Available slots","test":"sanity","node":306,"slots":3000}
{"time":"2026-01-30T11:55:43.213535074Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":307}
{"time":"2026-01-30T11:55:43.219625208Z","level":"INFO","msg":"Available slots","test":"sanity","node":307,"slots":3000}
{"time":"2026-01-30T11:59:54.790851253Z","level":"INFO","msg":"Extracted keyshares path","test":"sanity","module":"keys","path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json"}
{"time":"2026-01-30T11:59:54.791030928Z","level":"INFO","msg":"Key shares file generated successfully","test":"sanity","keyshare_path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json"}
{"time":"2026-01-30T12:00:01.50386034Z","level":"INFO","msg":"DIAGNOSTIC: Cluster snapshot (pre-registration)","test":"sanity","operator_ids":[304,305,306,307],"cluster_validator_count":1,"cluster_network_fee_index":70469584352,"cluster_index":772007289500,"cluster_active":true,"cluster_balance":"8999999865310720000000"}
{"time":"2026-01-30T12:00:01.504068571Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator amount calculation","test":"sanity","amount_per_validator":"2500000000000000000","validator_count":1,"required_amount":"2500000000000000000","current_cluster_balance":"8999999865310720000000","missing_amount":"-8997499865310720000000","missing_amount_rounded_1e8":"-8997499865310700000000","register_amount":"0"}
{"time":"2026-01-30T12:00:01.504315478Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator inputs prepared","test":"sanity","keyshare_path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json","validator_pubkey":"0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27","shares_data_len_chars":2626,"operator_ids":[304,305,306,307],"amount":"0"}
{"time":"2026-01-30T12:00:01.528431885Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator context","test":"sanity","module":"contract","op":"register_validator","from":"0x5cC0DdE14E7256340CC820415a6022a7d1c93A35","network_contract":"0x0AaACE4e8AFFC47C6834171c88D342A4aBD8f105","views_contract":"0x0AaACE4e8AFFC47C6834171c88D342A4aBD8f105","token_contract":"0x746C33ccC28b1363c35c09baDAF41b2FFa7E6D56","chain_id":"560048","latest_block":1876871,"chain_id_error":"","block_number_error":""}
{"time":"2026-01-30T12:00:01.531571206Z","level":"INFO","msg":"DIAGNOSTIC: Network contract code","test":"sanity","module":"contract","op":"register_validator","address":"0x0AaACE4e8AFFC47C6834171c88D342A4aBD8f105","code_len":170}
{"time":"2026-01-30T12:00:01.534354043Z","level":"INFO","msg":"DIAGNOSTIC: Views contract code","test":"sanity","module":"contract","op":"register_validator","address":"0x0AaACE4e8AFFC47C6834171c88D342A4aBD8f105","code_len":170}
{"time":"2026-01-30T12:00:01.53807719Z","level":"INFO","msg":"DIAGNOSTIC: Token contract code","test":"sanity","module":"contract","op":"register_validator","address":"0x746C33ccC28b1363c35c09baDAF41b2FFa7E6D56","code_len":7070}
{"time":"2026-01-30T12:00:01.538124755Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator transact opts (pre-send)","test":"sanity","module":"contract","op":"register_validator","auth_nonce":"<nil>","auth_value":"<nil>","auth_gas_price":"<nil>","auth_gas_fee_cap":"<nil>","auth_gas_tip_cap":"<nil>","auth_gas_limit":0,"auth_no_send":false}
{"time":"2026-01-30T12:00:01.540938921Z","level":"INFO","msg":"DIAGNOSTIC: ETH balance","test":"sanity","module":"contract","op":"register_validator","eth_balance_wei":"375397349204375230175"}
{"time":"2026-01-30T12:00:01.545025918Z","level":"INFO","msg":"DIAGNOSTIC: Token balance","test":"sanity","module":"contract","op":"register_validator","token_balance":"88232132530299440000000"}
{"time":"2026-01-30T12:00:01.545070692Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator operator IDs","test":"sanity","module":"contract","op":"register_validator","operator_ids":[304,305,306,307],"operator_ids_len":4,"operator_ids_sorted_asc":true,"operator_ids_unique":true}
{"time":"2026-01-30T12:00:01.550033757Z","level":"INFO","msg":"DIAGNOSTIC: Validator existence (views.getValidator)","test":"sanity","module":"contract","op":"register_validator","validator_exists":false}
{"time":"2026-01-30T12:00:01.553449389Z","level":"ERROR","msg":"DIAGNOSTIC: Call failed","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"views.getBalance","error_type":"*rpc.jsonError","error":"execution reverted"}
{"time":"2026-01-30T12:00:01.553512918Z","level":"ERROR","msg":"DIAGNOSTIC: Revert data decoded","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"views.getBalance","revert_data":"0x12e04c87","revert_reason":"","custom_error":"IncorrectClusterState","custom_error_args":[]}
{"time":"2026-01-30T12:00:01.553558102Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator payload summary","test":"sanity","module":"contract","op":"register_validator","public_key":"0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27","public_key_len_bytes":48,"public_key_keccak256":"0xab8efe86d8039dd91ba1d470441554cb9b9883b2620b7978d61dadd6660032ec","shares_data_len_bytes":1312,"shares_data_keccak256":"0x35c83ac6cc4d6be062d25470ed6376d343f4379e7830c20e4c361a5d9067bb12","amount":"0","amount_is_zero":true,"cluster_validator_count":1,"cluster_network_fee_index":70469584352,"cluster_index":772007289500,"cluster_active":true,"cluster_balance":"8999999865310720000000"}
{"time":"2026-01-30T12:00:01.556875527Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0x5cc0dde14e7256340cc820415a6022a7d1c93a35","allowance":"9999999999999999999999498323694910796200000000"}
{"time":"2026-01-30T12:00:01.556932005Z","level":"INFO","msg":"Threshold value","test":"sanity","module":"contract","threshold":"9000000000000000000000"}
{"time":"2026-01-30T12:00:01.556964012Z","level":"INFO","msg":"Sufficient allowance","test":"sanity","module":"contract","allowance margin":"6561000000000000000000000000000000000000000000000000000000000000000000000000000000000000"}
{"time":"2026-01-30T12:00:01.560729073Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0x5cc0dde14e7256340cc820415a6022a7d1c93a35","allowance":"9999999999999999999999498323694910796200000000"}
{"time":"2026-01-30T12:00:01.560808102Z","level":"INFO","msg":"Allowance","test":"sanity","module":"contract","allowance":"9999999999999999999999498323694910796200000000"}
{"time":"2026-01-30T12:00:01.560905252Z","level":"INFO","msg":"DIAGNOSTIC: registerValidator calldata summary","test":"sanity","module":"contract","op":"register_validator","method_id":"06e8fb9c","calldata_len":1892,"calldata_keccak256":"0xe7fbf809b7bd9b4e9e00034c2b6cf8686c90c432aedb578f6626e6f395d8adbf"}
{"time":"2026-01-30T12:00:01.674706084Z","level":"ERROR","msg":"DIAGNOSTIC: Call failed","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator","error_type":"*rpc.jsonError","error":"execution reverted"}
{"time":"2026-01-30T12:00:01.674818569Z","level":"ERROR","msg":"DIAGNOSTIC: Revert data decoded","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator","revert_data":"0x12e04c87","revert_reason":"","custom_error":"IncorrectClusterState","custom_error_args":[]}
{"time":"2026-01-30T12:00:01.679794088Z","level":"ERROR","msg":"DIAGNOSTIC: Call failed","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator/estimateGas","error_type":"*rpc.jsonError","error":"execution reverted"}
{"time":"2026-01-30T12:00:01.679909487Z","level":"ERROR","msg":"DIAGNOSTIC: Revert data decoded","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator/estimateGas","revert_data":"0x12e04c87","revert_reason":"","custom_error":"IncorrectClusterState","custom_error_args":[]}
{"time":"2026-01-30T12:00:01.684473156Z","level":"ERROR","msg":"DIAGNOSTIC: Call failed","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator/eth_call","error_type":"*rpc.jsonError","error":"execution reverted"}
{"time":"2026-01-30T12:00:01.684565817Z","level":"ERROR","msg":"DIAGNOSTIC: Revert data decoded","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator/eth_call","revert_data":"0x12e04c87","revert_reason":"","custom_error":"IncorrectClusterState","custom_error_args":[]}
{"time":"2026-01-30T12:00:01.684596547Z","level":"ERROR","msg":"❌ FAILURE: Sanity Test Suite Failed","module":"main","error":"execution reverted"}
{"time":"2026-01-30T12:00:01.684615403Z","level":"INFO","msg":"⏳ PROGRESS: Waiting for all tests to finish","module":"main"}
{"time":"2026-01-30T12:00:01.684682416Z","level":"ERROR","msg":"❌ FAILURE: Test Execution Completed with Failures","module":"main","passed_suites":null,"failed_suites":["sanity"],"skipped_suites":["isolated"]}
{"time":"2026-01-30T12:00:01.684729288Z","level":"ERROR","msg":"❌ FAILURE: Suite Failure Details","module":"main","suite":"sanity","error":"execution reverted"}
{"time":"2026-01-30T12:00:01.68475091Z","level":"INFO","msg":"🔑 KEY STEP: Starting resource teardown...","module":"main","usage_id":"96dc396adcfaeb31d5e42d2754ebdf35"}
{"time":"2026-01-30T12:00:01.69722647Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","module":"main","endpoint":"http://aetheria-orchestrator.aetheria.svc:8080"}
{"time":"2026-01-30T12:00:01.697340688Z","level":"INFO","msg":"🧹 Requesting resource teardown","module":"main","usage_id":"96dc396adcfaeb31d5e42d2754ebdf35"}
{"time":"2026-01-30T12:00:01.73743055Z","level":"INFO","msg":"✅ SUCCESS: Completed resource teardown successfully","module":"main","usage_id":"96dc396adcfaeb31d5e42d2754ebdf35"}

2026-01-30T12:00:05Z The test run failed with error

Result - 🔴 Failure

2026-01-30T12:15:09Z Finished cleaning reserved resources

vaclav-ssvlabs · 2026-01-30T12:58:28Z

/test cluster=304,305,306,307

ssv-test-bot · 2026-01-30T12:58:31Z

🔴 Test run for PR #2627

Branch: vaclav-test-aetheria-local-testnet
Commit: e5e11c1

Status

2026-01-30T12:58:31Z Starting test run
2026-01-30T12:58:32Z Test run submitted successfully for commit e5e11c1
2026-01-30T12:58:33Z container image for commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is build and present in container registry
2026-01-30T12:58:35Z using user defined node range cluster=304,305,306,307 on network=hoodi_stage
2026-01-30T12:58:36Z deploying commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 to ssv nodes 304,305,306,307
2026-01-30T12:59:23Z commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is deployed to ssv nodes 304,305,306,307
2026-01-30T12:59:24Z started test executor with tests=(sanity) and network=hoodi_stage

Click to expand executor logs

═══════════════════════════════════════════════════════════════════════════════════════════════════════
                🧪  WELCOME TO Executor - SSV's Distributed Test Automation Execution Layer  🧪            

                                  ___ __  __ ___  ___  _   _  _____  ___   ___ 
                                 | __|\\ \\/ /| __|/ __|| | | ||_   _|/ _ \\ | _ \\
                                 | _|  >  < | _|| (__ | |_| |  | | | (_) ||   /
                                 |___|/_/\\_\\|___|\\___| \\___/   |_|  \\___/ |_|_\\
                                        

═══════════════════════════════════════════════════════════════════════════════════════════════════════
{"time":"2026-01-30T12:59:26.192360752Z","level":"INFO","msg":"🐳 Docker mode: Using environment variables for config generation"}
{"time":"2026-01-30T12:59:26.213318113Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","endpoint":"http://aetheria-orchestrator.aetheria.svc:8080"}
{"time":"2026-01-30T12:59:26.213543095Z","level":"INFO","msg":"📝 Requesting configuration generation from orchestrator API","network":"hoodi_stage","clusters":[304,305,306,307],"tests":"(sanity)"}
{"time":"2026-01-30T12:59:26.308669365Z","level":"INFO","msg":"✅ Configuration generated successfully via API","resource_usage_id":"201e9781498eec74806f3a5d1e2c2e4e"}
{"time":"2026-01-30T12:59:26.309780657Z","level":"INFO","msg":"🚀 EXECUTOR Initialized Successfully"}
{"time":"2026-01-30T12:59:26.309838803Z","level":"INFO","msg":"⚙️ Test configuration loaded","network":"hoodi_stage","nodes":[304,305,306,307],"test_type":{"Sanity":{"ShouldRun":true,"E2M":false,"Threshold":false,"BulkAmount":0},"Bulk":{"ShouldRun":false,"E2M":false,"Threshold":false,"BulkAmount":0},"Isolated":{"Tests":null}}}
{"time":"2026-01-30T12:59:32.12696476Z","level":"INFO","msg":"Generated operator pubkeys file","module":"scanner","path":"/app/shared/data/operators/operator-pubkeys-hoodi_stage.json"}
{"time":"2026-01-30T12:59:32.127329327Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"consensus"}
{"time":"2026-01-30T12:59:32.127413652Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"flow-emits"}
{"time":"2026-01-30T12:59:32.127514609Z","level":"INFO","msg":"Validation client initialized","module":"main","loki_enabled":true,"e2m_enabled":false}
{"time":"2026-01-30T12:59:32.127608841Z","level":"INFO","msg":"🔑 KEY STEP: Starting Isolated Tests Concurrently if exists...","module":"main"}
{"time":"2026-01-30T12:59:32.127752168Z","level":"INFO","msg":"🔑 KEY STEP: Starting Sanity Test Suite","module":"main"}
{"time":"2026-01-30T12:59:32.127929483Z","level":"INFO","msg":"No isolated tests configured, skipping","module":"main"}
{"time":"2026-01-30T12:59:32.174521262Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":304}
{"time":"2026-01-30T12:59:32.195572314Z","level":"INFO","msg":"Available slots","test":"sanity","node":304,"slots":3000}
{"time":"2026-01-30T12:59:32.195669304Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":305}
{"time":"2026-01-30T12:59:32.203574425Z","level":"INFO","msg":"Available slots","test":"sanity","node":305,"slots":3000}
{"time":"2026-01-30T12:59:32.203660859Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":306}
{"time":"2026-01-30T12:59:32.212642407Z","level":"INFO","msg":"Available slots","test":"sanity","node":306,"slots":3000}
{"time":"2026-01-30T12:59:32.21272543Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":307}
{"time":"2026-01-30T12:59:32.222702692Z","level":"INFO","msg":"Available slots","test":"sanity","node":307,"slots":3000}
{"time":"2026-01-30T13:03:35.382235369Z","level":"INFO","msg":"Extracted keyshares path","test":"sanity","module":"keys","path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json"}
{"time":"2026-01-30T13:03:35.382435614Z","level":"INFO","msg":"Key shares file generated successfully","test":"sanity","keyshare_path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json"}
{"time":"2026-01-30T13:03:49.749911987Z","level":"INFO","msg":"DIAGNOSTIC: Cluster snapshot (pre-registration)","test":"sanity","operator_ids":[304,305,306,307],"cluster_validator_count":1,"cluster_network_fee_index":70469584352,"cluster_index":772007289500,"cluster_active":true,"cluster_balance":"8999999865310720000000"}
{"time":"2026-01-30T13:03:49.750078867Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator amount calculation","test":"sanity","amount_per_validator":"2500000000000000000","validator_count":1,"required_amount":"2500000000000000000","current_cluster_balance":"8999999865310720000000","missing_amount":"-8997499865310720000000","missing_amount_rounded_1e8":"-8997499865310700000000","register_amount":"0"}
{"time":"2026-01-30T13:03:49.750240499Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator inputs prepared","test":"sanity","keyshare_path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json","validator_pubkey":"0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27","shares_data_len_chars":2626,"operator_ids":[304,305,306,307],"amount":"0"}
{"time":"2026-01-30T13:03:49.862725882Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator context","test":"sanity","module":"contract","op":"register_validator","from":"0x5cC0DdE14E7256340CC820415a6022a7d1c93A35","network_contract":"0x0AaACE4e8AFFC47C6834171c88D342A4aBD8f105","views_contract":"0x0AaACE4e8AFFC47C6834171c88D342A4aBD8f105","token_contract":"0x746C33ccC28b1363c35c09baDAF41b2FFa7E6D56","chain_id":"560048","latest_block":2135196,"chain_id_error":"","block_number_error":""}
{"time":"2026-01-30T13:03:49.86928049Z","level":"INFO","msg":"DIAGNOSTIC: Network contract code","test":"sanity","module":"contract","op":"register_validator","address":"0x0AaACE4e8AFFC47C6834171c88D342A4aBD8f105","code_len":170}
{"time":"2026-01-30T13:03:49.878733264Z","level":"INFO","msg":"DIAGNOSTIC: Views contract code","test":"sanity","module":"contract","op":"register_validator","address":"0x0AaACE4e8AFFC47C6834171c88D342A4aBD8f105","code_len":170}
{"time":"2026-01-30T13:03:49.883618979Z","level":"INFO","msg":"DIAGNOSTIC: Token contract code","test":"sanity","module":"contract","op":"register_validator","address":"0x746C33ccC28b1363c35c09baDAF41b2FFa7E6D56","code_len":7070}
{"time":"2026-01-30T13:03:49.88371573Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator transact opts (pre-send)","test":"sanity","module":"contract","op":"register_validator","auth_nonce":"<nil>","auth_value":"<nil>","auth_gas_price":"<nil>","auth_gas_fee_cap":"<nil>","auth_gas_tip_cap":"<nil>","auth_gas_limit":0,"auth_no_send":false}
{"time":"2026-01-30T13:03:49.886915474Z","level":"INFO","msg":"DIAGNOSTIC: ETH balance","test":"sanity","module":"contract","op":"register_validator","eth_balance_wei":"340343095415624726626"}
{"time":"2026-01-30T13:03:49.894143119Z","level":"INFO","msg":"DIAGNOSTIC: Token balance","test":"sanity","module":"contract","op":"register_validator","token_balance":"75686386931269760000000"}
{"time":"2026-01-30T13:03:49.894488792Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator operator IDs","test":"sanity","module":"contract","op":"register_validator","operator_ids":[304,305,306,307],"operator_ids_len":4,"operator_ids_sorted_asc":true,"operator_ids_unique":true}
{"time":"2026-01-30T13:03:49.899554218Z","level":"INFO","msg":"DIAGNOSTIC: Validator existence (views.getValidator)","test":"sanity","module":"contract","op":"register_validator","validator_exists":false}
{"time":"2026-01-30T13:03:49.90652568Z","level":"INFO","msg":"DIAGNOSTIC: Cluster balance (views.getBalance)","test":"sanity","module":"contract","op":"register_validator","on_chain_balance":"0","scanner_balance":"8999999865310720000000","balance_match":false}
{"time":"2026-01-30T13:03:49.906629359Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator payload summary","test":"sanity","module":"contract","op":"register_validator","public_key":"0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27","public_key_len_bytes":48,"public_key_keccak256":"0xab8efe86d8039dd91ba1d470441554cb9b9883b2620b7978d61dadd6660032ec","shares_data_len_bytes":1312,"shares_data_keccak256":"0x89614a357e8f5e312e94dc310fd97687006c4b5fb7447ce91827ddc71ad2caaa","amount":"0","amount_is_zero":true,"cluster_validator_count":1,"cluster_network_fee_index":70469584352,"cluster_index":772007289500,"cluster_active":true,"cluster_balance":"8999999865310720000000"}
{"time":"2026-01-30T13:03:49.91068697Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0x5cc0dde14e7256340cc820415a6022a7d1c93a35","allowance":"9999999999999999999999473850789666515400000000"}
{"time":"2026-01-30T13:03:49.910798966Z","level":"INFO","msg":"Threshold value","test":"sanity","module":"contract","threshold":"9000000000000000000000"}
{"time":"2026-01-30T13:03:49.910849159Z","level":"INFO","msg":"Sufficient allowance","test":"sanity","module":"contract","allowance margin":"6561000000000000000000000000000000000000000000000000000000000000000000000000000000000000"}
{"time":"2026-01-30T13:03:49.916396376Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0x5cc0dde14e7256340cc820415a6022a7d1c93a35","allowance":"9999999999999999999999473850789666515400000000"}
{"time":"2026-01-30T13:03:49.916519693Z","level":"INFO","msg":"Allowance","test":"sanity","module":"contract","allowance":"9999999999999999999999473850789666515400000000"}
{"time":"2026-01-30T13:03:49.916660227Z","level":"INFO","msg":"DIAGNOSTIC: registerValidator calldata summary","test":"sanity","module":"contract","op":"register_validator","method_id":"06e8fb9c","calldata_len":1892,"calldata_keccak256":"0xd3301028cde8267d87bef79599358f40596d0742678d4ece595c2dd3bdaf418f"}
{"time":"2026-01-30T13:03:49.956578216Z","level":"ERROR","msg":"DIAGNOSTIC: Call failed","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator","error_type":"*rpc.jsonError","error":"execution reverted"}
{"time":"2026-01-30T13:03:49.961555897Z","level":"ERROR","msg":"DIAGNOSTIC: Call failed","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator/estimateGas","error_type":"*rpc.jsonError","error":"execution reverted"}
{"time":"2026-01-30T13:03:49.965436557Z","level":"ERROR","msg":"DIAGNOSTIC: Call failed","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator/eth_call","error_type":"*rpc.jsonError","error":"execution reverted"}
{"time":"2026-01-30T13:03:49.965549034Z","level":"ERROR","msg":"❌ FAILURE: Sanity Test Suite Failed","module":"main","error":"execution reverted"}
{"time":"2026-01-30T13:03:49.96557135Z","level":"INFO","msg":"⏳ PROGRESS: Waiting for all tests to finish","module":"main"}
{"time":"2026-01-30T13:03:49.965633713Z","level":"ERROR","msg":"❌ FAILURE: Test Execution Completed with Failures","module":"main","passed_suites":null,"failed_suites":["sanity"],"skipped_suites":["isolated"]}
{"time":"2026-01-30T13:03:49.96568507Z","level":"ERROR","msg":"❌ FAILURE: Suite Failure Details","module":"main","suite":"sanity","error":"execution reverted"}
{"time":"2026-01-30T13:03:49.965706422Z","level":"INFO","msg":"🔑 KEY STEP: Starting resource teardown...","module":"main","usage_id":"201e9781498eec74806f3a5d1e2c2e4e"}
{"time":"2026-01-30T13:03:49.975141347Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","module":"main","endpoint":"http://aetheria-orchestrator.aetheria.svc:8080"}
{"time":"2026-01-30T13:03:49.975247596Z","level":"INFO","msg":"🧹 Requesting resource teardown","module":"main","usage_id":"201e9781498eec74806f3a5d1e2c2e4e"}
{"time":"2026-01-30T13:03:50.0271041Z","level":"INFO","msg":"✅ SUCCESS: Completed resource teardown successfully","module":"main","usage_id":"201e9781498eec74806f3a5d1e2c2e4e"}

2026-01-30T13:03:53Z The test run failed with error

Result - 🔴 Failure

2026-01-30T13:18:57Z Finished cleaning reserved resources

vaclav-ssvlabs · 2026-01-31T06:00:09Z

/test cluster=304,305,306,307

ssv-test-bot · 2026-01-31T06:00:13Z

🔴 Test run for PR #2627

Branch: vaclav-test-aetheria-local-testnet
Commit: e5e11c1

Status

2026-01-31T06:00:12Z Starting test run
2026-01-31T06:00:14Z Test run submitted successfully for commit e5e11c1
2026-01-31T06:00:16Z container image for commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is build and present in container registry
2026-01-31T06:00:17Z using user defined node range cluster=304,305,306,307 on network=hoodi_stage
2026-01-31T06:00:19Z deploying commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 to ssv nodes 304,305,306,307
2026-01-31T06:01:33Z commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is deployed to ssv nodes 304,305,306,307
2026-01-31T06:01:35Z started test executor with tests=(sanity) and network=hoodi_stage

Click to expand executor logs

═══════════════════════════════════════════════════════════════════════════════════════════════════════
                🧪  WELCOME TO Executor - SSV's Distributed Test Automation Execution Layer  🧪            

                                  ___ __  __ ___  ___  _   _  _____  ___   ___ 
                                 | __|\\ \\/ /| __|/ __|| | | ||_   _|/ _ \\ | _ \\
                                 | _|  >  < | _|| (__ | |_| |  | | | (_) ||   /
                                 |___|/_/\\_\\|___|\\___| \\___/   |_|  \\___/ |_|_\\
                                        

═══════════════════════════════════════════════════════════════════════════════════════════════════════
{"time":"2026-01-31T06:02:02.624381877Z","level":"INFO","msg":"🐳 Docker mode: Using environment variables for config generation"}
{"time":"2026-01-31T06:02:02.633161433Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","endpoint":"http://aetheria-orchestrator.aetheria.svc:8080"}
{"time":"2026-01-31T06:02:02.633260415Z","level":"INFO","msg":"📝 Requesting configuration generation from orchestrator API","network":"hoodi_stage","clusters":[304,305,306,307],"tests":"(sanity)"}
{"time":"2026-01-31T06:02:02.817975466Z","level":"INFO","msg":"✅ Configuration generated successfully via API","resource_usage_id":"3ba3a8d532f0825a4523643ddaee30f6"}
{"time":"2026-01-31T06:02:02.818782336Z","level":"INFO","msg":"🚀 EXECUTOR Initialized Successfully"}
{"time":"2026-01-31T06:02:02.818807814Z","level":"INFO","msg":"⚙️ Test configuration loaded","network":"hoodi_stage","nodes":[304,305,306,307],"test_type":{"Sanity":{"ShouldRun":true,"E2M":false,"Threshold":false,"BulkAmount":0},"Bulk":{"ShouldRun":false,"E2M":false,"Threshold":false,"BulkAmount":0},"Isolated":{"Tests":null}}}
{"time":"2026-01-31T06:02:07.791336418Z","level":"INFO","msg":"Generated operator pubkeys file","module":"scanner","path":"/app/shared/data/operators/operator-pubkeys-hoodi_stage.json"}
{"time":"2026-01-31T06:02:07.791551431Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"consensus"}
{"time":"2026-01-31T06:02:07.7915774Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"flow-emits"}
{"time":"2026-01-31T06:02:07.791614478Z","level":"INFO","msg":"Validation client initialized","module":"main","loki_enabled":true,"e2m_enabled":false}
{"time":"2026-01-31T06:02:07.79164122Z","level":"INFO","msg":"🔑 KEY STEP: Starting Isolated Tests Concurrently if exists...","module":"main"}
{"time":"2026-01-31T06:02:07.791685463Z","level":"INFO","msg":"🔑 KEY STEP: Starting Sanity Test Suite","module":"main"}
{"time":"2026-01-31T06:02:07.79199805Z","level":"INFO","msg":"No isolated tests configured, skipping","module":"main"}
{"time":"2026-01-31T06:02:07.843056499Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":304}
{"time":"2026-01-31T06:02:07.85946127Z","level":"INFO","msg":"Available slots","test":"sanity","node":304,"slots":3000}
{"time":"2026-01-31T06:02:07.859591946Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":305}
{"time":"2026-01-31T06:02:07.870091468Z","level":"INFO","msg":"Available slots","test":"sanity","node":305,"slots":3000}
{"time":"2026-01-31T06:02:07.870155929Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":306}
{"time":"2026-01-31T06:02:07.881547014Z","level":"INFO","msg":"Available slots","test":"sanity","node":306,"slots":3000}
{"time":"2026-01-31T06:02:07.881667661Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":307}
{"time":"2026-01-31T06:02:07.890365219Z","level":"INFO","msg":"Available slots","test":"sanity","node":307,"slots":3000}
{"time":"2026-01-31T06:06:05.006932334Z","level":"INFO","msg":"Extracted keyshares path","test":"sanity","module":"keys","path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json"}
{"time":"2026-01-31T06:06:05.007032129Z","level":"INFO","msg":"Key shares file generated successfully","test":"sanity","keyshare_path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json"}
{"time":"2026-01-31T06:06:11.692329796Z","level":"INFO","msg":"DIAGNOSTIC: Cluster snapshot (pre-registration)","test":"sanity","operator_ids":[304,305,306,307],"cluster_validator_count":1,"cluster_network_fee_index":70469584352,"cluster_index":772007289500,"cluster_active":true,"cluster_balance":"8999999865310720000000"}
{"time":"2026-01-31T06:06:11.692569396Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator amount calculation","test":"sanity","amount_per_validator":"2500000000000000000","validator_count":1,"required_amount":"2500000000000000000","current_cluster_balance":"8999999865310720000000","missing_amount":"-8997499865310720000000","missing_amount_rounded_1e8":"-8997499865310700000000","register_amount":"0"}
{"time":"2026-01-31T06:06:11.692676472Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator inputs prepared","test":"sanity","keyshare_path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json","validator_pubkey":"0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27","shares_data_len_chars":2626,"operator_ids":[304,305,306,307],"amount":"0"}
{"time":"2026-01-31T06:06:11.728121928Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator context","test":"sanity","module":"contract","op":"register_validator","from":"0x5cC0DdE14E7256340CC820415a6022a7d1c93A35","network_contract":"0x0AaACE4e8AFFC47C6834171c88D342A4aBD8f105","views_contract":"0x9143b8C25EfA53F28de4cBeFd0b6dFd66D43FeA6","token_contract":"0x746C33ccC28b1363c35c09baDAF41b2FFa7E6D56","chain_id":"560048","latest_block":2139917,"views_latest_block":2139917,"token_latest_block":2139917,"chain_id_error":"","block_number_error":"","views_block_number_error":"","token_block_number_error":""}
{"time":"2026-01-31T06:06:11.74054615Z","level":"INFO","msg":"DIAGNOSTIC: Network contract code","test":"sanity","module":"contract","op":"register_validator","address":"0x0AaACE4e8AFFC47C6834171c88D342A4aBD8f105","code_len":170}
{"time":"2026-01-31T06:06:11.744160733Z","level":"INFO","msg":"DIAGNOSTIC: Views contract code","test":"sanity","module":"contract","op":"register_validator","address":"0x9143b8C25EfA53F28de4cBeFd0b6dFd66D43FeA6","code_len":170}
{"time":"2026-01-31T06:06:11.749097797Z","level":"INFO","msg":"DIAGNOSTIC: Token contract code","test":"sanity","module":"contract","op":"register_validator","address":"0x746C33ccC28b1363c35c09baDAF41b2FFa7E6D56","code_len":7070}
{"time":"2026-01-31T06:06:11.749196284Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator transact opts (pre-send)","test":"sanity","module":"contract","op":"register_validator","auth_nonce":"<nil>","auth_value":"<nil>","auth_gas_price":"<nil>","auth_gas_fee_cap":"<nil>","auth_gas_tip_cap":"<nil>","auth_gas_limit":0,"auth_no_send":false}
{"time":"2026-01-31T06:06:11.752120531Z","level":"INFO","msg":"DIAGNOSTIC: ETH balance","test":"sanity","module":"contract","op":"register_validator","eth_balance_wei":"340412645810925518573"}
{"time":"2026-01-31T06:06:11.757093055Z","level":"INFO","msg":"DIAGNOSTIC: Token balance","test":"sanity","module":"contract","op":"register_validator","token_balance":"75686386931269760000000"}
{"time":"2026-01-31T06:06:11.757221317Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator operator IDs","test":"sanity","module":"contract","op":"register_validator","operator_ids":[304,305,306,307],"operator_ids_len":4,"operator_ids_sorted_asc":true,"operator_ids_unique":true}
{"time":"2026-01-31T06:06:11.76151615Z","level":"INFO","msg":"DIAGNOSTIC: Validator existence (views.getValidator)","test":"sanity","module":"contract","op":"register_validator","validator_exists":false}
{"time":"2026-01-31T06:06:11.765987469Z","level":"INFO","msg":"DIAGNOSTIC: Cluster balance (views.getBalance)","test":"sanity","module":"contract","op":"register_validator","on_chain_balance":"0","scanner_balance":"8999999865310720000000","balance_match":false}
{"time":"2026-01-31T06:06:11.76627Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator payload summary","test":"sanity","module":"contract","op":"register_validator","public_key":"0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27","public_key_len_bytes":48,"public_key_keccak256":"0xab8efe86d8039dd91ba1d470441554cb9b9883b2620b7978d61dadd6660032ec","shares_data_len_bytes":1312,"shares_data_keccak256":"0xf611e33fd7aff10fb461f824fb0b1cf2baf19267b2f220923ab0bfc0f90ef102","amount":"0","amount_is_zero":true,"cluster_validator_count":1,"cluster_network_fee_index":70469584352,"cluster_index":772007289500,"cluster_active":true,"cluster_balance":"8999999865310720000000"}
{"time":"2026-01-31T06:06:11.771327605Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0x5cc0dde14e7256340cc820415a6022a7d1c93a35","allowance":"9999999999999999999999473850789666515400000000"}
{"time":"2026-01-31T06:06:11.771433572Z","level":"INFO","msg":"Threshold value","test":"sanity","module":"contract","threshold":"9000000000000000000000"}
{"time":"2026-01-31T06:06:11.771478609Z","level":"INFO","msg":"Sufficient allowance","test":"sanity","module":"contract","allowance margin":"6561000000000000000000000000000000000000000000000000000000000000000000000000000000000000"}
{"time":"2026-01-31T06:06:11.778686103Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0x5cc0dde14e7256340cc820415a6022a7d1c93a35","allowance":"9999999999999999999999473850789666515400000000"}
{"time":"2026-01-31T06:06:11.778857768Z","level":"INFO","msg":"Allowance","test":"sanity","module":"contract","allowance":"9999999999999999999999473850789666515400000000"}
{"time":"2026-01-31T06:06:11.779134091Z","level":"INFO","msg":"DIAGNOSTIC: registerValidator calldata summary","test":"sanity","module":"contract","op":"register_validator","method_id":"06e8fb9c","calldata_len":1892,"calldata_keccak256":"0x6d0d662f6fccfe9524dab70bb5db706eed5ed29f9b09a40fcff9f21646bd12da"}
{"time":"2026-01-31T06:06:11.802123603Z","level":"ERROR","msg":"DIAGNOSTIC: Call failed","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator","error_type":"*rpc.jsonError","error":"execution reverted"}
{"time":"2026-01-31T06:06:11.806888399Z","level":"ERROR","msg":"DIAGNOSTIC: Call failed","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator/estimateGas","error_type":"*rpc.jsonError","error":"execution reverted"}
{"time":"2026-01-31T06:06:11.810621352Z","level":"ERROR","msg":"DIAGNOSTIC: Call failed","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator/eth_call","error_type":"*rpc.jsonError","error":"execution reverted"}
{"time":"2026-01-31T06:06:11.810768321Z","level":"ERROR","msg":"❌ FAILURE: Sanity Test Suite Failed","module":"main","error":"execution reverted"}
{"time":"2026-01-31T06:06:11.810827219Z","level":"INFO","msg":"⏳ PROGRESS: Waiting for all tests to finish","module":"main"}
{"time":"2026-01-31T06:06:11.810916419Z","level":"ERROR","msg":"❌ FAILURE: Test Execution Completed with Failures","module":"main","passed_suites":null,"failed_suites":["sanity"],"skipped_suites":["isolated"]}
{"time":"2026-01-31T06:06:11.810966675Z","level":"ERROR","msg":"❌ FAILURE: Suite Failure Details","module":"main","suite":"sanity","error":"execution reverted"}
{"time":"2026-01-31T06:06:11.810988732Z","level":"INFO","msg":"🔑 KEY STEP: Starting resource teardown...","module":"main","usage_id":"3ba3a8d532f0825a4523643ddaee30f6"}
{"time":"2026-01-31T06:06:11.818375468Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","module":"main","endpoint":"http://aetheria-orchestrator.aetheria.svc:8080"}
{"time":"2026-01-31T06:06:11.818494758Z","level":"INFO","msg":"🧹 Requesting resource teardown","module":"main","usage_id":"3ba3a8d532f0825a4523643ddaee30f6"}
{"time":"2026-01-31T06:06:11.851710292Z","level":"INFO","msg":"✅ SUCCESS: Completed resource teardown successfully","module":"main","usage_id":"3ba3a8d532f0825a4523643ddaee30f6"}

2026-01-31T06:06:15Z The test run failed with error

Result - 🔴 Failure

2026-01-31T06:21:18Z Finished cleaning reserved resources

vaclav-ssvlabs · 2026-01-31T07:54:17Z

/test cluster=304,305,306,307

ssv-test-bot · 2026-01-31T07:54:20Z

🔴 Test run for PR #2627

Branch: vaclav-test-aetheria-local-testnet
Commit: e5e11c1

Status

2026-01-31T07:54:20Z Starting test run
2026-01-31T07:54:21Z Test run submitted successfully for commit e5e11c1
2026-01-31T07:54:23Z container image for commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is build and present in container registry
2026-01-31T07:54:24Z using user defined node range cluster=304,305,306,307 on network=hoodi_stage
2026-01-31T07:54:25Z deploying commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 to ssv nodes 304,305,306,307
2026-01-31T07:55:42Z commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is deployed to ssv nodes 304,305,306,307
2026-01-31T07:55:43Z started test executor with tests=(sanity) and network=hoodi_stage

Click to expand executor logs

═══════════════════════════════════════════════════════════════════════════════════════════════════════
                🧪  WELCOME TO Executor - SSV's Distributed Test Automation Execution Layer  🧪            

                                  ___ __  __ ___  ___  _   _  _____  ___   ___ 
                                 | __|\\ \\/ /| __|/ __|| | | ||_   _|/ _ \\ | _ \\
                                 | _|  >  < | _|| (__ | |_| |  | | | (_) ||   /
                                 |___|/_/\\_\\|___|\\___| \\___/   |_|  \\___/ |_|_\\
                                        

═══════════════════════════════════════════════════════════════════════════════════════════════════════
{"time":"2026-01-31T07:56:29.357392907Z","level":"INFO","msg":"🐳 Docker mode: Using environment variables for config generation"}
{"time":"2026-01-31T07:56:29.398220407Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","endpoint":"http://aetheria-orchestrator.aetheria.svc:8080"}
{"time":"2026-01-31T07:56:29.398376977Z","level":"INFO","msg":"📝 Requesting configuration generation from orchestrator API","network":"hoodi_stage","clusters":[304,305,306,307],"tests":"(sanity)"}
{"time":"2026-01-31T07:56:29.576073931Z","level":"INFO","msg":"✅ Configuration generated successfully via API","resource_usage_id":"8843acaaf72bb7f5fa62af84ef4da6a3"}
{"time":"2026-01-31T07:56:29.57791797Z","level":"INFO","msg":"🚀 EXECUTOR Initialized Successfully"}
{"time":"2026-01-31T07:56:29.577994424Z","level":"INFO","msg":"⚙️ Test configuration loaded","network":"hoodi_stage","nodes":[304,305,306,307],"test_type":{"Sanity":{"ShouldRun":true,"E2M":false,"Threshold":false,"BulkAmount":0},"Bulk":{"ShouldRun":false,"E2M":false,"Threshold":false,"BulkAmount":0},"Isolated":{"Tests":null}}}
{"time":"2026-01-31T07:56:34.657346574Z","level":"INFO","msg":"Generated operator pubkeys file","module":"scanner","path":"/app/shared/data/operators/operator-pubkeys-hoodi_stage.json"}
{"time":"2026-01-31T07:56:34.657557261Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"consensus"}
{"time":"2026-01-31T07:56:34.657590126Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"flow-emits"}
{"time":"2026-01-31T07:56:34.657658258Z","level":"INFO","msg":"Validation client initialized","module":"main","loki_enabled":true,"e2m_enabled":false}
{"time":"2026-01-31T07:56:34.657728431Z","level":"INFO","msg":"🔑 KEY STEP: Starting Isolated Tests Concurrently if exists...","module":"main"}
{"time":"2026-01-31T07:56:34.657794503Z","level":"INFO","msg":"🔑 KEY STEP: Starting Sanity Test Suite","module":"main"}
{"time":"2026-01-31T07:56:34.658062491Z","level":"INFO","msg":"No isolated tests configured, skipping","module":"main"}
{"time":"2026-01-31T07:56:34.711515361Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":304}
{"time":"2026-01-31T07:56:34.726963134Z","level":"INFO","msg":"Available slots","test":"sanity","node":304,"slots":3000}
{"time":"2026-01-31T07:56:34.727060671Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":305}
{"time":"2026-01-31T07:56:34.736166981Z","level":"INFO","msg":"Available slots","test":"sanity","node":305,"slots":3000}
{"time":"2026-01-31T07:56:34.736352814Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":306}
{"time":"2026-01-31T07:56:34.744684039Z","level":"INFO","msg":"Available slots","test":"sanity","node":306,"slots":3000}
{"time":"2026-01-31T07:56:34.744769602Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":307}
{"time":"2026-01-31T07:56:34.752863749Z","level":"INFO","msg":"Available slots","test":"sanity","node":307,"slots":3000}
{"time":"2026-01-31T07:58:33.822116522Z","level":"INFO","msg":"Extracted keyshares path","test":"sanity","module":"keys","path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json"}
{"time":"2026-01-31T07:58:33.823297721Z","level":"INFO","msg":"Key shares file generated successfully","test":"sanity","keyshare_path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json"}
{"time":"2026-01-31T07:58:48.017942113Z","level":"INFO","msg":"DIAGNOSTIC: Cluster snapshot (pre-registration)","test":"sanity","operator_ids":[304,305,306,307],"cluster_validator_count":1,"cluster_network_fee_index":70469584352,"cluster_index":772007289500,"cluster_active":true,"cluster_balance":"8999999865310720000000"}
{"time":"2026-01-31T07:58:48.018036279Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator amount calculation","test":"sanity","amount_per_validator":"2500000000000000000","validator_count":1,"required_amount":"2500000000000000000","current_cluster_balance":"8999999865310720000000","missing_amount":"-8997499865310720000000","missing_amount_rounded_1e8":"-8997499865310700000000","register_amount":"0"}
{"time":"2026-01-31T07:58:48.018070245Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator inputs prepared","test":"sanity","keyshare_path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json","validator_pubkey":"0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27","shares_data_len_chars":2626,"operator_ids":[304,305,306,307],"amount":"0"}
{"time":"2026-01-31T07:58:48.129858331Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator context","test":"sanity","module":"contract","op":"register_validator","from":"0x5cC0DdE14E7256340CC820415a6022a7d1c93A35","network_contract":"0x0AaACE4e8AFFC47C6834171c88D342A4aBD8f105","views_contract":"0x9143b8C25EfA53F28de4cBeFd0b6dFd66D43FeA6","token_contract":"0x746C33ccC28b1363c35c09baDAF41b2FFa7E6D56","chain_id":"560048","latest_block":2140413,"views_latest_block":2140413,"token_latest_block":2140413,"chain_id_error":"","block_number_error":"","views_block_number_error":"","token_block_number_error":""}
{"time":"2026-01-31T07:58:48.132856083Z","level":"INFO","msg":"DIAGNOSTIC: Network contract code","test":"sanity","module":"contract","op":"register_validator","address":"0x0AaACE4e8AFFC47C6834171c88D342A4aBD8f105","code_len":170}
{"time":"2026-01-31T07:58:48.135963116Z","level":"INFO","msg":"DIAGNOSTIC: Views contract code","test":"sanity","module":"contract","op":"register_validator","address":"0x9143b8C25EfA53F28de4cBeFd0b6dFd66D43FeA6","code_len":170}
{"time":"2026-01-31T07:58:48.14058793Z","level":"INFO","msg":"DIAGNOSTIC: Token contract code","test":"sanity","module":"contract","op":"register_validator","address":"0x746C33ccC28b1363c35c09baDAF41b2FFa7E6D56","code_len":7070}
{"time":"2026-01-31T07:58:48.140656217Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator transact opts (pre-send)","test":"sanity","module":"contract","op":"register_validator","auth_nonce":"<nil>","auth_value":"<nil>","auth_gas_price":"<nil>","auth_gas_fee_cap":"<nil>","auth_gas_tip_cap":"<nil>","auth_gas_limit":0,"auth_no_send":false}
{"time":"2026-01-31T07:58:48.144132881Z","level":"INFO","msg":"DIAGNOSTIC: ETH balance","test":"sanity","module":"contract","op":"register_validator","eth_balance_wei":"340438234370176109154"}
{"time":"2026-01-31T07:58:48.148696784Z","level":"INFO","msg":"DIAGNOSTIC: Token balance","test":"sanity","module":"contract","op":"register_validator","token_balance":"75686386931269760000000"}
{"time":"2026-01-31T07:58:48.1487511Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator operator IDs","test":"sanity","module":"contract","op":"register_validator","operator_ids":[304,305,306,307],"operator_ids_len":4,"operator_ids_sorted_asc":true,"operator_ids_unique":true}
{"time":"2026-01-31T07:58:48.156827751Z","level":"INFO","msg":"DIAGNOSTIC: Validator existence (views.getValidator)","test":"sanity","module":"contract","op":"register_validator","validator_exists":false}
{"time":"2026-01-31T07:58:48.163860066Z","level":"INFO","msg":"DIAGNOSTIC: Cluster balance (views.getBalance)","test":"sanity","module":"contract","op":"register_validator","on_chain_balance":"0","scanner_balance":"8999999865310720000000","balance_match":false}
{"time":"2026-01-31T07:58:48.164073834Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator payload summary","test":"sanity","module":"contract","op":"register_validator","public_key":"0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27","public_key_len_bytes":48,"public_key_keccak256":"0xab8efe86d8039dd91ba1d470441554cb9b9883b2620b7978d61dadd6660032ec","shares_data_len_bytes":1312,"shares_data_keccak256":"0xa739f40637c0f333ae5e2d08c86ac26e60dbc902b4b126e8f98d31f2b4d3e68f","amount":"0","amount_is_zero":true,"cluster_validator_count":1,"cluster_network_fee_index":70469584352,"cluster_index":772007289500,"cluster_active":true,"cluster_balance":"8999999865310720000000"}
{"time":"2026-01-31T07:58:48.171862784Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0x5cc0dde14e7256340cc820415a6022a7d1c93a35","allowance":"9999999999999999999999473850789666515400000000"}
{"time":"2026-01-31T07:58:48.171976515Z","level":"INFO","msg":"Threshold value","test":"sanity","module":"contract","threshold":"9000000000000000000000"}
{"time":"2026-01-31T07:58:48.172031168Z","level":"INFO","msg":"Sufficient allowance","test":"sanity","module":"contract","allowance margin":"6561000000000000000000000000000000000000000000000000000000000000000000000000000000000000"}
{"time":"2026-01-31T07:58:48.176537457Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0x5cc0dde14e7256340cc820415a6022a7d1c93a35","allowance":"9999999999999999999999473850789666515400000000"}
{"time":"2026-01-31T07:58:48.176663551Z","level":"INFO","msg":"Allowance","test":"sanity","module":"contract","allowance":"9999999999999999999999473850789666515400000000"}
{"time":"2026-01-31T07:58:48.176757798Z","level":"INFO","msg":"DIAGNOSTIC: registerValidator calldata summary","test":"sanity","module":"contract","op":"register_validator","method_id":"06e8fb9c","calldata_len":1892,"calldata_keccak256":"0x1253ba5847cbc17ce75bfdef6882a58ca97303c35ca60d2cdc5f156211021de3"}
{"time":"2026-01-31T07:58:48.257093661Z","level":"ERROR","msg":"DIAGNOSTIC: Call failed","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator","error_type":"*rpc.jsonError","error":"execution reverted"}
{"time":"2026-01-31T07:58:48.257403615Z","level":"INFO","msg":"DIAGNOSTIC: Raw RPC error data","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator","error_data_type":"string","error_data":"\\"0x\\""}
{"time":"2026-01-31T07:58:48.261208231Z","level":"ERROR","msg":"DIAGNOSTIC: Call failed","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator/estimateGas","error_type":"*rpc.jsonError","error":"execution reverted"}
{"time":"2026-01-31T07:58:48.261248994Z","level":"INFO","msg":"DIAGNOSTIC: Raw RPC error data","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator/estimateGas","error_data_type":"string","error_data":"\\"0x\\""}
{"time":"2026-01-31T07:58:48.264664115Z","level":"ERROR","msg":"DIAGNOSTIC: Call failed","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator/eth_call","error_type":"*rpc.jsonError","error":"execution reverted"}
{"time":"2026-01-31T07:58:48.264778468Z","level":"INFO","msg":"DIAGNOSTIC: Raw RPC error data","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator/eth_call","error_data_type":"string","error_data":"\\"0x\\""}
{"time":"2026-01-31T07:58:48.264831148Z","level":"ERROR","msg":"❌ FAILURE: Sanity Test Suite Failed","module":"main","error":"execution reverted"}
{"time":"2026-01-31T07:58:48.264841671Z","level":"INFO","msg":"⏳ PROGRESS: Waiting for all tests to finish","module":"main"}
{"time":"2026-01-31T07:58:48.264916988Z","level":"ERROR","msg":"❌ FAILURE: Test Execution Completed with Failures","module":"main","passed_suites":null,"failed_suites":["sanity"],"skipped_suites":["isolated"]}
{"time":"2026-01-31T07:58:48.264941547Z","level":"ERROR","msg":"❌ FAILURE: Suite Failure Details","module":"main","suite":"sanity","error":"execution reverted"}
{"time":"2026-01-31T07:58:48.26497317Z","level":"INFO","msg":"🔑 KEY STEP: Starting resource teardown...","module":"main","usage_id":"8843acaaf72bb7f5fa62af84ef4da6a3"}
{"time":"2026-01-31T07:58:48.278974203Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","module":"main","endpoint":"http://aetheria-orchestrator.aetheria.svc:8080"}
{"time":"2026-01-31T07:58:48.279067375Z","level":"INFO","msg":"🧹 Requesting resource teardown","module":"main","usage_id":"8843acaaf72bb7f5fa62af84ef4da6a3"}
{"time":"2026-01-31T07:58:48.335495765Z","level":"INFO","msg":"✅ SUCCESS: Completed resource teardown successfully","module":"main","usage_id":"8843acaaf72bb7f5fa62af84ef4da6a3"}

2026-01-31T07:58:52Z The test run failed with error

Result - 🔴 Failure

2026-01-31T08:13:55Z Finished cleaning reserved resources

vaclav-ssvlabs · 2026-02-02T13:56:07Z

/test cluster=304,305,306,307

ssv-test-bot · 2026-02-02T13:56:12Z

🔴 Test run for PR #2627

Branch: vaclav-test-aetheria-local-testnet
Commit: e5e11c1

Status

2026-02-02T13:56:11Z Starting test run
2026-02-02T13:56:13Z Test run submitted successfully for commit e5e11c1
2026-02-02T13:56:14Z container image for commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is build and present in container registry
2026-02-02T13:56:16Z using user defined node range cluster=304,305,306,307 on network=hoodi_stage
2026-02-02T13:56:17Z deploying commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 to ssv nodes 304,305,306,307
2026-02-02T13:57:04Z commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is deployed to ssv nodes 304,305,306,307
2026-02-02T13:57:05Z started test executor with tests=(sanity) and network=hoodi_stage

Click to expand executor logs

═══════════════════════════════════════════════════════════════════════════════════════════════════════
                🧪  WELCOME TO Executor - SSV's Distributed Test Automation Execution Layer  🧪            

                                  ___ __  __ ___  ___  _   _  _____  ___   ___ 
                                 | __|\\ \\/ /| __|/ __|| | | ||_   _|/ _ \\ | _ \\
                                 | _|  >  < | _|| (__ | |_| |  | | | (_) ||   /
                                 |___|/_/\\_\\|___|\\___| \\___/   |_|  \\___/ |_|_\\
                                        

═══════════════════════════════════════════════════════════════════════════════════════════════════════
{"time":"2026-02-02T13:57:08.615620995Z","level":"INFO","msg":"🐳 Docker mode: Using environment variables for config generation"}
{"time":"2026-02-02T13:57:08.627451248Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","endpoint":"http://aetheria-orchestrator.aetheria.svc:8080"}
{"time":"2026-02-02T13:57:08.62758542Z","level":"INFO","msg":"📝 Requesting configuration generation from orchestrator API","network":"hoodi_stage","clusters":[304,305,306,307],"tests":"(sanity)"}
{"time":"2026-02-02T13:57:08.719525611Z","level":"INFO","msg":"✅ Configuration generated successfully via API","resource_usage_id":"cb99125ffdcc38ffcdd5c78ca998dabb"}
{"time":"2026-02-02T13:57:08.720886266Z","level":"INFO","msg":"🚀 EXECUTOR Initialized Successfully"}
{"time":"2026-02-02T13:57:08.720936163Z","level":"INFO","msg":"⚙️ Test configuration loaded","network":"hoodi_stage","nodes":[304,305,306,307],"test_type":{"Sanity":{"ShouldRun":true,"E2M":false,"Threshold":false,"BulkAmount":0},"Bulk":{"ShouldRun":false,"E2M":false,"Threshold":false,"BulkAmount":0},"Isolated":{"Tests":null}}}
{"time":"2026-02-02T13:57:13.664821541Z","level":"INFO","msg":"Generated operator pubkeys file","module":"scanner","path":"/app/shared/data/operators/operator-pubkeys-hoodi_stage.json"}
{"time":"2026-02-02T13:57:13.665103264Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"consensus"}
{"time":"2026-02-02T13:57:13.665252188Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"flow-emits"}
{"time":"2026-02-02T13:57:13.665366778Z","level":"INFO","msg":"Validation client initialized","module":"main","loki_enabled":true,"e2m_enabled":false}
{"time":"2026-02-02T13:57:13.665417229Z","level":"INFO","msg":"🔑 KEY STEP: Starting Isolated Tests Concurrently if exists...","module":"main"}
{"time":"2026-02-02T13:57:13.665498719Z","level":"INFO","msg":"🔑 KEY STEP: Starting Sanity Test Suite","module":"main"}
{"time":"2026-02-02T13:57:13.665835801Z","level":"INFO","msg":"No isolated tests configured, skipping","module":"main"}
{"time":"2026-02-02T13:57:13.728705489Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":304}
{"time":"2026-02-02T13:57:13.746748287Z","level":"INFO","msg":"Available slots","test":"sanity","node":304,"slots":3000}
{"time":"2026-02-02T13:57:13.746836848Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":305}
{"time":"2026-02-02T13:57:13.755967956Z","level":"INFO","msg":"Available slots","test":"sanity","node":305,"slots":3000}
{"time":"2026-02-02T13:57:13.756153286Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":306}
{"time":"2026-02-02T13:57:13.763986111Z","level":"INFO","msg":"Available slots","test":"sanity","node":306,"slots":2998}
{"time":"2026-02-02T13:57:13.7640599Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":307}
{"time":"2026-02-02T13:57:13.771450733Z","level":"INFO","msg":"Available slots","test":"sanity","node":307,"slots":2997}
{"time":"2026-02-02T13:59:06.639238234Z","level":"INFO","msg":"Extracted keyshares path","test":"sanity","module":"keys","path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json"}
{"time":"2026-02-02T13:59:06.639403745Z","level":"INFO","msg":"Key shares file generated successfully","test":"sanity","keyshare_path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json"}
{"time":"2026-02-02T13:59:13.736457726Z","level":"INFO","msg":"DIAGNOSTIC: Cluster snapshot (pre-registration)","test":"sanity","operator_ids":[304,305,306,307],"cluster_validator_count":1,"cluster_network_fee_index":70469584352,"cluster_index":772007289500,"cluster_active":true,"cluster_balance":"8999999865310720000000"}
{"time":"2026-02-02T13:59:13.736606301Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator amount calculation","test":"sanity","amount_per_validator":"2500000000000000000","validator_count":1,"required_amount":"2500000000000000000","current_cluster_balance":"8999999865310720000000","missing_amount":"-8997499865310720000000","missing_amount_rounded_1e8":"-8997499865310700000000","register_amount":"0"}
{"time":"2026-02-02T13:59:13.736688344Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator inputs prepared","test":"sanity","keyshare_path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json","validator_pubkey":"0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27","shares_data_len_chars":2626,"operator_ids":[304,305,306,307],"amount":"0"}
{"time":"2026-02-02T13:59:13.781732651Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator context","test":"sanity","module":"contract","op":"register_validator","from":"0x5cC0DdE14E7256340CC820415a6022a7d1c93A35","network_contract":"0x0AaACE4e8AFFC47C6834171c88D342A4aBD8f105","views_contract":"0x9143b8C25EfA53F28de4cBeFd0b6dFd66D43FeA6","token_contract":"0x746C33ccC28b1363c35c09baDAF41b2FFa7E6D56","chain_id":"560048","latest_block":2155146,"views_latest_block":2155146,"token_latest_block":2155146,"chain_id_error":"","block_number_error":"","views_block_number_error":"","token_block_number_error":""}
{"time":"2026-02-02T13:59:13.786224533Z","level":"INFO","msg":"DIAGNOSTIC: Network contract code","test":"sanity","module":"contract","op":"register_validator","address":"0x0AaACE4e8AFFC47C6834171c88D342A4aBD8f105","code_len":170}
{"time":"2026-02-02T13:59:13.791237183Z","level":"INFO","msg":"DIAGNOSTIC: Views contract code","test":"sanity","module":"contract","op":"register_validator","address":"0x9143b8C25EfA53F28de4cBeFd0b6dFd66D43FeA6","code_len":170}
{"time":"2026-02-02T13:59:13.796721008Z","level":"INFO","msg":"DIAGNOSTIC: Token contract code","test":"sanity","module":"contract","op":"register_validator","address":"0x746C33ccC28b1363c35c09baDAF41b2FFa7E6D56","code_len":7070}
{"time":"2026-02-02T13:59:13.796971846Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator transact opts (pre-send)","test":"sanity","module":"contract","op":"register_validator","auth_nonce":"<nil>","auth_value":"<nil>","auth_gas_price":"<nil>","auth_gas_fee_cap":"<nil>","auth_gas_tip_cap":"<nil>","auth_gas_limit":0,"auth_no_send":false}
{"time":"2026-02-02T13:59:13.800260493Z","level":"INFO","msg":"DIAGNOSTIC: ETH balance","test":"sanity","module":"contract","op":"register_validator","eth_balance_wei":"330264727950240217710"}
{"time":"2026-02-02T13:59:13.804369235Z","level":"INFO","msg":"DIAGNOSTIC: Token balance","test":"sanity","module":"contract","op":"register_validator","token_balance":"76486370745597760000000"}
{"time":"2026-02-02T13:59:13.804479739Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator operator IDs","test":"sanity","module":"contract","op":"register_validator","operator_ids":[304,305,306,307],"operator_ids_len":4,"operator_ids_sorted_asc":true,"operator_ids_unique":true}
{"time":"2026-02-02T13:59:13.808064846Z","level":"INFO","msg":"DIAGNOSTIC: Validator existence (views.getValidator)","test":"sanity","module":"contract","op":"register_validator","validator_exists":false}
{"time":"2026-02-02T13:59:13.812116451Z","level":"INFO","msg":"DIAGNOSTIC: Cluster balance (views.getBalance)","test":"sanity","module":"contract","op":"register_validator","on_chain_balance":"0","scanner_balance":"8999999865310720000000","balance_match":false}
{"time":"2026-02-02T13:59:13.81236245Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator payload summary","test":"sanity","module":"contract","op":"register_validator","public_key":"0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27","public_key_len_bytes":48,"public_key_keccak256":"0xab8efe86d8039dd91ba1d470441554cb9b9883b2620b7978d61dadd6660032ec","shares_data_len_bytes":1312,"shares_data_keccak256":"0xcfa5209833ccb46bac59cabf2f6df3c8c30703026fdbce0b4b873f1fbdbab60b","amount":"0","amount_is_zero":true,"cluster_validator_count":1,"cluster_network_fee_index":70469584352,"cluster_index":772007289500,"cluster_active":true,"cluster_balance":"8999999865310720000000"}
{"time":"2026-02-02T13:59:13.818053972Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0x5cc0dde14e7256340cc820415a6022a7d1c93a35","allowance":"9999999999999999999999473850789666515400000000"}
{"time":"2026-02-02T13:59:13.818155712Z","level":"INFO","msg":"Threshold value","test":"sanity","module":"contract","threshold":"9000000000000000000000"}
{"time":"2026-02-02T13:59:13.81822144Z","level":"INFO","msg":"Sufficient allowance","test":"sanity","module":"contract","allowance margin":"6561000000000000000000000000000000000000000000000000000000000000000000000000000000000000"}
{"time":"2026-02-02T13:59:13.822401108Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0x5cc0dde14e7256340cc820415a6022a7d1c93a35","allowance":"9999999999999999999999473850789666515400000000"}
{"time":"2026-02-02T13:59:13.822514454Z","level":"INFO","msg":"Allowance","test":"sanity","module":"contract","allowance":"9999999999999999999999473850789666515400000000"}
{"time":"2026-02-02T13:59:13.822788967Z","level":"INFO","msg":"DIAGNOSTIC: registerValidator calldata summary","test":"sanity","module":"contract","op":"register_validator","method_id":"06e8fb9c","calldata_len":1892,"calldata_keccak256":"0xb47992b4fcb9954ba45047118a54ab19f769554438115c7ab0c1826b01218a6b"}
{"time":"2026-02-02T13:59:14.194618135Z","level":"ERROR","msg":"DIAGNOSTIC: Call failed","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator","error_type":"*rpc.jsonError","error":"execution reverted"}
{"time":"2026-02-02T13:59:14.194708821Z","level":"INFO","msg":"DIAGNOSTIC: Raw RPC error data","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator","error_data_type":"string","error_data":"\\"0x\\""}
{"time":"2026-02-02T13:59:14.198692864Z","level":"ERROR","msg":"DIAGNOSTIC: Call failed","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator/estimateGas","error_type":"*rpc.jsonError","error":"execution reverted"}
{"time":"2026-02-02T13:59:14.198788741Z","level":"INFO","msg":"DIAGNOSTIC: Raw RPC error data","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator/estimateGas","error_data_type":"string","error_data":"\\"0x\\""}
{"time":"2026-02-02T13:59:14.20332256Z","level":"ERROR","msg":"DIAGNOSTIC: Call failed","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator/eth_call","error_type":"*rpc.jsonError","error":"execution reverted"}
{"time":"2026-02-02T13:59:14.203445722Z","level":"INFO","msg":"DIAGNOSTIC: Raw RPC error data","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator/eth_call","error_data_type":"string","error_data":"\\"0x\\""}
{"time":"2026-02-02T13:59:14.203523978Z","level":"ERROR","msg":"❌ FAILURE: Sanity Test Suite Failed","module":"main","error":"execution reverted"}
{"time":"2026-02-02T13:59:14.203549967Z","level":"INFO","msg":"⏳ PROGRESS: Waiting for all tests to finish","module":"main"}
{"time":"2026-02-02T13:59:14.203619295Z","level":"ERROR","msg":"❌ FAILURE: Test Execution Completed with Failures","module":"main","passed_suites":null,"failed_suites":["sanity"],"skipped_suites":["isolated"]}
{"time":"2026-02-02T13:59:14.203665136Z","level":"ERROR","msg":"❌ FAILURE: Suite Failure Details","module":"main","suite":"sanity","error":"execution reverted"}
{"time":"2026-02-02T13:59:14.203697131Z","level":"INFO","msg":"🔑 KEY STEP: Starting resource teardown...","module":"main","usage_id":"cb99125ffdcc38ffcdd5c78ca998dabb"}
{"time":"2026-02-02T13:59:14.211999575Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","module":"main","endpoint":"http://aetheria-orchestrator.aetheria.svc:8080"}
{"time":"2026-02-02T13:59:14.212114858Z","level":"INFO","msg":"🧹 Requesting resource teardown","module":"main","usage_id":"cb99125ffdcc38ffcdd5c78ca998dabb"}
{"time":"2026-02-02T13:59:14.374311529Z","level":"INFO","msg":"✅ SUCCESS: Completed resource teardown successfully","module":"main","usage_id":"cb99125ffdcc38ffcdd5c78ca998dabb"}

2026-02-02T13:59:18Z The test run failed with error

Result - 🔴 Failure

2026-02-02T14:14:40Z Finished cleaning reserved resources

vaclav-ssvlabs · 2026-02-02T14:15:03Z

/test cluster=304,305,306,307

ssv-test-bot · 2026-02-02T14:15:07Z

🔴 Test run for PR #2627

Branch: vaclav-test-aetheria-local-testnet
Commit: e5e11c1

Status

2026-02-02T14:15:06Z Starting test run
2026-02-02T14:15:08Z Test run submitted successfully for commit e5e11c1
2026-02-02T14:15:09Z container image for commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is build and present in container registry
2026-02-02T14:15:10Z using user defined node range cluster=304,305,306,307 on network=hoodi_stage
2026-02-02T14:15:12Z deploying commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 to ssv nodes 304,305,306,307
2026-02-02T14:15:13Z commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is deployed to ssv nodes 304,305,306,307
2026-02-02T14:15:15Z started test executor with tests=(sanity) and network=hoodi_stage

Click to expand executor logs

═══════════════════════════════════════════════════════════════════════════════════════════════════════
                🧪  WELCOME TO Executor - SSV's Distributed Test Automation Execution Layer  🧪            

                                  ___ __  __ ___  ___  _   _  _____  ___   ___ 
                                 | __|\\ \\/ /| __|/ __|| | | ||_   _|/ _ \\ | _ \\
                                 | _|  >  < | _|| (__ | |_| |  | | | (_) ||   /
                                 |___|/_/\\_\\|___|\\___| \\___/   |_|  \\___/ |_|_\\
                                        

═══════════════════════════════════════════════════════════════════════════════════════════════════════
{"time":"2026-02-02T14:16:18.990712421Z","level":"INFO","msg":"🐳 Docker mode: Using environment variables for config generation"}
{"time":"2026-02-02T14:16:19.0005044Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","endpoint":"http://aetheria-orchestrator.aetheria.svc:8080"}
{"time":"2026-02-02T14:16:19.000586548Z","level":"INFO","msg":"📝 Requesting configuration generation from orchestrator API","network":"hoodi_stage","clusters":[304,305,306,307],"tests":"(sanity)"}
{"time":"2026-02-02T14:16:19.124676415Z","level":"INFO","msg":"✅ Configuration generated successfully via API","resource_usage_id":"a11d8959e7fa52c0727c9b2a14dc1939"}
{"time":"2026-02-02T14:16:19.125562004Z","level":"INFO","msg":"🚀 EXECUTOR Initialized Successfully"}
{"time":"2026-02-02T14:16:19.125607409Z","level":"INFO","msg":"⚙️ Test configuration loaded","network":"hoodi_stage","nodes":[304,305,306,307],"test_type":{"Sanity":{"ShouldRun":true,"E2M":false,"Threshold":false,"BulkAmount":0},"Bulk":{"ShouldRun":false,"E2M":false,"Threshold":false,"BulkAmount":0},"Isolated":{"Tests":null}}}
{"time":"2026-02-02T14:16:24.225092657Z","level":"INFO","msg":"Generated operator pubkeys file","module":"scanner","path":"/app/shared/data/operators/operator-pubkeys-hoodi_stage.json"}
{"time":"2026-02-02T14:16:24.225238289Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"consensus"}
{"time":"2026-02-02T14:16:24.225258937Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"flow-emits"}
{"time":"2026-02-02T14:16:24.225293308Z","level":"INFO","msg":"Validation client initialized","module":"main","loki_enabled":true,"e2m_enabled":false}
{"time":"2026-02-02T14:16:24.225318159Z","level":"INFO","msg":"🔑 KEY STEP: Starting Isolated Tests Concurrently if exists...","module":"main"}
{"time":"2026-02-02T14:16:24.225353315Z","level":"INFO","msg":"🔑 KEY STEP: Starting Sanity Test Suite","module":"main"}
{"time":"2026-02-02T14:16:24.225455355Z","level":"INFO","msg":"No isolated tests configured, skipping","module":"main"}
{"time":"2026-02-02T14:16:25.412792247Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":304}
{"time":"2026-02-02T14:16:25.566085941Z","level":"INFO","msg":"Available slots","test":"sanity","node":304,"slots":3000}
{"time":"2026-02-02T14:16:25.566141945Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":305}
{"time":"2026-02-02T14:16:25.715800363Z","level":"INFO","msg":"Available slots","test":"sanity","node":305,"slots":753}
{"time":"2026-02-02T14:16:25.715847304Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":306}
{"time":"2026-02-02T14:16:25.795464426Z","level":"INFO","msg":"Available slots","test":"sanity","node":306,"slots":449}
{"time":"2026-02-02T14:16:25.795528639Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":307}
{"time":"2026-02-02T14:16:25.870943411Z","level":"INFO","msg":"Available slots","test":"sanity","node":307,"slots":1070}
{"time":"2026-02-02T14:20:27.017810178Z","level":"INFO","msg":"Extracted keyshares path","test":"sanity","module":"keys","path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json"}
{"time":"2026-02-02T14:20:27.01797273Z","level":"INFO","msg":"Key shares file generated successfully","test":"sanity","keyshare_path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json"}
{"time":"2026-02-02T14:20:33.736948115Z","level":"INFO","msg":"DIAGNOSTIC: Cluster snapshot (pre-registration)","test":"sanity","operator_ids":[304,305,306,307],"cluster_validator_count":1,"cluster_network_fee_index":70469584352,"cluster_index":772007289500,"cluster_active":true,"cluster_balance":"8999999865310720000000"}
{"time":"2026-02-02T14:20:33.737138199Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator amount calculation","test":"sanity","amount_per_validator":"2500000000000000000","validator_count":1,"required_amount":"2500000000000000000","current_cluster_balance":"8999999865310720000000","missing_amount":"-8997499865310720000000","missing_amount_rounded_1e8":"-8997499865310700000000","register_amount":"0"}
{"time":"2026-02-02T14:20:33.737246337Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator inputs prepared","test":"sanity","keyshare_path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json","validator_pubkey":"0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27","shares_data_len_chars":2626,"operator_ids":[304,305,306,307],"amount":"0"}
{"time":"2026-02-02T14:20:33.77343421Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator context","test":"sanity","module":"contract","op":"register_validator","from":"0x5cC0DdE14E7256340CC820415a6022a7d1c93A35","network_contract":"0x0AaACE4e8AFFC47C6834171c88D342A4aBD8f105","views_contract":"0x9143b8C25EfA53F28de4cBeFd0b6dFd66D43FeA6","token_contract":"0x746C33ccC28b1363c35c09baDAF41b2FFa7E6D56","chain_id":"560048","latest_block":2155243,"views_latest_block":2155243,"token_latest_block":2155243,"chain_id_error":"","block_number_error":"","views_block_number_error":"","token_block_number_error":""}
{"time":"2026-02-02T14:20:33.778701539Z","level":"INFO","msg":"DIAGNOSTIC: Network contract code","test":"sanity","module":"contract","op":"register_validator","address":"0x0AaACE4e8AFFC47C6834171c88D342A4aBD8f105","code_len":170}
{"time":"2026-02-02T14:20:33.782554431Z","level":"INFO","msg":"DIAGNOSTIC: Views contract code","test":"sanity","module":"contract","op":"register_validator","address":"0x9143b8C25EfA53F28de4cBeFd0b6dFd66D43FeA6","code_len":170}
{"time":"2026-02-02T14:20:33.787109693Z","level":"INFO","msg":"DIAGNOSTIC: Token contract code","test":"sanity","module":"contract","op":"register_validator","address":"0x746C33ccC28b1363c35c09baDAF41b2FFa7E6D56","code_len":7070}
{"time":"2026-02-02T14:20:33.787206134Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator transact opts (pre-send)","test":"sanity","module":"contract","op":"register_validator","auth_nonce":"<nil>","auth_value":"<nil>","auth_gas_price":"<nil>","auth_gas_fee_cap":"<nil>","auth_gas_tip_cap":"<nil>","auth_gas_limit":0,"auth_no_send":false}
{"time":"2026-02-02T14:20:33.790500721Z","level":"INFO","msg":"DIAGNOSTIC: ETH balance","test":"sanity","module":"contract","op":"register_validator","eth_balance_wei":"330264727950240217710"}
{"time":"2026-02-02T14:20:33.797674571Z","level":"INFO","msg":"DIAGNOSTIC: Token balance","test":"sanity","module":"contract","op":"register_validator","token_balance":"76486370745597760000000"}
{"time":"2026-02-02T14:20:33.79782758Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator operator IDs","test":"sanity","module":"contract","op":"register_validator","operator_ids":[304,305,306,307],"operator_ids_len":4,"operator_ids_sorted_asc":true,"operator_ids_unique":true}
{"time":"2026-02-02T14:20:33.80230264Z","level":"INFO","msg":"DIAGNOSTIC: Validator existence (views.getValidator)","test":"sanity","module":"contract","op":"register_validator","validator_exists":false}
{"time":"2026-02-02T14:20:33.807607172Z","level":"INFO","msg":"DIAGNOSTIC: Cluster balance (views.getBalance)","test":"sanity","module":"contract","op":"register_validator","on_chain_balance":"0","scanner_balance":"8999999865310720000000","balance_match":false}
{"time":"2026-02-02T14:20:33.812456767Z","level":"INFO","msg":"DIAGNOSTIC: Cluster liquidated status (views.isLiquidated)","test":"sanity","module":"contract","op":"register_validator","is_liquidated":false}
{"time":"2026-02-02T14:20:33.816325332Z","level":"ERROR","msg":"DIAGNOSTIC: Call failed","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"views.isLiquidatable","error_type":"*rpc.jsonError","error":"execution reverted"}
{"time":"2026-02-02T14:20:33.816415994Z","level":"INFO","msg":"DIAGNOSTIC: Raw RPC error data","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"views.isLiquidatable","error_data_type":"string","error_data":"\\"0xf6749746\\""}
{"time":"2026-02-02T14:20:33.816501125Z","level":"ERROR","msg":"DIAGNOSTIC: Revert data decoded","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"views.isLiquidatable","revert_data":"0xf6749746","revert_reason":"","custom_error":"","custom_error_args":null}
{"time":"2026-02-02T14:20:33.825200481Z","level":"INFO","msg":"DIAGNOSTIC: Cluster burn rate (views.getBurnRate)","test":"sanity","module":"contract","op":"register_validator","burn_rate":"3550000000"}
{"time":"2026-02-02T14:20:33.830147581Z","level":"INFO","msg":"DIAGNOSTIC: Network fee (views.getNetworkFee)","test":"sanity","module":"contract","op":"register_validator","network_fee":"3550000000"}
{"time":"2026-02-02T14:20:33.833996472Z","level":"INFO","msg":"DIAGNOSTIC: Minimum liquidation collateral (views.getMinimumLiquidationCollateral)","test":"sanity","module":"contract","op":"register_validator","minimum_liquidation_collateral":"940000000000000"}
{"time":"2026-02-02T14:20:33.838233434Z","level":"INFO","msg":"DIAGNOSTIC: Liquidation threshold period (views.getLiquidationThresholdPeriod)","test":"sanity","module":"contract","op":"register_validator","minimum_blocks_before_liquidation":50190}
{"time":"2026-02-02T14:20:33.841953684Z","level":"INFO","msg":"DIAGNOSTIC: Operator details (views.getOperatorById)","test":"sanity","module":"contract","op":"register_validator","operator_id":304,"operator_owner":"0x97a6C1f3aaB5427B901fb135ED492749191C0f1F","operator_fee":"0","operator_validator_count":0,"operator_whitelisted_address":"0x0000000000000000000000000000000000000000","operator_is_private":false,"operator_is_active":false}
{"time":"2026-02-02T14:20:33.84649329Z","level":"INFO","msg":"DIAGNOSTIC: Operator details (views.getOperatorById)","test":"sanity","module":"contract","op":"register_validator","operator_id":305,"operator_owner":"0x97a6C1f3aaB5427B901fb135ED492749191C0f1F","operator_fee":"0","operator_validator_count":0,"operator_whitelisted_address":"0x0000000000000000000000000000000000000000","operator_is_private":false,"operator_is_active":false}
{"time":"2026-02-02T14:20:33.850048657Z","level":"INFO","msg":"DIAGNOSTIC: Operator details (views.getOperatorById)","test":"sanity","module":"contract","op":"register_validator","operator_id":306,"operator_owner":"0x97a6C1f3aaB5427B901fb135ED492749191C0f1F","operator_fee":"0","operator_validator_count":0,"operator_whitelisted_address":"0x0000000000000000000000000000000000000000","operator_is_private":false,"operator_is_active":false}
{"time":"2026-02-02T14:20:33.853893355Z","level":"INFO","msg":"DIAGNOSTIC: Operator details (views.getOperatorById)","test":"sanity","module":"contract","op":"register_validator","operator_id":307,"operator_owner":"0x97a6C1f3aaB5427B901fb135ED492749191C0f1F","operator_fee":"0","operator_validator_count":0,"operator_whitelisted_address":"0x0000000000000000000000000000000000000000","operator_is_private":false,"operator_is_active":false}
{"time":"2026-02-02T14:20:33.854015883Z","level":"INFO","msg":"DIAGNOSTIC: Cluster hashes (off-chain)","test":"sanity","module":"contract","op":"register_validator","cluster_key":"0xa5bd4894a93503b45c3b75e3e49351f5958d0b99d36b105ab45653e74de67d09","clusters_mapping_slot":"0xd56c4f4aab8ca22f9fde432777379f436593c6027698a6995e2daea890bed105","cluster_storage_location":"0x585903bce4f6c480645ba202fade19b331befe2d74bc24efd0e468874af08fc1","cluster_data_hash":"0x5a01074dd5cce7b54989cd9a57154220821074c856bcbe2d8469ec20304b73f7"}
{"time":"2026-02-02T14:20:33.857603311Z","level":"INFO","msg":"DIAGNOSTIC: Cluster hash (on-chain clusters mapping)","test":"sanity","module":"contract","op":"register_validator","on_chain_cluster_data_hash":"0x0000000000000000000000000000000000000000000000000000000000000000","cluster_data_hash_match":false}
{"time":"2026-02-02T14:20:33.857893124Z","level":"INFO","msg":"DIAGNOSTIC: RegisterValidator payload summary","test":"sanity","module":"contract","op":"register_validator","public_key":"0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27","public_key_len_bytes":48,"public_key_keccak256":"0xab8efe86d8039dd91ba1d470441554cb9b9883b2620b7978d61dadd6660032ec","shares_data_len_bytes":1312,"shares_data_keccak256":"0x8a2a55c6cc3ceaca146b09bc9e14967e467b39e5839aecb97c2affbaf689295c","amount":"0","amount_is_zero":true,"cluster_validator_count":1,"cluster_network_fee_index":70469584352,"cluster_index":772007289500,"cluster_active":true,"cluster_balance":"8999999865310720000000"}
{"time":"2026-02-02T14:20:33.86313616Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0x5cc0dde14e7256340cc820415a6022a7d1c93a35","allowance":"9999999999999999999999473850789666515400000000"}
{"time":"2026-02-02T14:20:33.863201837Z","level":"INFO","msg":"Threshold value","test":"sanity","module":"contract","threshold":"9000000000000000000000"}
{"time":"2026-02-02T14:20:33.863232508Z","level":"INFO","msg":"Sufficient allowance","test":"sanity","module":"contract","allowance margin":"6561000000000000000000000000000000000000000000000000000000000000000000000000000000000000"}
{"time":"2026-02-02T14:20:33.867225072Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0x5cc0dde14e7256340cc820415a6022a7d1c93a35","allowance":"9999999999999999999999473850789666515400000000"}
{"time":"2026-02-02T14:20:33.867360102Z","level":"INFO","msg":"Allowance","test":"sanity","module":"contract","allowance":"9999999999999999999999473850789666515400000000"}
{"time":"2026-02-02T14:20:33.867791709Z","level":"INFO","msg":"DIAGNOSTIC: registerValidator calldata summary","test":"sanity","module":"contract","op":"register_validator","method_id":"06e8fb9c","calldata_len":1892,"calldata_keccak256":"0x8124998c02723ae546c2de9159744b2313197b2c4aef6489b868211302ea97b9"}
{"time":"2026-02-02T14:20:33.887616487Z","level":"ERROR","msg":"DIAGNOSTIC: Call failed","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator","error_type":"*rpc.jsonError","error":"execution reverted"}
{"time":"2026-02-02T14:20:33.887858416Z","level":"INFO","msg":"DIAGNOSTIC: Raw RPC error data","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator","error_data_type":"<nil>","error_data":"<nil>"}
{"time":"2026-02-02T14:20:33.891426502Z","level":"ERROR","msg":"DIAGNOSTIC: Call failed","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator/estimateGas","error_type":"*rpc.jsonError","error":"execution reverted"}
{"time":"2026-02-02T14:20:33.891503731Z","level":"INFO","msg":"DIAGNOSTIC: Raw RPC error data","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator/estimateGas","error_data_type":"<nil>","error_data":"<nil>"}
{"time":"2026-02-02T14:20:33.895364621Z","level":"ERROR","msg":"DIAGNOSTIC: Call failed","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator/eth_call","error_type":"*rpc.jsonError","error":"execution reverted"}
{"time":"2026-02-02T14:20:33.895459567Z","level":"INFO","msg":"DIAGNOSTIC: Raw RPC error data","test":"sanity","module":"contract","op":"register_validator","diagnostic_label":"network.registerValidator/eth_call","error_data_type":"string","error_data":"\\"0x\\""}
{"time":"2026-02-02T14:20:33.895515875Z","level":"ERROR","msg":"❌ FAILURE: Sanity Test Suite Failed","module":"main","error":"execution reverted"}
{"time":"2026-02-02T14:20:33.895587115Z","level":"INFO","msg":"⏳ PROGRESS: Waiting for all tests to finish","module":"main"}
{"time":"2026-02-02T14:20:33.895681416Z","level":"ERROR","msg":"❌ FAILURE: Test Execution Completed with Failures","module":"main","passed_suites":null,"failed_suites":["sanity"],"skipped_suites":["isolated"]}
{"time":"2026-02-02T14:20:33.895794634Z","level":"ERROR","msg":"❌ FAILURE: Suite Failure Details","module":"main","suite":"sanity","error":"execution reverted"}
{"time":"2026-02-02T14:20:33.895856921Z","level":"INFO","msg":"🔑 KEY STEP: Starting resource teardown...","module":"main","usage_id":"a11d8959e7fa52c0727c9b2a14dc1939"}
{"time":"2026-02-02T14:20:33.905475385Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","module":"main","endpoint":"http://aetheria-orchestrator.aetheria.svc:8080"}
{"time":"2026-02-02T14:20:33.905638953Z","level":"INFO","msg":"🧹 Requesting resource teardown","module":"main","usage_id":"a11d8959e7fa52c0727c9b2a14dc1939"}
{"time":"2026-02-02T14:20:33.943658233Z","level":"INFO","msg":"✅ SUCCESS: Completed resource teardown successfully","module":"main","usage_id":"a11d8959e7fa52c0727c9b2a14dc1939"}

2026-02-02T14:20:36Z The test run failed with error

Result - 🔴 Failure

2026-02-02T14:35:40Z Finished cleaning reserved resources

vaclav-ssvlabs · 2026-02-02T14:50:33Z

/test cluster=304,305,306,307

ssv-test-bot · 2026-02-02T14:50:38Z

🔴 Test run for PR #2627

Branch: vaclav-test-aetheria-local-testnet
Commit: e5e11c1

Status

2026-02-02T14:50:37Z Starting test run
2026-02-02T14:50:39Z Test run submitted successfully for commit e5e11c1
2026-02-02T14:50:40Z container image for commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is build and present in container registry
2026-02-02T14:50:42Z using user defined node range cluster=304,305,306,307 on network=hoodi_stage
2026-02-02T14:50:44Z deploying commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 to ssv nodes 304,305,306,307
2026-02-02T14:51:30Z commit e5e11c13a2445ecfe4999ab39f4fa75b5569d102 is deployed to ssv nodes 304,305,306,307
2026-02-02T14:51:32Z started test executor with tests=(sanity) and network=hoodi_stage

Click to expand executor logs

═══════════════════════════════════════════════════════════════════════════════════════════════════════
                🧪  WELCOME TO Executor - SSV's Distributed Test Automation Execution Layer  🧪            

                                  ___ __  __ ___  ___  _   _  _____  ___   ___ 
                                 | __|\\ \\/ /| __|/ __|| | | ||_   _|/ _ \\ | _ \\
                                 | _|  >  < | _|| (__ | |_| |  | | | (_) ||   /
                                 |___|/_/\\_\\|___|\\___| \\___/   |_|  \\___/ |_|_\\
                                        

═══════════════════════════════════════════════════════════════════════════════════════════════════════
{"time":"2026-02-02T14:51:40.701828517Z","level":"INFO","msg":"🐳 Docker mode: Using environment variables for config generation"}
{"time":"2026-02-02T14:51:40.713366454Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","endpoint":"http://aetheria-orchestrator.aetheria.svc:8080"}
{"time":"2026-02-02T14:51:40.713711697Z","level":"INFO","msg":"📝 Requesting configuration generation from orchestrator API","network":"hoodi_stage","clusters":[304,305,306,307],"tests":"(sanity)"}
{"time":"2026-02-02T14:51:40.799816636Z","level":"INFO","msg":"✅ Configuration generated successfully via API","resource_usage_id":"c7ca25845440cf812aedfce502ca785c"}
{"time":"2026-02-02T14:51:40.80112145Z","level":"INFO","msg":"🚀 EXECUTOR Initialized Successfully"}
{"time":"2026-02-02T14:51:40.801206524Z","level":"INFO","msg":"⚙️ Test configuration loaded","network":"hoodi_stage","nodes":[304,305,306,307],"test_type":{"Sanity":{"ShouldRun":true,"E2M":false,"Threshold":false,"BulkAmount":0},"Bulk":{"ShouldRun":false,"E2M":false,"Threshold":false,"BulkAmount":0},"Isolated":{"Tests":null}}}
{"time":"2026-02-02T14:51:45.579936124Z","level":"INFO","msg":"Generated operator pubkeys file","module":"scanner","path":"/app/shared/data/operators/operator-pubkeys-hoodi_stage.json"}
{"time":"2026-02-02T14:51:45.580145726Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"consensus"}
{"time":"2026-02-02T14:51:45.580162219Z","level":"INFO","msg":"Registered parser","module":"main","component":"parser-client","parser_type":"flow-emits"}
{"time":"2026-02-02T14:51:45.580226136Z","level":"INFO","msg":"Validation client initialized","module":"main","loki_enabled":true,"e2m_enabled":false}
{"time":"2026-02-02T14:51:45.580252513Z","level":"INFO","msg":"🔑 KEY STEP: Starting Isolated Tests Concurrently if exists...","module":"main"}
{"time":"2026-02-02T14:51:45.580284638Z","level":"INFO","msg":"🔑 KEY STEP: Starting Sanity Test Suite","module":"main"}
{"time":"2026-02-02T14:51:45.580535746Z","level":"INFO","msg":"No isolated tests configured, skipping","module":"main"}
{"time":"2026-02-02T14:51:45.626991568Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":304}
{"time":"2026-02-02T14:51:45.641444906Z","level":"INFO","msg":"Available slots","test":"sanity","node":304,"slots":3000}
{"time":"2026-02-02T14:51:45.641505744Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":305}
{"time":"2026-02-02T14:51:45.648523078Z","level":"INFO","msg":"Available slots","test":"sanity","node":305,"slots":3000}
{"time":"2026-02-02T14:51:45.648636392Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":306}
{"time":"2026-02-02T14:51:45.658906271Z","level":"INFO","msg":"Available slots","test":"sanity","node":306,"slots":2998}
{"time":"2026-02-02T14:51:45.658990305Z","level":"INFO","msg":"Getting available slots","test":"sanity","node":307}
{"time":"2026-02-02T14:51:45.665297787Z","level":"INFO","msg":"Available slots","test":"sanity","node":307,"slots":2997}
{"time":"2026-02-02T14:55:38.182273513Z","level":"INFO","msg":"Extracted keyshares path","test":"sanity","module":"keys","path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json"}
{"time":"2026-02-02T14:55:38.182384919Z","level":"INFO","msg":"Key shares file generated successfully","test":"sanity","keyshare_path":"/app/shared/data/keyshares/0xa5ca3ded89c314e0f707c8753208f53d8778197ace8463544ec2144e701b32eeaeb2f3ea28c2c3cacc6e11adcc219d27.json"}
{"time":"2026-02-02T14:55:44.740248104Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0x5cc0dde14e7256340cc820415a6022a7d1c93a35","allowance":"9999999999999999999999473850789666515400000000"}
{"time":"2026-02-02T14:55:44.740332681Z","level":"INFO","msg":"Threshold value","test":"sanity","module":"contract","threshold":"9000000000000000000000"}
{"time":"2026-02-02T14:55:44.740354861Z","level":"INFO","msg":"Sufficient allowance","test":"sanity","module":"contract","allowance margin":"6561000000000000000000000000000000000000000000000000000000000000000000000000000000000000"}
{"time":"2026-02-02T14:55:44.745473222Z","level":"INFO","msg":"Allowance fetched successfully","test":"sanity","module":"contract","account":"0x5cc0dde14e7256340cc820415a6022a7d1c93a35","allowance":"9999999999999999999999473850789666515400000000"}
{"time":"2026-02-02T14:55:44.745548791Z","level":"INFO","msg":"Allowance","test":"sanity","module":"contract","allowance":"9999999999999999999999473850789666515400000000"}
{"time":"2026-02-02T14:55:44.766598777Z","level":"ERROR","msg":"Failed to register validator","test":"sanity","module":"contract","error":"execution reverted"}
{"time":"2026-02-02T14:55:44.766798491Z","level":"ERROR","msg":"❌ FAILURE: Sanity Test Suite Failed","module":"main","error":"execution reverted"}
{"time":"2026-02-02T14:55:44.766843528Z","level":"INFO","msg":"⏳ PROGRESS: Waiting for all tests to finish","module":"main"}
{"time":"2026-02-02T14:55:44.766943411Z","level":"ERROR","msg":"❌ FAILURE: Test Execution Completed with Failures","module":"main","passed_suites":null,"failed_suites":["sanity"],"skipped_suites":["isolated"]}
{"time":"2026-02-02T14:55:44.76701426Z","level":"ERROR","msg":"❌ FAILURE: Suite Failure Details","module":"main","suite":"sanity","error":"execution reverted"}
{"time":"2026-02-02T14:55:44.767050198Z","level":"INFO","msg":"🔑 KEY STEP: Starting resource teardown...","module":"main","usage_id":"c7ca25845440cf812aedfce502ca785c"}
{"time":"2026-02-02T14:55:44.776001032Z","level":"INFO","msg":"🌐 Orchestrator API Client initialized","module":"main","endpoint":"http://aetheria-orchestrator.aetheria.svc:8080"}
{"time":"2026-02-02T14:55:44.776126953Z","level":"INFO","msg":"🧹 Requesting resource teardown","module":"main","usage_id":"c7ca25845440cf812aedfce502ca785c"}
{"time":"2026-02-02T14:55:44.813272316Z","level":"INFO","msg":"✅ SUCCESS: Completed resource teardown successfully","module":"main","usage_id":"c7ca25845440cf812aedfce502ca785c"}

2026-02-02T14:55:48Z The test run failed with error

Result - 🔴 Failure

2026-02-02T15:10:52Z Finished cleaning reserved resources

vaclav-test-aetheria-local-testnet

e998ec6

vaclav-ssvlabs requested review from a team as code owners December 17, 2025 09:23

Conversation

vaclav-ssvlabs commented Dec 17, 2025

Uh oh!

codecov bot commented Dec 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

ssv-test-bot bot commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔴 Test run for PR #2627

Status

Result - 🔴 Failure

Uh oh!

vaclav-ssvlabs commented Dec 18, 2025

Uh oh!

ssv-test-bot bot commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔴 Test run for PR #2627

Status

Result - 🔴 Failure

Uh oh!

vaclav-ssvlabs commented Dec 18, 2025

Uh oh!

ssv-test-bot bot commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔴 Test run for PR #2627

Status

Result - 🔴 Failure

Uh oh!

vaclav-ssvlabs commented Dec 18, 2025

Uh oh!

ssv-test-bot bot commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🟡 Test run for PR #2627

Status

Uh oh!

vaclav-ssvlabs commented Dec 18, 2025

Uh oh!

ssv-test-bot bot commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔴 Test run for PR #2627

Status

Result - 🔴 Failure

Uh oh!

vaclav-ssvlabs commented Dec 19, 2025

Uh oh!

ssv-test-bot bot commented Dec 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔴 Test run for PR #2627

Status

Result - 🔴 Failure

Uh oh!

vaclav-ssvlabs commented Dec 19, 2025

Uh oh!

ssv-test-bot bot commented Dec 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔴 Test run for PR #2627

Status

Result - 🔴 Failure

Uh oh!

vaclav-ssvlabs commented Dec 19, 2025

Uh oh!

ssv-test-bot bot commented Dec 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔴 Test run for PR #2627

Status

Result - 🔴 Failure

Uh oh!

vaclav-ssvlabs commented Dec 19, 2025

Uh oh!

ssv-test-bot bot commented Dec 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔴 Test run for PR #2627

Status

Result - 🔴 Failure

Uh oh!

vaclav-ssvlabs commented Dec 19, 2025

Uh oh!

ssv-test-bot bot commented Dec 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

codecov bot commented Dec 17, 2025 •

edited

Loading

ssv-test-bot bot commented Dec 18, 2025 •

edited

Loading

ssv-test-bot bot commented Dec 18, 2025 •

edited

Loading

ssv-test-bot bot commented Dec 18, 2025 •

edited

Loading

ssv-test-bot bot commented Dec 18, 2025 •

edited

Loading

ssv-test-bot bot commented Dec 18, 2025 •

edited

Loading

ssv-test-bot bot commented Dec 19, 2025 •

edited

Loading

ssv-test-bot bot commented Dec 19, 2025 •

edited

Loading

ssv-test-bot bot commented Dec 19, 2025 •

edited

Loading

ssv-test-bot bot commented Dec 19, 2025 •

edited

Loading

ssv-test-bot bot commented Dec 19, 2025 •

edited

Loading

ssv-test-bot bot commented Dec 20, 2025 •

edited

Loading

ssv-test-bot bot commented Dec 20, 2025 •

edited

Loading

ssv-test-bot bot commented Dec 20, 2025 •

edited

Loading

ssv-test-bot bot commented Jan 29, 2026 •

edited

Loading

ssv-test-bot bot commented Jan 29, 2026 •

edited

Loading

ssv-test-bot bot commented Jan 29, 2026 •

edited

Loading

ssv-test-bot bot commented Jan 29, 2026 •

edited

Loading

ssv-test-bot bot commented Jan 29, 2026 •

edited

Loading