ci(l1): fail L1 job if hive tests fail #4827

mpaulucci · 2025-10-09T15:52:56Z

Motivation
When a Hive job failed, the L1 workflow didn't fail. This is because we started using ethpandaops/hive-github-action

Description

Adds a script to read log files from the Hive jobs and fail if any of the tests failed
Adds the list of failed jobs to the job summary
Uploads the logs of the simulation with the failed Hive tests.
Merged snap and full sync into one simulation (we were testing the same thing twice, this PR should fix it clients/ethrex: add full and snap sync tests. Default to full sync ethereum/hive#1353

Here is an example of a run with failing hive tests:
https://github.com/lambdaclass/ethrex/actions/runs/18411165213?pr=4827

github-actions · 2025-10-13T18:11:16Z

Lines of code report

Total lines added: 0
Total lines removed: 2
Total lines changed: 2

Detailed view

+----------------------------------------------------+-------+------+
| File                                               | Lines | Diff |
+----------------------------------------------------+-------+------+
| ethrex/crates/networking/rpc/engine/fork_choice.rs | 354   | -2   |
+----------------------------------------------------+-------+------+

…-hive-fails

mpaulucci · 2025-10-14T18:47:51Z

.github/scripts/check-hive-results.sh

@@ -0,0 +1,192 @@
+#!/usr/bin/env bash


This script searches for the log files of failed tests and updates them to an artifact

mpaulucci · 2025-10-14T18:48:02Z

.github/workflows/pr-main_l1.yaml

          - name: "Rpc Compat tests"
            simulation: ethereum/rpc-compat
-            limit: ""
+            limit: "rpc-compat/(debug_[^/]+/.*|eth_blobBaseFee/.*|eth_blockNumber/.*|eth_call/.*|eth_chainId/.*|eth_createAccessList/.*|eth_estimateGas/.*|eth_feeHistory/.*|eth_getBalance/.*|eth_getBlockByHash/.*|eth_getBlockByNumber/.*|eth_getBlockReceipts/.*|eth_getBlockTransactionCountByHash/.*|eth_getBlockTransactionCountByNumber/.*|eth_getCode/.*|eth_getLogs/.*|eth_getProof/.*|eth_getStorageAt/.*|eth_getTransactionByBlockHashAndIndex/.*|eth_getTransactionByBlockNumberAndIndex/.*|eth_getTransactionByHash/.*|eth_getTransactionCount/.*|eth_getTransactionReceipt/.*|eth_sendRawTransaction/.*)"


eth-syncing is now failing

Can we add a comment mentioning this?

mpaulucci · 2025-10-14T18:48:17Z

.github/workflows/pr-main_l1.yaml

            simulation: devp2p
-            limit: discv4|eth|snap/Ping|Findnode/WithoutEndpointProof|Findnode/PastExpiration|Amplification|Status|StorageRanges|ByteCodes|GetBlockHeaders|SimultaneousRequests|SameRequestID|ZeroRequestID|GetBlockBodies|MaliciousHandshake|MaliciousStatus|Transaction|NewPooledTxs|GetBlockReceipts|LargeTxRequest|InvalidTxs|BlockRangeUpdate
-            # AccountRange and GetTrieNodes don't pass anymore.
+            limit: discv4|eth|snap/Ping|Findnode/WithoutEndpointProof|Findnode/PastExpiration|Amplification|Status|StorageRanges|ByteCodes|GetBlockHeaders|SimultaneousRequests|SameRequestID|ZeroRequestID|GetBlockBodies|MaliciousHandshake|MaliciousStatus|NewPooledTxs|GetBlockReceipts|BlockRangeUpdate|GetTrieNodes


there were some regressions

Copilot

Pull Request Overview

This PR modifies the CI workflow to properly fail L1 jobs when Hive tests fail, addressing an issue where the workflow would continue even when tests failed due to the use of ethpandaops/hive-github-action.

Adds a custom script to parse Hive JSON results and fail the workflow when tests fail
Updates Hive versions and consolidates duplicate sync tests
Adds artifact upload for failed test logs with GitHub summary integration

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
`.github/workflows/pr-main_l1.yaml`	Updates workflow to check Hive results, upload failure logs, and consolidates test configurations
`.github/scripts/check-hive-results.sh`	New script that parses Hive JSON results, identifies failures, and creates detailed failure reports

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-10-14T18:48:19Z

.github/workflows/pr-main_l1.yaml

      - name: Run unit tests
        run: |
          make test



The conditional logic skipping EF tests for merge groups lacks documentation. Consider adding a comment explaining why EF tests are skipped for merge group events.

Suggested change

# EF tests are skipped for merge group events to reduce CI load and avoid redundant test runs,

# since merge group events are used for speculative merges and do not require full EF test coverage.

.github/workflows/pr-main_l1.yaml

Copilot · 2025-10-14T18:48:20Z

.github/scripts/check-hive-results.sh

+    suite_slug="$(printf '%s' "${suite_slug_raw}" | tr '[:upper:]' '[:lower:]')"
+    suite_slug="$(printf '%s' "${suite_slug}" | sed -E 's/[^a-z0-9._-]+/-/g')"
+    suite_slug="${suite_slug#-}"
+    suite_slug="${suite_slug%-}"


Multiple separate calls to external commands for string manipulation could be combined into a single operation or use bash parameter expansion for better performance.

Suggested change

suite_slug="$(printf '%s' "${suite_slug_raw}" | tr '[:upper:]' '[:lower:]')"

suite_slug="$(printf '%s' "${suite_slug}" | sed -E 's/[^a-z0-9._-]+/-/g')"

suite_slug="${suite_slug#-}"

suite_slug="${suite_slug%-}"

suite_slug="${suite_slug_raw,,}" # lowercase

suite_slug="${suite_slug//[^a-z0-9._-]/-}" # replace non-matching chars with '-'

suite_slug="${suite_slug#-}" # trim leading dash

suite_slug="${suite_slug%-}" # trim trailing dash

Copilot · 2025-10-14T18:48:20Z

.github/scripts/check-hive-results.sh

+
+          for search_root in "${search_roots[@]}"; do
+            [ -d "${search_root}" ] || continue
+            found_log="$(find "${search_root}" -type f -name "$(basename "${log_rel}")" -print -quit 2>/dev/null || true)"


The find command with basename could be inefficient for large directory trees. Consider using more specific search patterns or limiting search depth if the log structure is known.

Suggested change

found_log="$(find "${search_root}" -type f -name "$(basename "${log_rel}")" -print -quit 2>/dev/null || true)"

found_log="$(find "${search_root}" -maxdepth 2 -type f -name "$(basename "${log_rel}")" -print -quit 2>/dev/null || true)"

mpaulucci · 2025-10-14T18:48:22Z

.github/workflows/pr-main_l1.yaml

-            hive_repository: ethereum/hive
-            hive_version: 2b7a9c007770b10cb1a242a6c6de88c87a383e5a
-            artifact_prefix: engine_cancun
+          # - name: "Cancun Engine tests"


mpaulucci · 2025-10-14T18:48:31Z

.github/workflows/pr-main_l1.yaml

-            hive_repository: ethereum/hive
-            hive_version: 2b7a9c007770b10cb1a242a6c6de88c87a383e5a
-            artifact_prefix: sync_snap
+          # Investigate this test


github-actions bot assigned mpaulucci Oct 9, 2025

github-actions bot added the L1 Ethereum client label Oct 9, 2025

github-project-automation bot added this to ethrex_l1 Oct 9, 2025

Print failed jobs and fail if a hive job failed.

6c2a849

mpaulucci force-pushed the fail-job-if-hive-fails branch from afc11b1 to 6c2a849 Compare October 10, 2025 13:45

mpaulucci added 4 commits October 10, 2025 16:22

Upload failed tests to a file.

71798a1

Move script to another file.

f6fbd90

Dont upload artifacts.

9370a21

Use one sync simulation.

6e884bd

mpaulucci added 2 commits October 14, 2025 15:52

Merge branch 'main' of github.com:lambdaclass/ethrex into fail-job-if…

b27761b

…-hive-fails

Update hive version.

9dae1ce

mpaulucci force-pushed the fail-job-if-hive-fails branch from c4a5f99 to 9dae1ce Compare October 14, 2025 13:54

mpaulucci added 5 commits October 14, 2025 18:31

Merge branch 'main' of github.com:lambdaclass/ethrex into fail-job-if…

49ef6d7

…-hive-fails

Comment out suites that are failing.

dce2e92

Improve script.

39503f8

Remove rpc compat test that is failing.

018e9ab

Update limit

bbb45fe

mpaulucci marked this pull request as ready for review October 14, 2025 18:47

mpaulucci requested a review from a team as a code owner October 14, 2025 18:47

Copilot AI review requested due to automatic review settings October 14, 2025 18:47

ethrex-project-sync bot moved this to In Review in ethrex_l1 Oct 14, 2025

mpaulucci commented Oct 14, 2025

View reviewed changes

Copilot AI reviewed Oct 14, 2025

View reviewed changes

mpaulucci commented Oct 14, 2025

View reviewed changes

fedacking approved these changes Oct 14, 2025

View reviewed changes

mpaulucci enabled auto-merge October 14, 2025 19:20

MegaRedHand approved these changes Oct 14, 2025

View reviewed changes

mpaulucci added this pull request to the merge queue Oct 14, 2025

cdiielsi approved these changes Oct 14, 2025

View reviewed changes

Merged via the queue into main with commit f09eb24 Oct 14, 2025
42 checks passed

mpaulucci deleted the fail-job-if-hive-fails branch October 14, 2025 21:29

github-project-automation bot moved this from In Review to Done in ethrex_l1 Oct 14, 2025



	# EF tests are skipped for merge group events to reduce CI load and avoid redundant test runs,
	# since merge group events are used for speculative merges and do not require full EF test coverage.

	found_log="$(find "${search_root}" -type f -name "$(basename "${log_rel}")" -print -quit 2>/dev/null \|\| true)"
	found_log="$(find "${search_root}" -maxdepth 2 -type f -name "$(basename "${log_rel}")" -print -quit 2>/dev/null \|\| true)"

ci(l1): fail L1 job if hive tests fail #4827

ci(l1): fail L1 job if hive tests fail #4827

Uh oh!

Conversation

mpaulucci commented Oct 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 13, 2025

Lines of code report

Uh oh!

mpaulucci Oct 14, 2025

Choose a reason for hiding this comment

Uh oh!

mpaulucci Oct 14, 2025

Choose a reason for hiding this comment

Uh oh!

MegaRedHand Oct 14, 2025

Choose a reason for hiding this comment

Uh oh!

mpaulucci Oct 14, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Oct 14, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Oct 14, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 14, 2025

Choose a reason for hiding this comment

Uh oh!

mpaulucci Oct 14, 2025

Choose a reason for hiding this comment

Uh oh!

mpaulucci Oct 14, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

mpaulucci commented Oct 9, 2025 •

edited

Loading