Running report action within matrix overwrites results

[jbeemster]

Hi there - we have been trying to use this action within a matrix and we are running into an issue where it each run within the matrix ends up deleting / overwriting prior results as all of them end up with the same github_run_id and github_run_num.

Is it possible to publish results as part of a matrix execution? Can we manually override the run_id or run_num to solve this? Or do we need to coalesce the matrix back to have this action run as a singleton?

[mgrybyk]

Hello @jbeemster,

Please consider using one of these:

• https://github.com/mgrybyk/allure-report-branch-js-action (requires runners to have Java installed, should be there by default)
• https://github.com/mgrybyk/allure-report-branch-action (docker action)

[mjburch]

@jbeemster, in our case, we were using matrix to split one large test project across different runners in parallel to save time.

To get a combined report, what we did was at the end of our matrix, upload the partial report as an artifact named allure-results-${{ strategy.job-index }}. Then we created a separate job that ran after the matrix job which downloaded all the report artifacts and merged them into one directory before then publishing that.

Something like:

jobs:
  run-ui-tests:
    runs-on: 'ubuntu-24.04'
    strategy:
      fail-fast: false
      matrix:
        environmentName: ["env1", "env2", "env3"]
    env:
      jobTotal: ${{ strategy.job-total }}
      jobIndex: ${{ strategy.job-index }}
      envName: ${{ matrix.environmentName }}
    steps:
      - uses: actions/checkout@v4
      - name: Filter tests
        runs: |
          # Partitions available tests and outputs 'testFilter' expression
          ./.github/workflows/scripts/filter-tests.sh "${jobTotal}" "${jobIndex}"
      - name: Run tests
        runs: 'dotnet test UITests --filter "${testFilter}" -e ENV_NAME=${envName}'
      - name: Upload Test Results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: allure-results-${{ strategy.job-index }}
          path: UITests/bin/Debug/net9.0/allure-results 

  publish-ui-tests:
    runs-on: 'ubuntu-24.04'
    needs: run-ui-tests
    if: always()
    steps:
      - name: Merge Artifacts
        uses: actions/download-artifact@v4
        with:
          path: allure-results-combined
          pattern: allure-results*
          merge-multiple: true

      - name: 'Load test report history'
        uses: actions/checkout@v4
        if: always()
        continue-on-error: true
        with:
          ref: gh-pages
          path: gh-pages

      - name: Build test report
        uses: simple-elf/allure-report-action@v1.12
        if: always()
        with:
          gh_pages: gh-pages
          allure_results: allure-results-combined
          allure_history: allure-history
          allure_report: allure-report

      - name: Publish test report
        uses: peaceiris/actions-gh-pages@v4
        if: always()
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          publish_branch: gh-pages
          publish_dir: allure-history