Skip to content

Added automated accessiblity tests#1396

Open
efinder2 wants to merge 4 commits into
havit:mainfrom
efinder2:feature/axeTest
Open

Added automated accessiblity tests#1396
efinder2 wants to merge 4 commits into
havit:mainfrom
efinder2:feature/axeTest

Conversation

@efinder2

@efinder2 efinder2 commented May 4, 2026

Copy link
Copy Markdown
Contributor

I've added the execution of automated accessibility tests via axe as discussed in #1192. This is at the moment only proof of concept. I'm not sure where to run them correctly. Currently they are run in the clean up method. It was an easy way to execute it after every test, so i don't have to copy every test. This may not be the final way. The current implementation causes problems with the termination of the chrome headless shell. They are orphans after the execution. It freezes my Laptop with 16GB of RAM. With 32 on my Desktop the tests run through and are killed after the execution by the IDE.

I added a list of the found problems.
axe-results-20260504-194743.md

There are multiple ways to implement them further. I don't recommend to add them directly to the pipeline. There are too many findings to fix them at once. So there needs to be a switch to enable them. In the current implementation this is solved by a compile directive.
The easiest way would be to copy the tests, add a RunAxe at the end and filter by a TestCategory. An alternative would be to call RunAxe after each test (and in relevant steps in between). This method filters by environment variables if the accessibility tests are activated.
Alternatively we run them always. For now we could exclude all rules with warnings or suppress the rule for some css selectors if they aren't many of this category.

@hakenr

hakenr commented May 22, 2026

Copy link
Copy Markdown
Member

@copilot resolve the merge conflicts in this pull request

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a proof-of-concept for running automated accessibility (axe) checks as part of the existing Playwright-based E2E test suite, gated behind a dedicated build configuration.

Changes:

  • Adds an ACCESSIBLITYTESTS build configuration across the solution/projects to toggle accessibility checks.
  • Integrates Playwright.Axe into Havit.Blazor.E2ETests and runs axe scans from TestCleanup, aggregating results into a Markdown report.
  • Updates centralized package versions and contributing docs related to running the tests.

Reviewed changes

Copilot reviewed 35 out of 35 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
Havit.SourceGenerators.StrongApiStringLocalizers/Havit.SourceGenerators.StrongApiStringLocalizers.csproj Adds custom configuration/platform entries.
Havit.SourceGenerators.StrongApiStringLocalizers.Tests/Havit.SourceGenerators.StrongApiStringLocalizers.Tests.csproj Adds custom configuration/platform entries.
Havit.Extensions.Localization/Havit.Extensions.Localization.csproj Adds custom configuration/platform entries.
Havit.Blazor.TestApp/Havit.Blazor.TestApp/Havit.Blazor.TestApp.csproj Adds custom configuration/platform entries.
Havit.Blazor.TestApp/Havit.Blazor.TestApp.Client/Havit.Blazor.TestApp.Client.csproj Adds custom configuration/platform entries.
Havit.Blazor.slnx Adds solution build configuration for accessibility tests.
Havit.Blazor.Grpc.TestContracts/Havit.Blazor.Grpc.TestContracts.csproj Adds custom configuration/platform entries.
Havit.Blazor.Grpc.Server/Havit.Blazor.Grpc.Server.csproj Adds custom configuration/platform entries.
Havit.Blazor.Grpc.Server.Tests/Havit.Blazor.Grpc.Server.Tests.csproj Adds custom configuration/platform entries.
Havit.Blazor.Grpc.Core/Havit.Blazor.Grpc.Core.csproj Adds custom configuration/platform entries.
Havit.Blazor.Grpc.Core.Tests/Havit.Blazor.Grpc.Core.Tests.csproj Adds custom configuration/platform entries.
Havit.Blazor.Grpc.Client/Havit.Blazor.Grpc.Client.csproj Adds custom configuration/platform entries.
Havit.Blazor.Grpc.Client.WebAssembly/Havit.Blazor.Grpc.Client.WebAssembly.csproj Adds custom configuration/platform entries.
Havit.Blazor.Grpc.Client.Tests/Havit.Blazor.Grpc.Client.Tests.csproj Adds custom configuration/platform entries.
Havit.Blazor.Grpc.Client.ServerSideRendering/Havit.Blazor.Grpc.Client.ServerSideRendering.csproj Adds custom configuration/platform entries.
Havit.Blazor.GoogleTagManager/Havit.Blazor.GoogleTagManager.csproj Adds custom configuration/platform entries.
Havit.Blazor.E2ETests/TestAppTestBase.cs Adds axe execution in test cleanup and report aggregation.
Havit.Blazor.E2ETests/TestAppAssemblyInitializer.cs Writes aggregated axe results during assembly cleanup.
Havit.Blazor.E2ETests/Havit.Blazor.E2ETests.csproj Adds configuration/platform entries and references Playwright.Axe.
Havit.Blazor.Documentation/Havit.Blazor.Documentation.csproj Adds custom configuration/platform entries.
Havit.Blazor.Documentation.Tests/Havit.Blazor.Documentation.Tests.csproj Adds custom configuration/platform entries.
Havit.Blazor.Documentation.Server/Havit.Blazor.Documentation.Server.csproj Adds custom configuration/platform entries.
Havit.Blazor.Documentation.RepoDumpGenerator/Havit.Blazor.Documentation.RepoDumpGenerator.csproj Adds custom configuration/platform entries.
Havit.Blazor.Documentation.Mcp/Havit.Blazor.Documentation.Mcp.csproj Adds custom configuration/platform entries.
Havit.Blazor.Components.Web/Havit.Blazor.Components.Web.csproj Adds custom configuration/platform entries.
Havit.Blazor.Components.Web.Tests/Havit.Blazor.Components.Web.Tests.csproj Adds custom configuration/platform entries.
Havit.Blazor.Components.Web.ECharts/Havit.Blazor.Components.Web.ECharts.csproj Adds custom configuration/platform entries.
Havit.Blazor.Components.Web.Bootstrap/Havit.Blazor.Components.Web.Bootstrap.csproj Adds custom configuration/platform entries.
Havit.Blazor.Components.Web.Bootstrap.Tests/Havit.Blazor.Components.Web.Bootstrap.Tests.csproj Adds custom configuration/platform entries.
Havit.Blazor.Components.Web.Bootstrap.Smart/Havit.Blazor.Components.Web.Bootstrap.Smart.csproj Adds custom configuration/platform entries.
Havit.Blazor.Components.Web.Bootstrap.InternalSourceGenerators/Havit.Blazor.Components.Web.Bootstrap.InternalSourceGenerators.csproj Adds custom configuration/platform entries.
Havit.Blazor.Components.Web.Bootstrap.EntityFrameworkCore.Tests/Havit.Blazor.Components.Web.Bootstrap.EntityFrameworkCore.Tests.csproj Adds custom configuration/platform entries.
Directory.Packages.props Adds central package versions for axe/playwright integration.
CONTRIBUTING.md Adds initial guidance for running accessibility tests (browser install).
BlazorAppTest/BlazorAppTest.csproj Adds custom configuration/platform entries.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread Havit.Blazor.E2ETests/TestAppTestBase.cs
Comment thread Havit.Blazor.E2ETests/TestAppTestBase.cs Outdated
Comment thread Havit.Blazor.E2ETests/TestAppTestBase.cs Outdated
Comment thread Havit.Blazor.E2ETests/TestAppTestBase.cs Outdated
Comment thread Havit.Blazor.E2ETests/TestAppAssemblyInitializer.cs Outdated
Comment thread Havit.Blazor.slnx
Comment on lines 2 to +5
<Configurations>
<BuildType Name="ACCESSIBLITYTESTS" />
<BuildType Name="Debug" />
<BuildType Name="Release" />
Comment thread Havit.Blazor.E2ETests/Havit.Blazor.E2ETests.csproj
Comment thread CONTRIBUTING.md
@hakenr

hakenr commented May 22, 2026

Copy link
Copy Markdown
Member

@efinder2 Thank you for your contribution. Please take a look at the Copilot code review findings. Just let me know whether you are willing to polish the PR on your own or if we should take over.

@efinder2 efinder2 marked this pull request as draft May 29, 2026 19:10
@efinder2 efinder2 marked this pull request as ready for review May 30, 2026 18:05
@efinder2

Copy link
Copy Markdown
Contributor Author

I solved the issues brought by copilot and found, that i don't need to add the configuration to every csproj. How do you prefer the usage of this tool?

I would exclude the existing problems from the execution. The test will then pass but new accessiblity problems would cause them to fail. I'd suggest to create issues in github for the ignored violations. a lot of them may be grouped in a shared issue because they have the same root problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants