Added automated accessiblity tests#1396
Conversation
|
@copilot resolve the merge conflicts in this pull request |
There was a problem hiding this comment.
Pull request overview
This PR introduces a proof-of-concept for running automated accessibility (axe) checks as part of the existing Playwright-based E2E test suite, gated behind a dedicated build configuration.
Changes:
- Adds an
ACCESSIBLITYTESTSbuild configuration across the solution/projects to toggle accessibility checks. - Integrates
Playwright.AxeintoHavit.Blazor.E2ETestsand runs axe scans fromTestCleanup, aggregating results into a Markdown report. - Updates centralized package versions and contributing docs related to running the tests.
Reviewed changes
Copilot reviewed 35 out of 35 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| Havit.SourceGenerators.StrongApiStringLocalizers/Havit.SourceGenerators.StrongApiStringLocalizers.csproj | Adds custom configuration/platform entries. |
| Havit.SourceGenerators.StrongApiStringLocalizers.Tests/Havit.SourceGenerators.StrongApiStringLocalizers.Tests.csproj | Adds custom configuration/platform entries. |
| Havit.Extensions.Localization/Havit.Extensions.Localization.csproj | Adds custom configuration/platform entries. |
| Havit.Blazor.TestApp/Havit.Blazor.TestApp/Havit.Blazor.TestApp.csproj | Adds custom configuration/platform entries. |
| Havit.Blazor.TestApp/Havit.Blazor.TestApp.Client/Havit.Blazor.TestApp.Client.csproj | Adds custom configuration/platform entries. |
| Havit.Blazor.slnx | Adds solution build configuration for accessibility tests. |
| Havit.Blazor.Grpc.TestContracts/Havit.Blazor.Grpc.TestContracts.csproj | Adds custom configuration/platform entries. |
| Havit.Blazor.Grpc.Server/Havit.Blazor.Grpc.Server.csproj | Adds custom configuration/platform entries. |
| Havit.Blazor.Grpc.Server.Tests/Havit.Blazor.Grpc.Server.Tests.csproj | Adds custom configuration/platform entries. |
| Havit.Blazor.Grpc.Core/Havit.Blazor.Grpc.Core.csproj | Adds custom configuration/platform entries. |
| Havit.Blazor.Grpc.Core.Tests/Havit.Blazor.Grpc.Core.Tests.csproj | Adds custom configuration/platform entries. |
| Havit.Blazor.Grpc.Client/Havit.Blazor.Grpc.Client.csproj | Adds custom configuration/platform entries. |
| Havit.Blazor.Grpc.Client.WebAssembly/Havit.Blazor.Grpc.Client.WebAssembly.csproj | Adds custom configuration/platform entries. |
| Havit.Blazor.Grpc.Client.Tests/Havit.Blazor.Grpc.Client.Tests.csproj | Adds custom configuration/platform entries. |
| Havit.Blazor.Grpc.Client.ServerSideRendering/Havit.Blazor.Grpc.Client.ServerSideRendering.csproj | Adds custom configuration/platform entries. |
| Havit.Blazor.GoogleTagManager/Havit.Blazor.GoogleTagManager.csproj | Adds custom configuration/platform entries. |
| Havit.Blazor.E2ETests/TestAppTestBase.cs | Adds axe execution in test cleanup and report aggregation. |
| Havit.Blazor.E2ETests/TestAppAssemblyInitializer.cs | Writes aggregated axe results during assembly cleanup. |
| Havit.Blazor.E2ETests/Havit.Blazor.E2ETests.csproj | Adds configuration/platform entries and references Playwright.Axe. |
| Havit.Blazor.Documentation/Havit.Blazor.Documentation.csproj | Adds custom configuration/platform entries. |
| Havit.Blazor.Documentation.Tests/Havit.Blazor.Documentation.Tests.csproj | Adds custom configuration/platform entries. |
| Havit.Blazor.Documentation.Server/Havit.Blazor.Documentation.Server.csproj | Adds custom configuration/platform entries. |
| Havit.Blazor.Documentation.RepoDumpGenerator/Havit.Blazor.Documentation.RepoDumpGenerator.csproj | Adds custom configuration/platform entries. |
| Havit.Blazor.Documentation.Mcp/Havit.Blazor.Documentation.Mcp.csproj | Adds custom configuration/platform entries. |
| Havit.Blazor.Components.Web/Havit.Blazor.Components.Web.csproj | Adds custom configuration/platform entries. |
| Havit.Blazor.Components.Web.Tests/Havit.Blazor.Components.Web.Tests.csproj | Adds custom configuration/platform entries. |
| Havit.Blazor.Components.Web.ECharts/Havit.Blazor.Components.Web.ECharts.csproj | Adds custom configuration/platform entries. |
| Havit.Blazor.Components.Web.Bootstrap/Havit.Blazor.Components.Web.Bootstrap.csproj | Adds custom configuration/platform entries. |
| Havit.Blazor.Components.Web.Bootstrap.Tests/Havit.Blazor.Components.Web.Bootstrap.Tests.csproj | Adds custom configuration/platform entries. |
| Havit.Blazor.Components.Web.Bootstrap.Smart/Havit.Blazor.Components.Web.Bootstrap.Smart.csproj | Adds custom configuration/platform entries. |
| Havit.Blazor.Components.Web.Bootstrap.InternalSourceGenerators/Havit.Blazor.Components.Web.Bootstrap.InternalSourceGenerators.csproj | Adds custom configuration/platform entries. |
| Havit.Blazor.Components.Web.Bootstrap.EntityFrameworkCore.Tests/Havit.Blazor.Components.Web.Bootstrap.EntityFrameworkCore.Tests.csproj | Adds custom configuration/platform entries. |
| Directory.Packages.props | Adds central package versions for axe/playwright integration. |
| CONTRIBUTING.md | Adds initial guidance for running accessibility tests (browser install). |
| BlazorAppTest/BlazorAppTest.csproj | Adds custom configuration/platform entries. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| <Configurations> | ||
| <BuildType Name="ACCESSIBLITYTESTS" /> | ||
| <BuildType Name="Debug" /> | ||
| <BuildType Name="Release" /> |
|
@efinder2 Thank you for your contribution. Please take a look at the Copilot code review findings. Just let me know whether you are willing to polish the PR on your own or if we should take over. |
|
I solved the issues brought by copilot and found, that i don't need to add the configuration to every csproj. How do you prefer the usage of this tool? I would exclude the existing problems from the execution. The test will then pass but new accessiblity problems would cause them to fail. I'd suggest to create issues in github for the ignored violations. a lot of them may be grouped in a shared issue because they have the same root problem. |
I've added the execution of automated accessibility tests via axe as discussed in #1192. This is at the moment only proof of concept. I'm not sure where to run them correctly. Currently they are run in the clean up method. It was an easy way to execute it after every test, so i don't have to copy every test. This may not be the final way. The current implementation causes problems with the termination of the chrome headless shell. They are orphans after the execution. It freezes my Laptop with 16GB of RAM. With 32 on my Desktop the tests run through and are killed after the execution by the IDE.
I added a list of the found problems.
axe-results-20260504-194743.md
There are multiple ways to implement them further. I don't recommend to add them directly to the pipeline. There are too many findings to fix them at once. So there needs to be a switch to enable them. In the current implementation this is solved by a compile directive.
The easiest way would be to copy the tests, add a RunAxe at the end and filter by a TestCategory. An alternative would be to call RunAxe after each test (and in relevant steps in between). This method filters by environment variables if the accessibility tests are activated.
Alternatively we run them always. For now we could exclude all rules with warnings or suppress the rule for some css selectors if they aren't many of this category.