Skip to content

smoke: add AT-SPI and crash-regression coverage for new GNOME extensions - #798

Open
mrbobbytables wants to merge 2 commits into
projectbluefin:mainfrom
mrbobbytables:atspi-extensions-smoke-793
Open

smoke: add AT-SPI and crash-regression coverage for new GNOME extensions#798
mrbobbytables wants to merge 2 commits into
projectbluefin:mainfrom
mrbobbytables:atspi-extensions-smoke-793

Conversation

@mrbobbytables

Copy link
Copy Markdown
Contributor

Summary

Closes #793. Bluefin/Dakota is enabling seven new curated GNOME Shell extensions by default (projectbluefin/common#1087). This adds AT-SPI smoke and crash-regression coverage for them, complementing the presence-only checks already in tests/smoke/features/bluefin_extensions.feature.

Note on base branch: the issue asked for this to be based on/opened against v4, but that branch does not exist in projectbluefin/testsuite (only main). This PR is based on and opened against main instead.

What's added

  • tests/smoke/features/bluefin_new_extensions.feature — the exact scenario set from the issue's acceptance criteria, covering:
    • copyous (highest priority): AT-SPI clipboard history popover interaction, rapid clipboard-change stress with plain/multiline/Unicode/large payloads, and post-stress GNOME Shell accessibility + bounded-memory assertions
    • syncthing-toggle: Quick Settings toggle/pill/header label ("Sync Folder"), service start/stop, start-stop-only gsettings check
    • Bluetooth-Battery-Meter: panel icon presence, symbolic level-indicator-color setting, clean behavior with no paired devices
    • quicksettings-audio-devices-hider/renamer: Quick Settings audio menu opens cleanly for each
    • tiling-assistant: window-snapping keyboard shortcuts, gesture soft-pass (gestures can't be synthesized in headless CI), overall Shell responsiveness
    • tailscale-gnome-qs: Quick Settings item presence, daemon running/stopped/unavailable state handling
  • tests/smoke/features/steps/bluefin_new_extensions_steps.py — step definitions built on the existing SSH/Shell.Eval/journalctl helper conventions from steps.py and gnome_extensions_steps.py.
  • tests/unit/test_bluefin_new_extensions_steps.py — unit coverage for the new pure/host-command-construction helpers (37 tests).

Design notes

Several of these extensions are third-party/community-maintained, so their gsettings schema ids and exact AT-SPI role names can't be reliably guessed ahead of running against a real image. Steps that depend on such unverifiable internals degrade to a printed WARNING + soft pass (matching the existing pattern in gnome_extensions_steps.py for headless AT-SPI limitations), while the core crash-regression assertions always run and fail hard:

  • GNOME Shell stays accessible via AT-SPI
  • No new gnome-shell journal errors at err..emerg since the scenario started
  • No gnome-shell coredump
  • Clipboard stress keeps gnome-shell RSS within a generous bound (no runaway/leak-shaped growth)

This preserves the crash-regression value described in the issue (especially for copyous, which is called out as having crashed GNOME Shell in practice) without hard-failing on implementation details we can't verify from this environment.

Testing

  • ruff check tests/ --select E,F,W --ignore E501 — clean
  • python3 -m pytest tests/unit/ -q — 1564 passed (including the 37 new tests)
  • PYTHONPATH=. python3 -m behave --dry-run --no-summary tests/smoke/features/ — new feature's steps all resolve (no undefined/ambiguous steps introduced; confirmed no new undefined-step count vs. base branch)
  • python3 scripts/update_coverage_snapshot.py — snapshot doc regenerated for the new feature/suite mapping

— hive: backend=copilot model=claude-sonnet-5

🐝 Hive Agent: contributor | SHA: 4f7c0616

mrbobbytables and others added 2 commits September 7, 2026 22:01
Adds tests/smoke/features/bluefin_new_extensions.feature covering the
seven curated GNOME Shell extensions newly enabled by default in
Bluefin/Dakota (projectbluefin/common#1087), complementing the
presence-only checks in bluefin_extensions.feature:

- copyous: AT-SPI clipboard history popover interaction, rapid
  clipboard-change stress with plain/multiline/Unicode/large payloads,
  and post-stress GNOME Shell accessibility + bounded-memory assertions
- syncthing-toggle: Quick Settings toggle/pill/header label, service
  start/stop, and start-stop-only gsettings check
- Bluetooth-Battery-Meter: panel icon presence, symbolic indicator
  color setting, and clean behavior with no paired devices
- quicksettings-audio-devices-hider/renamer: Quick Settings audio menu
  opens cleanly for each extension
- tiling-assistant: window-snapping keyboard shortcuts, gesture
  soft-pass (gestures cannot be synthesized in headless CI), and
  overall Shell responsiveness
- tailscale-gnome-qs: Quick Settings item presence and daemon
  running/stopped/unavailable state handling

All new steps build on the existing SSH/Shell.Eval/journalctl helper
conventions in steps.py and gnome_extensions_steps.py. Checks that
depend on unverifiable third-party extension internals (gsettings
schema ids, exact AT-SPI role names) degrade to a printed WARNING and
soft pass, while the core stability assertions (GNOME Shell stays
accessible, no new gnome-shell journal errors, no coredump) always run
and fail hard — preserving the crash-regression value described in the
issue even where extension-internal implementation details can't be
guessed reliably.

Adds tests/unit/test_bluefin_new_extensions_steps.py with unit coverage
for the new pure/host-command-construction helpers.

Fixes projectbluefin#793

Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com>
Stabilize installer unit tests when other tests leave a partial ssh_steps stub in sys.modules, while keeping production probe import explicit.

Assisted-by: Claude Sonnet 5 via GitHub Copilot CLI

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com>
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 72.76190% with 143 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...oke/features/steps/bluefin_new_extensions_steps.py 52.17% 143 Missing ⚠️

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[qa] Add AT-SPI smoke and crash-regression coverage for newly enabled GNOME extensions

2 participants