Skip to content

fix(e2e): robust toggle-updates probe, storage service lifecycle, and Firefox window lookup - #792

Open
castrojo wants to merge 11 commits into
mainfrom
fix/e2e-gate-failures
Open

fix(e2e): robust toggle-updates probe, storage service lifecycle, and Firefox window lookup#792
castrojo wants to merge 11 commits into
mainfrom
fix/e2e-gate-failures

Conversation

@castrojo

@castrojo castrojo commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

This fixes multiple root causes that resulted in red E2E gates across downstream and upstream test runs:

  1. _has_toggle_action probe in tests/common/features/environment.py:
    Previously, _has_toggle_action probed with timeout 15 ujust toggle-updates cancel. In the older unpatched update.just recipe, running without a TTY caused gum choose to fail and set SELECTED_OPTION="", which hit [[ "${SELECTED_OPTION}" == "Cancel" || "${SELECTED_OPTION}" == "" ]] && exit 0. Thus the probe exited 0 even on unpatched images (such as bluefin:testing), falsely activating the non-interactive toggle scenario that subsequently failed.
    The probe now inspects the recipe body via ujust --show toggle-updates 2>/dev/null | grep -q 'ACTION_VALUE'.

  2. bootc-unified-storage.service lifecycle:
    In tests/common/features/common_services.feature, bootc unified storage service completed successfully asserts systemctl show bootc-unified-storage.service --property=Result --value is success. If the service was interrupted or stopped with a signal during earlier runner setup or boot, before_scenario restarts it so it runs to completion before the check runs.

  3. Firefox window resolution in tests/smoke/features/steps/firefox_steps.py:
    In GNOME 50, _window_candidates used recursive findChildren over the entire AT-SPI tree of Firefox, which discovered iframe/webview sub-frames. When an iframe with push button was found, _firefox_window preferred it as a frame over the top-level filler window that contained the actual browser chrome (address bar and tabs), causing Address bar is present in Firefox to fail. _window_candidates now checks immediate application children first and prioritizes windows exposing browser chrome (entry, autocomplete, page tab list).

Validation

  • pytest tests/unit/test_common_steps.py tests/unit/test_firefox_steps.py passes with all 74 unit tests green.
  • python3 -m py_compile clean across all touched files.

Signed-off-by: Jorge O. Castro jorge@jorge.fyi
Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

@castrojo
fix(e2e): robust toggle-updates probe, storage service lifecycle, and…
0799b2f 
… Firefox window lookup

1. Probe ujust toggle-updates by checking recipe body for ACTION_VALUE rather
   than invoking 'ujust toggle-updates cancel' (which exited 0 without TTY even
   on unpatched images due to gum choose failure fallback).
2. Ensure bootc-unified-storage.service completes with Result=success before
   the common services scenario asserts its state.
3. In smoke suite, prioritize browser windows with address bar/tab chrome and
   query top-level application children in AT-SPI to prevent picking webview
   sub-frames.

Signed-off-by: Jorge O. Castro <jorge@jorge.fyi>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@codecov-commenter

codecov-commenter commented Sep 7, 2026 •
edited
Loading

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 87.08551% with 74 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
tests/smoke/features/steps/firefox_steps.py 75.15% 41 Missing ⚠️
tests/smoke/features/environment.py 48.33% 31 Missing ⚠️
tests/unit/test_common_steps.py 98.07% 1 Missing ⚠️
tests/unit/test_installer_environment.py 93.75% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@mrbobbytables mrbobbytables mentioned this pull request Sep 7, 2026
Merged
castrojo added a commit to projectbluefin/bluefin that referenced this pull request Sep 8, 2026
@mrbobbytables @castrojo
ci: consumer validation for testsuite#792 (issue #989) (#1215)
edc6871 
* docs: document testsuite#792 dependency for issue #989

Issue #989 blocks the :testing→main promotion. The root cause is Firefox
AT-SPI test failures in run-e2e / GNOME 50 — smoke-a, which fails because
Firefox window resolution incorrectly selects webview sub-frames instead of
the main window with browser chrome.

This is a consumer validation PR waiting for projectbluefin/testsuite#792,
which adds robust window resolution logic that prioritizes top-level windows
and validates presence of browser chrome (entry, autocomplete, tab list).

Once testsuite#792 is merged:
- The v1 tag auto-updates (via testsuite's update-v1-tag.yml)
- Bluefin automatically picks up the fix (run-testsuite.yml uses @v1)
- Firefox tests pass and promote-to-testing gate unblocks

Refs: #989, #400 (related GNOME 50 AT-SPI issues)
Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com>

* fix(ci): remove non-workflow note file from .github/workflows

---------

Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com>
Co-authored-by: mrbobbytables <mrbobbytables@users.noreply.github.com>
Co-authored-by: Jorge O. Castro <jorge.castro@gmail.com>
castrojo and others added 10 commits September 8, 2026 12:27
@castrojo
fix(e2e): support GNOME 50 autocomplete address bar and update docs
06fb0bc 
- Include autocomplete role in FIREFOX_CHROME_ROLES for GNOME 50 compatibility.
- Add focused unit tests for address bar autocomplete role, window candidates, tab count, and fallback handling.
- Document GNOME 50 Firefox AT-SPI window resolution in gnome/SKILL.md.
- Update ujust-noninteractive documentation for toggle-updates probe.

Assisted-by: Claude Sonnet via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@castrojo
fix(installer): isolate ssh_steps probe patching under parallel xdist
4654353 
- In _patch_probe, allow setting run_ssh on stubbed ssh_steps modules
  using raising=False so workers running after tests that injected
  partial ssh_steps stubs do not raise AttributeError.
- In env_module fixture, evict any bare non-package tests.shared stubs
  left by prior tests to ensure submodule imports resolve cleanly.
- Add focused regression test covering probe execution when ssh_steps
  stub lacks the run_ssh attribute.
- Document parallel xdist helper monkeypatching in unit-test-module-stubs.md.

Assisted-by: Claude Sonnet via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@castrojo
fix(e2e): robust Firefox window resolution, address bar combo box, an…
90ee80f 
…d tab management fallback

- Filter out Mozilla crash reporter windows in _firefox_window to avoid selecting stale crash dialogs over genuine browser windows.
- Support GNOME 50 address bar combo box role and button roles in FIREFOX_CHROME_ROLES and _address_bar.
- Prioritize capitalized "Firefox" in FIREFOX_APP_NAMES to eliminate 10s lookup timeout.
- Provide resilient AT-SPI button fallback for tab open/close in headless Wayland container environments where uinput events are not routed to windows.
- Add unit tests for crash reporter filtering, combo box recognition, and tab open/close fallbacks.
- Update GNOME testing skill documentation.

Assisted-by: Claude Sonnet via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@castrojo
fix(e2e): cache Firefox application handle with retry and add navigat…
9f6e8ef 
…ion fallback

- Cache context.firefox_app with a bounded retry loop to guard against transient AT-SPI bus lookup drops.
- Fallback to set_text_contents on address bar when uinput cannot type URL characters.
- Add unit tests for app handle caching, transient retry, and URL navigation fallback.

Assisted-by: Claude Sonnet via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@castrojo
fix(e2e): robust character input mapping, URL navigation fallback, an…
d25bbec 
…d clean close

- Patch keyboard_character_input in smoke environment to map punctuation, shifted keys, and uppercase to evdev events, avoiding NoneType unpack crashes.
- Add remote IPC navigation fallback and multi-target assertion in navigate_firefox_to for headless Wayland.
- Add fallback termination in firefox_is_no_longer_running when uinput Ctrl+Q is not delivered by compositor.
- Add unit tests for character mapping, URL navigation variants, and shutdown fallback.
- Update GNOME testing skill documentation.

Assisted-by: Claude Sonnet via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@castrojo
fix(e2e): robust Firefox URL navigation and address polling
84f47c2 
- Invoke firefox CLI directly via Popen/ssh_launch with arguments rather than passing command string into launch_background.
- Poll for loaded page URL across address bar, document web, and tab titles for up to 7.5s.
- Update unit tests with subprocess.Popen mock.

Assisted-by: Claude Sonnet via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@castrojo
fix(e2e): prioritize newest window and search all candidates during U…
ddd81f8 
…RL navigation

- Search window candidates in reverse (newest first) in _firefox_window to match newly spawned browser windows.
- Check all non-crash window candidates across address bar text, document web, and tab titles in navigate_firefox_to.

Assisted-by: Claude Sonnet via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@castrojo
fix(e2e): reuse established Firefox window and poll address bar acros…
5229fa9 
…s navigation

- Reuse context.firefox_window within scenario to maintain window consistency across steps.
- Add bounded polling loop in _address_bar to eliminate navigation transition timing races.

Assisted-by: Claude Sonnet via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@castrojo
fix(e2e): patch keyboard_key_input with Return/Enter alias mapping
e88fdcd 
- Patch keyboard_key_input in smoke environment to map 'Return' to 'ENTER' in uinput evdev codes.
- Use evdev native 'Enter' key name in navigate_firefox_to step.
- Add unit test for key name alias mapping.

Assisted-by: Claude Sonnet via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@castrojo
fix(e2e): robust smoke session environment, extension list D-Bus fall…
1cdf1af 
…back, and core app discovery

- Automatically provision /tmp/session.env with active session variables in smoke environment to prevent POSIX /bin/sh source aborts in container mode.
- Execute _run_host commands under bash with session environment preservation across steps, extensions, and system health.
- Fall back to org.gnome.Shell.Extensions.ListExtensions via D-Bus when gnome-extensions CLI hangs on portal activation.
- Use tree.root.applications() in _settings_app and _nautilus_app to prevent dogtail 10s blocking search freezes.
- Allow Ptyxis terminal assertion to accept shell prompt window titles (user@host:~).
- Add unit tests for session env execution, app list resolution, Ptyxis window title, and extension D-Bus fallbacks.
- Update GNOME testing skill documentation.

Assisted-by: Claude Sonnet via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@castrojo @codecov-commenter