Open
Conversation
Contributor
📝 WalkthroughSummary by CodeRabbit
WalkthroughAdds many Playwright E2E tests, page objects, data factories, test fixtures, and test IDs across governance, model-limits, MCP, observability, providers, virtual-keys, routing, plugins, and config; updates Playwright discovery and CI runner script. (50 words) Changes
Sequence Diagram(s)sequenceDiagram
participant Test as E2E Test Suite
participant Page as GovernancePage
participant UI as Browser/DOM
participant Toast as Success Toast
Test->>Page: gotoTeams()
Page->>UI: navigate to /workspace/governance/teams
UI-->>Page: view ready
Test->>Page: createTeam(config)
Page->>UI: open sheet, fill fields, save
UI->>Toast: success toast
Toast-->>Page: observed
Page-->>Test: team created
sequenceDiagram
participant Test as E2E Test
participant Page as ConfigSettingsPage
participant UI as Browser/DOM
participant Toast as Success Toast
Test->>Page: goto()
Page->>UI: navigate to config view
UI-->>Page: ready
Test->>Page: setPricingDatasheetUrl(url)
Page->>UI: fill pricingDatasheetUrlInput
Test->>Page: triggerForceSync()
Page->>UI: click pricingForceSyncBtn
UI->>Toast: success
Toast-->>Page: observed
Test->>Page: savePricingConfig()
Page->>UI: click pricingSaveBtn
UI->>Toast: success
Toast-->>Page: observed
Page-->>Test: pricing config saved
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
🧪 Test Suite AvailableThis PR can be tested by a repository admin. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 19
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
tests/e2e/features/observability/pages/observability.page.ts (1)
1-1:⚠️ Potential issue | 🟠 MajorRemove
expectimport from@playwright/testin this E2E page object.E2E test page objects must import
testandexpectfrom../../core/fixtures/base.fixture, not from@playwright/test. Keep Playwright type imports only.Suggested fix
-import { Page, Locator, expect } from '@playwright/test' +import type { Page, Locator } from '@playwright/test'🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/observability/pages/observability.page.ts` at line 1, Remove the runtime import of expect from '@playwright/test' and instead import test and expect from the shared fixture; change the top-level imports so Playwright symbols used only for typing come in as type imports (e.g., Page, Locator) from '@playwright/test' and add an import { test, expect } from '../../core/fixtures/base.fixture' so the page object uses the project's fixture expect/test rather than Playwright's runtime expect.
🧹 Nitpick comments (9)
tests/e2e/features/placeholders/placeholders.spec.ts (1)
3-69: Consider consolidating this suite behind a placeholder page object + table-driven cases.The repeated
goto + wait + assertpattern is high-maintenance. Since this PR stack already expands shared E2E infrastructure, moving these selectors/flows into aBasePage-derived page object withgetByTestId()-based helpers would improve consistency and reduce brittleness.♻️ Example refactor shape
+const placeholderRoutes = [ + { path: '/workspace/guardrails', testId: 'guardrails-placeholder' }, + { path: '/workspace/audit-logs', testId: 'audit-logs-placeholder' }, + { path: '/workspace/cluster', testId: 'cluster-placeholder' }, +] + +for (const { path, testId } of placeholderRoutes) { + test(`should load ${path}`, async ({ page }) => { + await page.goto(path) + await expect(page).toHaveURL(path) + await expect(page.getByTestId(testId)).toBeVisible() + }) +}As per coding guidelines
tests/e2e/**/*.ts: “E2E test page objects must extend BasePage, use getByTestId() as the primary selector strategy, and import test/expect from ../../core/fixtures/base.fixture, never from@playwright/test.”🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/placeholders/placeholders.spec.ts` around lines 3 - 69, Create a PlaceholderPage that extends BasePage and move repeated navigation/assertion helpers (gotoAndWait, assertBodyVisible, assertTextAndButton) into it; refactor the suite to a table-driven test that iterates over an array of cases (route, expectedText?, expectButton?) and calls PlaceholderPage methods instead of repeating goto + wait + assert, and ensure selectors use getByTestId() in those page methods; also replace imports of test/expect from `@playwright/test` with the core fixtures import from ../../core/fixtures/base.fixture to comply with the guideline.tests/e2e/features/observability/pages/observability.page.ts (1)
143-144: Wait for toggle state transition before returning fromtoggleConnector.Returning immediately after click can race subsequent
isConnectorEnabledreads.Suggested refactor
async toggleConnector(connector: ObservabilityConnector): Promise<boolean> { const toggle = this.getConnectorToggle(connector) const isVisible = await toggle.isVisible().catch(() => false) if (!isVisible) return false const isDisabled = await toggle.isDisabled() if (isDisabled) return false + const previous = await toggle.getAttribute('data-state') await toggle.click() + const expected = previous === 'checked' ? 'unchecked' : 'checked' + await this.waitForStateChange(toggle, 'data-state', expected) return true }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/observability/pages/observability.page.ts` around lines 143 - 144, The toggleConnector function returns immediately after await toggle.click(), causing race conditions with subsequent isConnectorEnabled checks; change toggleConnector (the method containing await toggle.click()) to wait for the toggle's visual/state transition before returning—e.g., poll or use a waitFor/waitUntil on the toggle element's attribute/class or call isConnectorEnabled in a short retry loop with a timeout until the expected enabled/disabled state is observed; ensure the wait uses the same criterion as isConnectorEnabled so callers observe the final state.tests/e2e/features/dashboard/dashboard.spec.ts (1)
214-219: Assert the expectedvolume_chartvalue, not just parameter presence.This test can pass even if
volume_chartis rewritten to the wrong value.🎯 Suggested assertion
const url = dashboardPage.page.url() expect(url).toContain('period=7d') - expect(url).toMatch(/volume_chart=/) + const params = new URL(url).searchParams + expect(params.get('volume_chart')).toBe('line')🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/dashboard/dashboard.spec.ts` around lines 214 - 219, The test currently only checks for presence of the volume_chart parameter (using dashboardPage.page.url()), which can mask incorrect values; update the assertion to verify the exact expected value (e.g., assert that the URL contains "volume_chart=line" or parse the query and assert the volume_chart param equals "line") so the test fails if the parameter is rewritten to a wrong value.ui/app/workspace/model-limits/views/modelLimitsTable.tsx (1)
154-154: Normalize dynamicdata-testidsuffixes before interpolation.Using raw
model_name/providerin test IDs can make selectors brittle when values include spaces or symbols. Slugging the dynamic part will make tests more resilient.♻️ Suggested hardening
+const toTestIdPart = (value: string) => + value.toLowerCase().replace(/[^a-z0-9]+/g, "-").replace(/(^-|-$)/g, ""); ... -<TableRow key={config.id} data-testid={`model-limit-row-${config.model_name}-${config.provider || "all"}`} className={cn("group transition-colors", isExhausted && "bg-red-500/5 hover:bg-red-500/10")}> +<TableRow + key={config.id} + data-testid={`model-limit-row-${toTestIdPart(config.model_name)}-${toTestIdPart(config.provider || "all")}`} + className={cn("group transition-colors", isExhausted && "bg-red-500/5 hover:bg-red-500/10")} +> ... - data-testid={`model-limit-button-edit-${config.model_name}-${config.provider || "all"}`} + data-testid={`model-limit-button-edit-${toTestIdPart(config.model_name)}-${toTestIdPart(config.provider || "all")}`} ... - data-testid={`model-limit-button-delete-${config.model_name}-${config.provider || "all"}`} + data-testid={`model-limit-button-delete-${toTestIdPart(config.model_name)}-${toTestIdPart(config.provider || "all")}`}As per coding guidelines,
ui/app/**/*.{tsx,ts}: E2E test data-testid attributes must follow the convention 'data-testid="<entity>-<element>-<qualifier>"'.Also applies to: 303-303, 316-316
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/model-limits/views/modelLimitsTable.tsx` at line 154, The data-testid built in the TableRow uses raw config.model_name and config.provider which can include spaces/symbols and break E2E selectors; normalize those values (e.g., with a slugify or normalizeTestId helper) before interpolation and use the normalized strings in the data-testid template for the TableRow (where data-testid={`model-limit-row-${config.model_name}-${config.provider || "all"}`}) and the other similar test-id usages referenced around the same component (the other occurrences using config.model_name/config.provider), ensuring the test id follows the "<entity>-<element>-<qualifier>" convention.tests/e2e/features/model-limits/model-limits.data.ts (1)
3-10: Make default factory output collision-safe for create flows.Current defaults are static, so repeated calls can produce identical model-limit payloads unless every callsite overrides values. Please make the generated defaults unique (or enforce unique override inputs).
🔧 Suggested refactor
export function createModelLimitData(overrides: Partial<ModelLimitConfig> = {}): ModelLimitConfig { const timestamp = Date.now() return { provider: 'openai', - modelName: 'gpt-4o-mini', + modelName: overrides.modelName ?? `gpt-4o-mini-${timestamp}`, budget: { maxLimit: 10, resetDuration: '1M' }, ...overrides, } }As per coding guidelines, "E2E test data factories must use Date.now() for unique names to prevent resource name collision in parallel test runs."
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/model-limits/model-limits.data.ts` around lines 3 - 10, The createModelLimitData factory returns static defaults causing collisions; update createModelLimitData (and its returned modelName/provider fields) to incorporate the timestamp variable (Date.now()) into a generated unique value (e.g., append or interpolate timestamp into modelName like `gpt-4o-mini-${timestamp}`) so each call produces a unique ModelLimitConfig by default while still allowing overrides via the overrides parameter.tests/e2e/features/providers/providers.spec.ts (1)
364-373: Make the Models assertion meaningful.
expect(typeof modelsVisible).toBe('boolean')is always true, so this test can pass without validating the UI behavior. Consider asserting one explicit expected state (or explicit skip condition) instead.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/providers/providers.spec.ts` around lines 364 - 373, Replace the no-op type assertion with an explicit expectation of the Models UI state: remove the line using expect(typeof modelsVisible).toBe('boolean') and instead assert the concrete expected value for modelsVisible (for example expect(modelsVisible).toBe(true)), or if the Models section is optional make the test skip/mark pending when it is not applicable; locate and change the assertion around modelsSection and modelsVisible (from providersPage.page.getByText(/Models/i)) and keep the existing branch that calls await expect(modelsSection).toBeVisible() when modelsVisible is true.tests/e2e/features/model-limits/model-limits.spec.ts (1)
31-44: Consider unique model names per test for isolation.All tests use the same hardcoded
modelName: 'gpt-4o-mini'andprovider: 'openai'. If any test's cleanup fails or a previous test leaves residual data, subsequent tests may fail or produce false positives. While serial execution (CI=true) helps, adding a timestamp suffix to model names would improve test isolation.However, if the model selection UI only allows choosing from predefined models (not arbitrary names), this approach is acceptable given the afterEach cleanup. Based on learnings, the CI runs with
workers: 1making module-level cleanup state safe from parallel execution issues.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/model-limits/model-limits.spec.ts` around lines 31 - 44, The test uses a hardcoded modelName ('gpt-4o-mini') which can cause cross-test interference; update the test that calls createModelLimitData to generate a unique model name (e.g., append a timestamp or UUID) and use that generated name in both the createdLimits.push and the call to modelLimitsPage.createModelLimit, keeping calls to modelLimitsPage.modelLimitExists in sync; if the UI restricts names to predefined values, add a comment in the test near createModelLimitData explaining why a fixed name is required instead of a dynamic one so future maintainers understand the constraint.tests/e2e/features/governance/governance.spec.ts (1)
26-37: Customer cleanup in Teams afterEach could be optimized.The cleanup navigates to the customers page for each customer individually (
await governancePage.gotoCustomers()inside the loop). Consider navigating once before the loop if there are multiple customers to clean up:♻️ Optional optimization
createdTeams.length = 0 + if (createdCustomers.length > 0) { + await governancePage.gotoCustomers() + } for (const name of [...createdCustomers]) { try { - await governancePage.gotoCustomers() const exists = await governancePage.customerExists(name)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/governance/governance.spec.ts` around lines 26 - 37, The loop is calling governancePage.gotoCustomers() for every customer; move the navigation outside the loop to call governancePage.gotoCustomers() once (before iterating createdCustomers) and then inside the loop call governancePage.customerExists(name) and governancePage.deleteCustomer(name) per item; keep the try/catch around the per-customer existence/delete calls so failures for one name don't stop others and still clear createdCustomers.length = 0 at the end.tests/e2e/features/providers/pages/providers.page.ts (1)
211-216: Normalize extracted weight text before returning.Raw
textContent()may include whitespace/newlines and make assertions brittle.🔧 Suggested fix
async getKeyWeight(name: string): Promise<string> { const keyRow = this.getKeyRow(name) // Weight is in the second TableCell (index 1) const weightCell = keyRow.locator('td').nth(1) - return (await weightCell.textContent()) ?? '' + return ((await weightCell.textContent()) ?? '').trim() }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/providers/pages/providers.page.ts` around lines 211 - 216, The getKeyWeight method returns raw text from weightCell which can include extra whitespace/newlines; update getKeyWeight (and references keyRow and weightCell) to normalize the extracted string before returning — at minimum trim leading/trailing whitespace and collapse internal runs of whitespace/newlines into a single space so assertions are stable.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/e2e/features/config/config.spec.ts`:
- Around line 73-97: The test "should set and save datasheet URL" mutates
workspace config but doesn't restore it; capture the current datasheet URL
before calling setPricingDatasheetUrl (use
configSettingsPage.pricingDatasheetUrlInput or a getter), run the change/save,
then in a finally/after step restore the original value by calling
configSettingsPage.setPricingDatasheetUrl(original) and
configSettingsPage.savePricingConfig (and dismiss toasts) so state is returned
even if the test skipped or fails; ensure the restoration only runs when the
save button was enabled and reference the test name and methods
configSettingsPage.setPricingDatasheetUrl, configSettingsPage.savePricingConfig,
configSettingsPage.pricingDatasheetUrlInput (or getter) so the fix is easy to
locate.
- Around line 368-385: Guard the environment-dependent security controls by
checking the control’s presence before asserting or interacting: for tests
referencing configSettingsPage.enforceAuthOnInferenceSwitch and
configSettingsPage.requiredHeadersTextarea, first verify the element exists/ is
available (e.g., via a utility like
configSettingsPage.isElementVisibleOrExists(enforceAuthOnInferenceSwitch) or a
feature-flag check on the page object) and skip/return the test early if it’s
not present; only call configSettingsPage.getSwitchState,
configSettingsPage.toggleEnforceAuthOnInference,
configSettingsPage.hasPendingChanges, and configSettingsPage.saveSettings when
the switch is confirmed available.
In `@tests/e2e/features/governance/pages/governance.page.ts`:
- Around line 91-94: The numeric budget check currently skips legitimate zero
values because it uses a truthy check (if (config.budget?.maxLimit)), so change
that condition to an explicit undefined check (if (config.budget?.maxLimit !==
undefined)) for the '#budgetMaxLimit' flow and apply the same fix to the other
numeric budget checks referenced around lines 134-137; this ensures 0 is
accepted while still skipping when the property is absent.
- Around line 51-53: customerDialog is using a too-broad locator
(page.locator('[role="dialog"]')) and customerNameInput uses getByLabel — update
both to use a specific test-id strategy: add a test id (e.g.,
data-testid="customer-dialog") to the customer dialog element in the UI (or use
a more specific scoped locator combining role with a unique parent container if
you cannot change the UI), then change customerDialog to use
page.getByTestId('customer-dialog') and change customerNameInput to
page.getByTestId('customer-name') (or the actual test id you add); ensure the
selector is distinct from teamDialog to avoid cross-matching.
In `@tests/e2e/features/mcp-registry/pages/mcp-registry.page.ts`:
- Around line 601-609: Update getClientConnectionType to first query the
connection-type cell using a test-id locator on the client row (e.g., call
row.getByTestId('client-connection-type') or similar) and return its trimmed
textContent; if that locator returns nothing, fall back to the existing
cells.nth(1) approach for backward compatibility. Modify only
getClientConnectionType (which uses getClientRow and cells.nth(1)) so it prefers
the test-id selector as primary and retains the column-index lookup as a
fallback.
In `@tests/e2e/features/mcp-settings/pages/mcp-settings.page.ts`:
- Around line 16-17: The two Playwright locators maxAgentDepthInput and
toolTimeoutInput currently use page.locator('#mcp-agent-depth') and
page.locator('#mcp-tool-execution-timeout'); change them to use
page.getByTestId('mcp-agent-depth') and
page.getByTestId('mcp-tool-execution-timeout') as the primary selectors and keep
the original id locators as fallbacks (e.g., try getByTestId first, then locator
by id) so the page object uses getByTestId() primarily while preserving the
id-based fallback behavior.
In `@tests/e2e/features/model-limits/pages/model-limits.page.ts`:
- Around line 56-59: The current truthy check on config.budget?.maxLimit skips
valid numeric 0; change the guard so it tests for existence rather than
truthiness (e.g., check config.budget?.maxLimit !== undefined &&
config.budget?.maxLimit !== null or use Number.isFinite) before locating
'#modelBudgetMaxLimit' and calling
budgetInput.fill(String(config.budget.maxLimit)); this ensures 0 is applied
correctly.
- Line 1: The import currently brings in expect from '@playwright/test' which is
incorrect for page objects; remove expect from the import line (keep Locator and
Page) and obtain the fixture-exported expect instead — either by accepting an
injected expect parameter on methods/constructors of the ModelLimitsPage class
(or whatever page object uses Page/Locator) or by importing the shared fixture
module that exports expect; update usages in model-limits.page.ts to reference
that fixture-exported expect rather than the one from '@playwright/test'.
- Around line 48-55: The fixed sleep in the model selection flow is fragile: in
the test function that calls fillSelect(...) and then sets const modelInput =
this.sheet.getByPlaceholder(/Search for a model/i) replace the await
this.page.waitForTimeout(500) with an explicit wait for the options to render
(e.g. await this.page.waitForSelector('[role="option"]', { timeout: 5000 }) or
similar) before calling this.page.getByRole('option').filter({ hasText:
config.modelName }).first().click(); also consider adding stable identifiers
(data-testid) to the provider SelectTrigger and the model input in the UI
component (modelLimitSheet.tsx) so the page object can use getByTestId instead
of fragile role/text selectors.
In `@tests/e2e/features/observability/pages/observability.page.ts`:
- Around line 236-246: The loop over connectors is missing the 'newrelic'
connector so ObservabilityConnector/ObservabilityState's newRelicEnabled isn't
populated or cleaned up; add 'newrelic' to the connectors array used in the
checks (the one iterating with isConnectorAvailable, selectConnector,
isConnectorEnabled) so that when connector === 'newrelic' you set
state.newRelicEnabled, and also add 'newrelic' to the other connectors array
referenced at lines ~256-257 to ensure the cleanup loop handles New Relic as
well.
- Around line 259-267: The current empty catch is swallowing cleanup failures in
the block that calls selectConnector, isConnectorEnabled, isToggleEnabled,
toggleConnector, and saveConfiguration; update this block to surface failures
instead of ignoring them by catching errors, logging a clear message including
the connector identifier and the error, and rethrowing (or failing the test) so
downstream tests cannot proceed with leaked connector state — ensure the catch
preserves the original error (or wraps it) and that saveConfiguration errors are
not silently ignored.
- Around line 52-55: The getConnectorToggle fallback using
this.page.locator('button[role="switch"]').first() is nondeterministic; change
getConnectorToggle(connector: ObservabilityConnector) to throw or assert when
CONNECTOR_TOGGLE_TESTIDS[connector] is missing (so callers like
isConnectorEnabled and toggleConnector always receive a specific Locator),
update any call sites to only pass connectors that exist in
ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS, ensure the page object extends
BasePage, use this.getByTestId(testId) as the sole selector strategy, and verify
imports of test/expect come from ../../core/fixtures/base.fixture rather than
`@playwright/test`.
In `@tests/e2e/features/placeholders/placeholders.spec.ts`:
- Around line 16-50: Each test that currently asserts
expect(page.locator('body')).toBeVisible() (e.g., the tests named "should load
guardrails page", "should load audit-logs page", "should load cluster page",
"should load custom-pricing page", "should load rbac page", "should load scim
page") should instead assert a route-specific signal: either check the URL with
expect(page).toHaveURL('/workspace/guardrails') etc. immediately after goto, or
assert a stable page element like
expect(page.locator('[data-testid="guardrails-page"]')).toBeVisible() (use a
corresponding data-testid per route such as guardrails-page, audit-logs-page,
cluster-page, custom-pricing-page, rbac-page, scim-page); update each test to
use the appropriate URL or data-testid locator rather than 'body' to prevent
false positives.
- Around line 9-14: Update the "should load alert-channels page" test to assert
the CTA's destination instead of only visibility: locate the CTA via
page.getByRole('button', { name: /Read more/i }) (or page.getByRole('link', {
name: /Read more/i }) if it is an anchor), then either read its href (await
page.getByRole(...).getAttribute('href') and expect it to equal the expected
URL) or perform a click and await navigation (await page.getByRole(...).click();
await page.waitForURL('/expected-path');
expect(page).toHaveURL('/expected-path')). Apply the same change to the other
test block covering lines 52-57 so both verify actual navigation/target URL
rather than just visibility.
In `@tests/e2e/features/plugins/plugins.spec.ts`:
- Around line 341-351: The test branch for an invalid path currently only
cancels the sheet when pluginsPage.sheet.isVisible() is true, so add an explicit
assertion that an inline validation or server error message appears before
calling pluginsPage.cancelPlugin(); specifically, when sheetVisible is true
assert the presence of the expected error indicator (e.g., an error element
inside pluginsPage.sheet or a toast/label selector used for validation) rather
than just checking visibility, using the same selectors you use for
hasErrorToast to verify inline/server error messaging, then call
pluginsPage.cancelPlugin(); ensure you reference pluginsPage.sheet, the inline
error selector, and hasErrorToast in the assertion so the test fails if no error
message is present.
- Around line 240-252: The test currently treats "sheet stayed open" as
sufficient proof of an error and treats any toast-like element as an error;
tighten assertions so both branches verify an explicit duplicate-name error
signal: when pluginsPage.sheet.isVisible() is true, assert a validation/error
element exists inside pluginsPage.sheet (e.g., locate a role="alert" or a
`.text-destructive` inside pluginsPage.sheet) before calling
pluginsPage.cancelPlugin(), and when the sheet is closed assert that
errorElements contains the expected duplicate error (either by asserting count
of role="alert" or `.text-destructive` > 0 or by matching an expected duplicate
text string). Target the existing symbols pluginsPage.sheet,
pluginsPage.cancelPlugin, errorElements and hasError when adding these explicit
checks.
In `@tests/e2e/features/providers/providers.spec.ts`:
- Around line 199-214: Track the created provider in a cleanup list before
performing the delete flow so failures don't leak resources: when you call
createCustomProviderData() and await providersPage.createProvider(providerData)
inside the test 'should delete custom provider and update UI', push providerData
(or its name) into a shared createdProviders array, and ensure an afterEach hook
iterates that array and calls providersPage.deleteProvider(name) (or the
appropriate deletion helper) to remove any leftover providers; also clear the
array after cleanup so tests remain isolated.
- Around line 764-794: The test 'should save valid pricing overrides JSON'
modifies workspace-level pricing overrides but doesn't restore them; update the
test to read and store the current pricing overrides before calling
providersPage.setPricingOverrides(validOverrides), and then in a
finally/teardown block call providersPage.setPricingOverrides(originalOverrides)
followed by providersPage.savePricingOverrides() and
providersPage.dismissToasts() to ensure the original state is restored
regardless of test outcome; use the existing helpers
(providersPage.setPricingOverrides, providersPage.savePricingOverrides,
providersPage.dismissToasts) and wrap the restore logic so it always runs after
the test actions.
In `@tests/e2e/features/virtual-keys/pages/virtual-keys.page.ts`:
- Around line 145-151: Replace the brittle DOM selector in isKeyRevealed by
using a dedicated test-id locator: instead of row.locator('code').first() use
row.getByTestId('virtual-key-value') (or the agreed test id for the key cell) to
obtain the key element, then read its textContent the same way; update
getVirtualKeyRow / page markup to ensure the element has that test id if
missing. This keeps the helper consistent with BasePage/getByTestId selector
strategy and prevents breakage when row markup changes.
---
Outside diff comments:
In `@tests/e2e/features/observability/pages/observability.page.ts`:
- Line 1: Remove the runtime import of expect from '@playwright/test' and
instead import test and expect from the shared fixture; change the top-level
imports so Playwright symbols used only for typing come in as type imports
(e.g., Page, Locator) from '@playwright/test' and add an import { test, expect }
from '../../core/fixtures/base.fixture' so the page object uses the project's
fixture expect/test rather than Playwright's runtime expect.
---
Nitpick comments:
In `@tests/e2e/features/dashboard/dashboard.spec.ts`:
- Around line 214-219: The test currently only checks for presence of the
volume_chart parameter (using dashboardPage.page.url()), which can mask
incorrect values; update the assertion to verify the exact expected value (e.g.,
assert that the URL contains "volume_chart=line" or parse the query and assert
the volume_chart param equals "line") so the test fails if the parameter is
rewritten to a wrong value.
In `@tests/e2e/features/governance/governance.spec.ts`:
- Around line 26-37: The loop is calling governancePage.gotoCustomers() for
every customer; move the navigation outside the loop to call
governancePage.gotoCustomers() once (before iterating createdCustomers) and then
inside the loop call governancePage.customerExists(name) and
governancePage.deleteCustomer(name) per item; keep the try/catch around the
per-customer existence/delete calls so failures for one name don't stop others
and still clear createdCustomers.length = 0 at the end.
In `@tests/e2e/features/model-limits/model-limits.data.ts`:
- Around line 3-10: The createModelLimitData factory returns static defaults
causing collisions; update createModelLimitData (and its returned
modelName/provider fields) to incorporate the timestamp variable (Date.now())
into a generated unique value (e.g., append or interpolate timestamp into
modelName like `gpt-4o-mini-${timestamp}`) so each call produces a unique
ModelLimitConfig by default while still allowing overrides via the overrides
parameter.
In `@tests/e2e/features/model-limits/model-limits.spec.ts`:
- Around line 31-44: The test uses a hardcoded modelName ('gpt-4o-mini') which
can cause cross-test interference; update the test that calls
createModelLimitData to generate a unique model name (e.g., append a timestamp
or UUID) and use that generated name in both the createdLimits.push and the call
to modelLimitsPage.createModelLimit, keeping calls to
modelLimitsPage.modelLimitExists in sync; if the UI restricts names to
predefined values, add a comment in the test near createModelLimitData
explaining why a fixed name is required instead of a dynamic one so future
maintainers understand the constraint.
In `@tests/e2e/features/observability/pages/observability.page.ts`:
- Around line 143-144: The toggleConnector function returns immediately after
await toggle.click(), causing race conditions with subsequent isConnectorEnabled
checks; change toggleConnector (the method containing await toggle.click()) to
wait for the toggle's visual/state transition before returning—e.g., poll or use
a waitFor/waitUntil on the toggle element's attribute/class or call
isConnectorEnabled in a short retry loop with a timeout until the expected
enabled/disabled state is observed; ensure the wait uses the same criterion as
isConnectorEnabled so callers observe the final state.
In `@tests/e2e/features/placeholders/placeholders.spec.ts`:
- Around line 3-69: Create a PlaceholderPage that extends BasePage and move
repeated navigation/assertion helpers (gotoAndWait, assertBodyVisible,
assertTextAndButton) into it; refactor the suite to a table-driven test that
iterates over an array of cases (route, expectedText?, expectButton?) and calls
PlaceholderPage methods instead of repeating goto + wait + assert, and ensure
selectors use getByTestId() in those page methods; also replace imports of
test/expect from `@playwright/test` with the core fixtures import from
../../core/fixtures/base.fixture to comply with the guideline.
In `@tests/e2e/features/providers/pages/providers.page.ts`:
- Around line 211-216: The getKeyWeight method returns raw text from weightCell
which can include extra whitespace/newlines; update getKeyWeight (and references
keyRow and weightCell) to normalize the extracted string before returning — at
minimum trim leading/trailing whitespace and collapse internal runs of
whitespace/newlines into a single space so assertions are stable.
In `@tests/e2e/features/providers/providers.spec.ts`:
- Around line 364-373: Replace the no-op type assertion with an explicit
expectation of the Models UI state: remove the line using expect(typeof
modelsVisible).toBe('boolean') and instead assert the concrete expected value
for modelsVisible (for example expect(modelsVisible).toBe(true)), or if the
Models section is optional make the test skip/mark pending when it is not
applicable; locate and change the assertion around modelsSection and
modelsVisible (from providersPage.page.getByText(/Models/i)) and keep the
existing branch that calls await expect(modelsSection).toBeVisible() when
modelsVisible is true.
In `@ui/app/workspace/model-limits/views/modelLimitsTable.tsx`:
- Line 154: The data-testid built in the TableRow uses raw config.model_name and
config.provider which can include spaces/symbols and break E2E selectors;
normalize those values (e.g., with a slugify or normalizeTestId helper) before
interpolation and use the normalized strings in the data-testid template for the
TableRow (where
data-testid={`model-limit-row-${config.model_name}-${config.provider ||
"all"}`}) and the other similar test-id usages referenced around the same
component (the other occurrences using config.model_name/config.provider),
ensuring the test id follows the "<entity>-<element>-<qualifier>" convention.
ℹ️ Review info
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (41)
tests/e2e/core/fixtures/base.fixture.tstests/e2e/features/config/config.spec.tstests/e2e/features/config/pages/config-settings.page.tstests/e2e/features/dashboard/dashboard.spec.tstests/e2e/features/governance/governance.data.tstests/e2e/features/governance/governance.spec.tstests/e2e/features/governance/pages/governance.page.tstests/e2e/features/mcp-auth-config/mcp-auth-config.spec.tstests/e2e/features/mcp-auth-config/pages/mcp-auth-config.page.tstests/e2e/features/mcp-registry/mcp-registry.spec.tstests/e2e/features/mcp-registry/pages/mcp-registry.page.tstests/e2e/features/mcp-settings/mcp-settings.spec.tstests/e2e/features/mcp-settings/pages/mcp-settings.page.tstests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.tstests/e2e/features/mcp-tool-groups/pages/mcp-tool-groups.page.tstests/e2e/features/model-limits/model-limits.data.tstests/e2e/features/model-limits/model-limits.spec.tstests/e2e/features/model-limits/pages/model-limits.page.tstests/e2e/features/observability/observability.spec.tstests/e2e/features/observability/pages/observability.page.tstests/e2e/features/placeholders/placeholders.spec.tstests/e2e/features/plugins/plugins.spec.tstests/e2e/features/providers/pages/providers.page.tstests/e2e/features/providers/providers.spec.tstests/e2e/features/read-more-links/read-more-links.spec.tstests/e2e/features/routing-rules/pages/routing-rules.page.tstests/e2e/features/routing-rules/routing-rules.spec.tstests/e2e/features/virtual-keys/pages/virtual-keys.page.tstests/e2e/features/virtual-keys/virtual-keys.spec.tsui/app/workspace/config/views/clientSettingsView.tsxui/app/workspace/config/views/mcpView.tsxui/app/workspace/config/views/pricingConfigView.tsxui/app/workspace/config/views/securityView.tsxui/app/workspace/governance/views/teamDialog.tsxui/app/workspace/governance/views/teamsTable.tsxui/app/workspace/mcp-registry/views/mcpServersEmptyState.tsxui/app/workspace/model-limits/views/modelLimitSheet.tsxui/app/workspace/model-limits/views/modelLimitsEmptyState.tsxui/app/workspace/model-limits/views/modelLimitsTable.tsxui/app/workspace/plugins/fragments/pluginFormFragments.tsxui/app/workspace/providers/views/providersEmptyState.tsx
Radheshg04
force-pushed
the
02-27-feat_extend_e2e_ui_tests
branch
from
58dca48 to
c34b8a3
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (5)
tests/e2e/features/providers/providers.spec.ts (1)
768-807:⚠️ Potential issue | 🟠 MajorGuarantee pricing override restoration with
finally.If the test fails after saving overrides but before the restore block, workspace config leaks into later tests.
🔧 Suggested hardening
await providersPage.selectConfigTab('governance') const initialValue = await providersPage.pricingOverridesJsonInput.inputValue() - const validOverrides = `[ - { - "model_pattern": "gpt-4o*", - "match_type": "wildcard", - "request_types": ["chat_completion"], - "input_cost_per_token": 0.000005, - "output_cost_per_token": 0.000015 - } -]` - - await providersPage.setPricingOverrides(validOverrides) - - const isSaveEnabled = await providersPage.pricingOverridesSaveBtn.isDisabled().then((d) => !d) - if (!isSaveEnabled) { - test.skip(true, 'Save button disabled (RBAC or no changes detected)') - return - } - - await providersPage.savePricingOverrides() - await providersPage.dismissToasts() - - // Restore original value for test isolation - await providersPage.setPricingOverrides(initialValue) - const restoreEnabled = await providersPage.pricingOverridesSaveBtn.isDisabled().then((d) => !d) - if (restoreEnabled) { - await providersPage.savePricingOverrides() - await providersPage.dismissToasts() - } + try { + const validOverrides = `[ + { + "model_pattern": "gpt-4o*", + "match_type": "wildcard", + "request_types": ["chat_completion"], + "input_cost_per_token": 0.000005, + "output_cost_per_token": 0.000015 + } +]` + + await providersPage.setPricingOverrides(validOverrides) + const isSaveEnabled = await providersPage.pricingOverridesSaveBtn.isDisabled().then((d) => !d) + if (!isSaveEnabled) { + test.skip(true, 'Save button disabled (RBAC or no changes detected)') + return + } + + await providersPage.savePricingOverrides() + await providersPage.dismissToasts() + } finally { + await providersPage.setPricingOverrides(initialValue) + const restoreEnabled = await providersPage.pricingOverridesSaveBtn.isDisabled().then((d) => !d) + if (restoreEnabled) { + await providersPage.savePricingOverrides() + await providersPage.dismissToasts() + } + }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/providers/providers.spec.ts` around lines 768 - 807, The test can leak workspace config if it fails before the manual restore; wrap the main save flow in a try/finally so the original value is always restored: after reading initialValue from providersPage.pricingOverridesJsonInput.inputValue(), perform the setPricingOverrides(validOverrides), savePricingOverrides(), and dismissToasts() inside try, and in finally call providersPage.setPricingOverrides(initialValue) and then conditionally call providersPage.pricingOverridesSaveBtn.isDisabled().then(d => !d) followed by providersPage.savePricingOverrides() and providersPage.dismissToasts() if restore is enabled; reference providersPage.setPricingOverrides, providersPage.savePricingOverrides, providersPage.dismissToasts, and providersPage.pricingOverridesSaveBtn to locate where to add the try/finally.tests/e2e/features/observability/pages/observability.page.ts (2)
261-266:⚠️ Potential issue | 🟠 MajorDo not suppress
saveConfigurationerrors during cleanup.Line 265 hides persistence failures, so cleanup can silently fail and leak connector state into following tests.
🛠️ Suggested fix
- await this.saveConfiguration().catch(() => {}) + await this.saveConfiguration()🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/observability/pages/observability.page.ts` around lines 261 - 266, The cleanup currently swallows errors from saveConfiguration() causing silent failures; update the try block around selectConnector/ toggleConnector to await saveConfiguration() without a .catch(() => {}) so persistence errors surface (or rethrow them), and if necessary wrap the cleanup in a try/finally or propagate the error from the method containing selectConnector, isConnectorEnabled, isToggleEnabled, toggleConnector and saveConfiguration so test failures are reported instead of being suppressed.
52-55:⚠️ Potential issue | 🟠 MajorRemove the generic
first()toggle fallback for unmapped connectors.Line 54 can bind to the wrong switch when multiple toggles are present, making
isConnectorEnabledandtoggleConnectornondeterministic.🔧 Suggested fix
getConnectorToggle(connector: ObservabilityConnector): Locator { const testId = ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS[connector] - return testId ? this.page.getByTestId(testId) : this.page.locator('button[role="switch"]').first() + if (!testId) { + throw new Error(`Missing toggle test id mapping for connector: ${connector}`) + } + return this.page.getByTestId(testId) }As per coding guidelines "
tests/e2e/**/*.ts: E2E test page objects must extend BasePage, use getByTestId() as the primary selector strategy, and import test/expect from ../../core/fixtures/base.fixture, never from@playwright/test."🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/observability/pages/observability.page.ts` around lines 52 - 55, The getConnectorToggle function currently falls back to page.locator('button[role="switch"]').first(), which can bind the wrong switch; remove that generic first() fallback and instead throw a clear error or assert when ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS[connector] is missing so tests fail deterministically; update callers isConnectorEnabled and toggleConnector to rely exclusively on getConnectorToggle returning a getByTestId locator; also ensure the page object extends BasePage and imports test/expect from ../../core/fixtures/base.fixture per the E2E guidelines so getByTestId is the primary selector strategy.tests/e2e/features/placeholders/placeholders.spec.ts (1)
13-14:⚠️ Potential issue | 🟡 MinorValidate the
Read moredestination, not only visibility.Line 13 and Line 56 verify presence only; broken or misrouted CTAs would still pass.
✅ Example assertion pattern
- await expect(page.getByRole('button', { name: /Read more/i })).toBeVisible() + const readMore = page.getByRole('link', { name: /Read more/i }) + await expect(readMore).toBeVisible() + await expect(readMore).toHaveAttribute('href', /https?:\/\//)Also applies to: 56-57
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/placeholders/placeholders.spec.ts` around lines 13 - 14, The test only checks visibility of the "Read more" CTA (locator: page.getByRole('button', { name: /Read more/i }) used at the occurrences around lines 13 and 56) so broken or misrouted links would pass; update both assertions to validate the destination by either (a) reading the element's href/aria attribute (locator.getAttribute('href') or getAttribute('aria-label') as appropriate) and asserting it matches the expected path, or (b) performing a click on the locator and asserting navigation via expect(page).toHaveURL(expectedUrl) (or checking the resulting page content); apply the same change for both places where the "Read more" locator is asserted.tests/e2e/features/governance/pages/governance.page.ts (1)
91-93:⚠️ Potential issue | 🟠 MajorKeep form input locators test-id-first and scoped to active dialogs.
Line 91 and Line 135 use
#budgetMaxLimit, and Line 191-Line 192 bypasscustomerNameInputwithgetByLabel. This weakens selector stability and can cross-match when multiple overlays/inputs exist.♻️ Proposed refactor
- const budgetInput = this.page.locator('#budgetMaxLimit') + const budgetInput = this.teamDialog.locator('#budgetMaxLimit') await budgetInput.fill(String(config.budget.maxLimit))- const budgetInput = this.page.locator('#budgetMaxLimit') + const budgetInput = this.customerDialog.locator('#budgetMaxLimit') await budgetInput.fill(String(config.budget.maxLimit))- await this.page.getByLabel('Customer Name').clear() - await this.page.getByLabel('Customer Name').fill(updates.name) + await this.customerNameInput.clear() + await this.customerNameInput.fill(updates.name)As per coding guidelines "
tests/e2e/**/*.ts: E2E test page objects must extend BasePage, use getByTestId() as the primary selector strategy, and import test/expect from ../../core/fixtures/base.fixture, never from@playwright/test."Also applies to: 134-136, 190-193
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/governance/pages/governance.page.ts` around lines 91 - 93, The selectors in the governance page use brittle global selectors and getByLabel calls; replace usages of '#budgetMaxLimit' (budgetInput) and the getByLabel-based customerNameInput with test-id-first, dialog-scoped locators: for example, locate the active dialog container then call getByTestId('budgetMaxLimit') or getByTestId('customerName') on that container (i.e., change code that sets budgetInput and customerNameInput to use this.page.locator(activeDialogSelector).getByTestId(...)); ensure all tests in this file use getByTestId() as primary strategy and import test/expect from ../../core/fixtures/base.fixture per BasePage patterns so selectors are stable across overlays.
🧹 Nitpick comments (7)
tests/e2e/features/plugins/plugins.spec.ts (1)
337-339: Consider replacing hardcoded timeout with a deterministic wait.The
waitForTimeout(1000)is a fixed delay that could make the test flaky (if the response takes longer) or unnecessarily slow (if it completes faster). Consider usingwaitForResponseor polling for a state change instead.♻️ Suggested improvement
- // Wait for response - await pluginsPage.page.waitForTimeout(1000) + // Wait for save operation to complete + await pluginsPage.page.waitForResponse( + (resp) => resp.url().includes('/plugins') && resp.status() !== 0, + { timeout: 5000 } + ).catch(() => {}) // Ignore if no network call (validation may be client-side)Alternatively, wait for either the sheet to close or an error to appear:
await Promise.race([ pluginsPage.sheet.waitFor({ state: 'hidden', timeout: 5000 }), pluginsPage.page.locator('[data-sonner-toast][data-type="error"]').waitFor({ timeout: 5000 }), pluginsPage.sheet.locator('[role="alert"], .text-destructive').waitFor({ timeout: 5000 }), ]).catch(() => {})🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/plugins/plugins.spec.ts` around lines 337 - 339, Replace the hardcoded await pluginsPage.page.waitForTimeout(1000) with a deterministic wait: use Promise.race to wait for either pluginsPage.sheet to become hidden (sheet.waitFor with state 'hidden') or for an error indicator (pluginsPage.page.locator('[data-sonner-toast][data-type="error"]').waitFor or pluginsPage.sheet.locator('[role="alert"], .text-destructive').waitFor) with a sensible timeout (e.g., 5s), and swallow timeout rejections; this ensures the test proceeds as soon as the sheet closes or an error appears rather than relying on a fixed delay.tests/e2e/features/virtual-keys/virtual-keys.spec.ts (2)
121-123: Move limit-input selectors behind the page object and prefergetByTestId.Line 121 and Line 166 use direct
page.locator('#...')selectors in the spec. Please expose these viaVirtualKeysPageusinggetByTestId-first locators to keep selector strategy consistent and reduce DOM-coupling.As per coding guidelines
tests/e2e/**/*.ts: “E2E test page objects must extend BasePage, use getByTestId() as the primary selector strategy...”.Also applies to: 166-168
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/virtual-keys/virtual-keys.spec.ts` around lines 121 - 123, The spec directly uses page.locator('#budgetMaxLimit') instead of delegating to the VirtualKeysPage page object and using getByTestId; update VirtualKeysPage to expose a getter or method (e.g., budgetMaxLimit or getBudgetMaxLimit()) that returns this.getByTestId('budgetMaxLimit') and change the test to call virtualKeysPage.budgetMaxLimit (or virtualKeysPage.getBudgetMaxLimit()) and assert its value, leaving the existing virtualKeysPage.closeSheet() call intact; do the same refactor for the other direct locator usages around lines using ids at 166-168 so all selectors go through the page object and prefer getByTestId.
353-358: Consider asserting copy success explicitly in the test body.Line 357-358 checks row existence, which is a weak proxy for copy behavior. Add an explicit post-copy signal assertion (e.g., helper-returned success state or toast assertion) so this test fails only on copy regressions.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/virtual-keys/virtual-keys.spec.ts` around lines 353 - 358, The current test only checks virtualKeysPage.virtualKeyExists(vkName) after calling virtualKeysPage.copyVirtualKeyValue(vkName), which is a weak proxy for copy behavior; update the test to assert the explicit copy success signal by either (a) changing copyVirtualKeyValue to return a boolean/Promise<boolean> and asserting its result (expect(await virtualKeysPage.copyVirtualKeyValue(vkName)).toBe(true)) or (b) adding and invoking a helper like virtualKeysPage.waitForCopySuccessToast() / virtualKeysPage.hasCopySuccessToast() and asserting it returns true immediately after calling copyVirtualKeyValue, while keeping the existing virtualKeyExists assertion.ui/app/workspace/config/views/mcpView.tsx (1)
163-170: Consider addingdata-testidto the Tool Sync Interval input for consistency.The other numeric inputs (
mcp-agent-depth-input,mcp-tool-timeout-input) havedata-testidattributes, but the Tool Sync Interval input does not. If this input will be tested, consider adding adata-testidfor completeness.Suggested change
<Input id="mcp-tool-sync-interval" + data-testid="mcp-tool-sync-interval-input" type="number" className="w-24" value={localValues.mcp_tool_sync_interval} onChange={(e) => handleToolSyncIntervalChange(e.target.value)} min="0" />🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/config/views/mcpView.tsx` around lines 163 - 170, Add a data-testid to the Tool Sync Interval input so it matches the other numeric inputs; locate the Input with id "mcp-tool-sync-interval" that binds value={localValues.mcp_tool_sync_interval} and onChange={e => handleToolSyncIntervalChange(e.target.value)} and add a matching data-testid (e.g., "mcp-tool-sync-interval") to that component for test consistency.tests/e2e/features/providers/providers.spec.ts (1)
876-880: This branch can skip a real regression in vLLM key fields.When provider selection succeeds, absence of both vLLM fields currently leads to
skipinstead of failure, so the test may not detect breakage in expected vLLM-specific UI.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/providers/providers.spec.ts` around lines 876 - 880, The current test branch silently skips when both vLLM-specific fields (urlVisible and modelVisible) are absent after provider selection; update the block so absence of both fields causes a test failure instead of test.skip. Specifically, replace the test.skip(...) call in the conditional that checks urlVisible and modelVisible with an explicit assertion or fail (e.g., throw new Error(...) or expect(urlVisible || modelVisible).toBe(true)) so the test fails when provider selection succeeds but vLLM fields are missing.tests/e2e/features/mcp-settings/mcp-settings.spec.ts (1)
12-15: Use page-object locators for MCP field assertions.Line 13 and Line 14 hard-code label selectors instead of using
mcpSettingsPagefield locators, which is more brittle over UI copy changes.♻️ Proposed refactor
- await expect(mcpSettingsPage.page.getByLabel('Max Agent Depth')).toBeVisible() - await expect(mcpSettingsPage.page.getByLabel('Tool Execution Timeout (seconds)')).toBeVisible() + await expect(mcpSettingsPage.maxAgentDepthInput).toBeVisible() + await expect(mcpSettingsPage.toolTimeoutInput).toBeVisible()As per coding guidelines "
tests/e2e/**/*.ts: E2E test page objects must extend BasePage, use getByTestId() as the primary selector strategy, and import test/expect from ../../core/fixtures/base.fixture, never from@playwright/test."🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/mcp-settings/mcp-settings.spec.ts` around lines 12 - 15, Replace the direct getByLabel assertions in the test 'should display MCP settings form fields' with the page-object locators exposed by mcpSettingsPage (e.g., use mcpSettingsPage.maxAgentDepthField or mcpSettingsPage.getMaxAgentDepth() and mcpSettingsPage.toolExecutionTimeoutField or mcpSettingsPage.getToolExecutionTimeout() to call .isVisible()/toBeVisible() via the test/expect imported from ../../core/fixtures/base.fixture); also ensure the test imports test and expect from the shared base fixture rather than `@playwright/test` so the page object’s getByTestId()-backed locators are used per the E2E guidelines.tests/e2e/features/governance/pages/governance.page.ts (1)
156-199:editTeam/editCustomeraccept update fields they don’t apply.The signatures allow budget/rate-limit updates, but the implementation only applies name (and customerId for team). This can silently no-op valid caller inputs.
🧩 Proposed API narrowing
+type TeamEditUpdates = Pick<Partial<TeamConfig>, 'name' | 'customerId'> +type CustomerEditUpdates = Pick<Partial<CustomerConfig>, 'name'> - async editTeam(name: string, updates: Partial<TeamConfig>): Promise<void> { + async editTeam(name: string, updates: TeamEditUpdates): Promise<void> { - async editCustomer(name: string, updates: Partial<CustomerConfig>): Promise<void> { + async editCustomer(name: string, updates: CustomerEditUpdates): Promise<void> {🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/governance/pages/governance.page.ts` around lines 156 - 199, The editTeam and editCustomer methods accept Partial<TeamConfig> and Partial<CustomerConfig> but only apply name (and customerId for teams), causing silent no-ops for fields like budget/rateLimit; create narrowed payload types (e.g., TeamEditPayload = { name?: string; customerId?: string | '' } and CustomerEditPayload = { name?: string }) and update the method signatures (editTeam(name: string, updates: Partial<TeamEditPayload>) and editCustomer(name: string, updates: Partial<CustomerEditPayload>)), adjust any callers to the new types, and keep the existing UI handling in editTeam/editCustomer as-is so only applicable fields are accepted and processed.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/e2e/features/governance/pages/governance.page.ts`:
- Line 1: The file imports expect directly from '@playwright/test' which
violates the E2E convention; update the import so that expect is sourced from
the repository's centralized base fixture instead. In governance.page.ts remove
expect from the import list coming from '@playwright/test' (keep Locator and
Page) and add an import that brings expect from the project’s base fixture
module (the centralized base.fixture used across tests) so all assertions use
the shared fixture.
In `@tests/e2e/features/model-limits/model-limits.data.ts`:
- Around line 3-10: The createModelLimitData function contains an unused local
variable timestamp; remove the dead variable declaration (timestamp =
Date.now()) from the function body so the returned ModelLimitConfig only
contains provider, modelName, budget and any overrides. Ensure no other code in
createModelLimitData references timestamp and run tests to confirm no usage
remains.
---
Duplicate comments:
In `@tests/e2e/features/governance/pages/governance.page.ts`:
- Around line 91-93: The selectors in the governance page use brittle global
selectors and getByLabel calls; replace usages of '#budgetMaxLimit'
(budgetInput) and the getByLabel-based customerNameInput with test-id-first,
dialog-scoped locators: for example, locate the active dialog container then
call getByTestId('budgetMaxLimit') or getByTestId('customerName') on that
container (i.e., change code that sets budgetInput and customerNameInput to use
this.page.locator(activeDialogSelector).getByTestId(...)); ensure all tests in
this file use getByTestId() as primary strategy and import test/expect from
../../core/fixtures/base.fixture per BasePage patterns so selectors are stable
across overlays.
In `@tests/e2e/features/observability/pages/observability.page.ts`:
- Around line 261-266: The cleanup currently swallows errors from
saveConfiguration() causing silent failures; update the try block around
selectConnector/ toggleConnector to await saveConfiguration() without a
.catch(() => {}) so persistence errors surface (or rethrow them), and if
necessary wrap the cleanup in a try/finally or propagate the error from the
method containing selectConnector, isConnectorEnabled, isToggleEnabled,
toggleConnector and saveConfiguration so test failures are reported instead of
being suppressed.
- Around line 52-55: The getConnectorToggle function currently falls back to
page.locator('button[role="switch"]').first(), which can bind the wrong switch;
remove that generic first() fallback and instead throw a clear error or assert
when ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS[connector] is missing so tests
fail deterministically; update callers isConnectorEnabled and toggleConnector to
rely exclusively on getConnectorToggle returning a getByTestId locator; also
ensure the page object extends BasePage and imports test/expect from
../../core/fixtures/base.fixture per the E2E guidelines so getByTestId is the
primary selector strategy.
In `@tests/e2e/features/placeholders/placeholders.spec.ts`:
- Around line 13-14: The test only checks visibility of the "Read more" CTA
(locator: page.getByRole('button', { name: /Read more/i }) used at the
occurrences around lines 13 and 56) so broken or misrouted links would pass;
update both assertions to validate the destination by either (a) reading the
element's href/aria attribute (locator.getAttribute('href') or
getAttribute('aria-label') as appropriate) and asserting it matches the expected
path, or (b) performing a click on the locator and asserting navigation via
expect(page).toHaveURL(expectedUrl) (or checking the resulting page content);
apply the same change for both places where the "Read more" locator is asserted.
In `@tests/e2e/features/providers/providers.spec.ts`:
- Around line 768-807: The test can leak workspace config if it fails before the
manual restore; wrap the main save flow in a try/finally so the original value
is always restored: after reading initialValue from
providersPage.pricingOverridesJsonInput.inputValue(), perform the
setPricingOverrides(validOverrides), savePricingOverrides(), and dismissToasts()
inside try, and in finally call providersPage.setPricingOverrides(initialValue)
and then conditionally call
providersPage.pricingOverridesSaveBtn.isDisabled().then(d => !d) followed by
providersPage.savePricingOverrides() and providersPage.dismissToasts() if
restore is enabled; reference providersPage.setPricingOverrides,
providersPage.savePricingOverrides, providersPage.dismissToasts, and
providersPage.pricingOverridesSaveBtn to locate where to add the try/finally.
---
Nitpick comments:
In `@tests/e2e/features/governance/pages/governance.page.ts`:
- Around line 156-199: The editTeam and editCustomer methods accept
Partial<TeamConfig> and Partial<CustomerConfig> but only apply name (and
customerId for teams), causing silent no-ops for fields like budget/rateLimit;
create narrowed payload types (e.g., TeamEditPayload = { name?: string;
customerId?: string | '' } and CustomerEditPayload = { name?: string }) and
update the method signatures (editTeam(name: string, updates:
Partial<TeamEditPayload>) and editCustomer(name: string, updates:
Partial<CustomerEditPayload>)), adjust any callers to the new types, and keep
the existing UI handling in editTeam/editCustomer as-is so only applicable
fields are accepted and processed.
In `@tests/e2e/features/mcp-settings/mcp-settings.spec.ts`:
- Around line 12-15: Replace the direct getByLabel assertions in the test
'should display MCP settings form fields' with the page-object locators exposed
by mcpSettingsPage (e.g., use mcpSettingsPage.maxAgentDepthField or
mcpSettingsPage.getMaxAgentDepth() and mcpSettingsPage.toolExecutionTimeoutField
or mcpSettingsPage.getToolExecutionTimeout() to call .isVisible()/toBeVisible()
via the test/expect imported from ../../core/fixtures/base.fixture); also ensure
the test imports test and expect from the shared base fixture rather than
`@playwright/test` so the page object’s getByTestId()-backed locators are used per
the E2E guidelines.
In `@tests/e2e/features/plugins/plugins.spec.ts`:
- Around line 337-339: Replace the hardcoded await
pluginsPage.page.waitForTimeout(1000) with a deterministic wait: use
Promise.race to wait for either pluginsPage.sheet to become hidden
(sheet.waitFor with state 'hidden') or for an error indicator
(pluginsPage.page.locator('[data-sonner-toast][data-type="error"]').waitFor or
pluginsPage.sheet.locator('[role="alert"], .text-destructive').waitFor) with a
sensible timeout (e.g., 5s), and swallow timeout rejections; this ensures the
test proceeds as soon as the sheet closes or an error appears rather than
relying on a fixed delay.
In `@tests/e2e/features/providers/providers.spec.ts`:
- Around line 876-880: The current test branch silently skips when both
vLLM-specific fields (urlVisible and modelVisible) are absent after provider
selection; update the block so absence of both fields causes a test failure
instead of test.skip. Specifically, replace the test.skip(...) call in the
conditional that checks urlVisible and modelVisible with an explicit assertion
or fail (e.g., throw new Error(...) or expect(urlVisible ||
modelVisible).toBe(true)) so the test fails when provider selection succeeds but
vLLM fields are missing.
In `@tests/e2e/features/virtual-keys/virtual-keys.spec.ts`:
- Around line 121-123: The spec directly uses page.locator('#budgetMaxLimit')
instead of delegating to the VirtualKeysPage page object and using getByTestId;
update VirtualKeysPage to expose a getter or method (e.g., budgetMaxLimit or
getBudgetMaxLimit()) that returns this.getByTestId('budgetMaxLimit') and change
the test to call virtualKeysPage.budgetMaxLimit (or
virtualKeysPage.getBudgetMaxLimit()) and assert its value, leaving the existing
virtualKeysPage.closeSheet() call intact; do the same refactor for the other
direct locator usages around lines using ids at 166-168 so all selectors go
through the page object and prefer getByTestId.
- Around line 353-358: The current test only checks
virtualKeysPage.virtualKeyExists(vkName) after calling
virtualKeysPage.copyVirtualKeyValue(vkName), which is a weak proxy for copy
behavior; update the test to assert the explicit copy success signal by either
(a) changing copyVirtualKeyValue to return a boolean/Promise<boolean> and
asserting its result (expect(await
virtualKeysPage.copyVirtualKeyValue(vkName)).toBe(true)) or (b) adding and
invoking a helper like virtualKeysPage.waitForCopySuccessToast() /
virtualKeysPage.hasCopySuccessToast() and asserting it returns true immediately
after calling copyVirtualKeyValue, while keeping the existing virtualKeyExists
assertion.
In `@ui/app/workspace/config/views/mcpView.tsx`:
- Around line 163-170: Add a data-testid to the Tool Sync Interval input so it
matches the other numeric inputs; locate the Input with id
"mcp-tool-sync-interval" that binds value={localValues.mcp_tool_sync_interval}
and onChange={e => handleToolSyncIntervalChange(e.target.value)} and add a
matching data-testid (e.g., "mcp-tool-sync-interval") to that component for test
consistency.
ℹ️ Review info
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (44)
tests/e2e/core/fixtures/base.fixture.tstests/e2e/features/config/config.spec.tstests/e2e/features/config/pages/config-settings.page.tstests/e2e/features/dashboard/dashboard.spec.tstests/e2e/features/governance/governance.data.tstests/e2e/features/governance/governance.spec.tstests/e2e/features/governance/pages/governance.page.tstests/e2e/features/mcp-auth-config/mcp-auth-config.spec.tstests/e2e/features/mcp-auth-config/pages/mcp-auth-config.page.tstests/e2e/features/mcp-registry/mcp-registry.spec.tstests/e2e/features/mcp-registry/pages/mcp-registry.page.tstests/e2e/features/mcp-settings/mcp-settings.spec.tstests/e2e/features/mcp-settings/pages/mcp-settings.page.tstests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.tstests/e2e/features/mcp-tool-groups/pages/mcp-tool-groups.page.tstests/e2e/features/model-limits/model-limits.data.tstests/e2e/features/model-limits/model-limits.spec.tstests/e2e/features/model-limits/pages/model-limits.page.tstests/e2e/features/observability/observability.spec.tstests/e2e/features/observability/pages/observability.page.tstests/e2e/features/placeholders/placeholders.spec.tstests/e2e/features/plugins/plugins.spec.tstests/e2e/features/providers/pages/providers.page.tstests/e2e/features/providers/providers.spec.tstests/e2e/features/read-more-links/read-more-links.spec.tstests/e2e/features/routing-rules/pages/routing-rules.page.tstests/e2e/features/routing-rules/routing-rules.spec.tstests/e2e/features/virtual-keys/pages/virtual-keys.page.tstests/e2e/features/virtual-keys/virtual-keys.spec.tsui/app/workspace/config/views/clientSettingsView.tsxui/app/workspace/config/views/mcpView.tsxui/app/workspace/config/views/pricingConfigView.tsxui/app/workspace/config/views/securityView.tsxui/app/workspace/governance/views/customerDialog.tsxui/app/workspace/governance/views/teamDialog.tsxui/app/workspace/governance/views/teamsTable.tsxui/app/workspace/mcp-registry/views/mcpClientsTable.tsxui/app/workspace/mcp-registry/views/mcpServersEmptyState.tsxui/app/workspace/model-limits/views/modelLimitSheet.tsxui/app/workspace/model-limits/views/modelLimitsEmptyState.tsxui/app/workspace/model-limits/views/modelLimitsTable.tsxui/app/workspace/plugins/fragments/pluginFormFragments.tsxui/app/workspace/providers/views/providersEmptyState.tsxui/app/workspace/virtual-keys/views/virtualKeysTable.tsx
✅ Files skipped from review due to trivial changes (2)
- ui/app/workspace/mcp-registry/views/mcpClientsTable.tsx
- ui/app/workspace/virtual-keys/views/virtualKeysTable.tsx
🚧 Files skipped from review as they are similar to previous changes (17)
- ui/app/workspace/model-limits/views/modelLimitSheet.tsx
- ui/app/workspace/providers/views/providersEmptyState.tsx
- tests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.ts
- tests/e2e/features/virtual-keys/pages/virtual-keys.page.ts
- tests/e2e/features/config/config.spec.ts
- tests/e2e/features/routing-rules/routing-rules.spec.ts
- ui/app/workspace/model-limits/views/modelLimitsTable.tsx
- tests/e2e/features/model-limits/pages/model-limits.page.ts
- tests/e2e/features/routing-rules/pages/routing-rules.page.ts
- tests/e2e/features/mcp-tool-groups/pages/mcp-tool-groups.page.ts
- tests/e2e/features/mcp-registry/mcp-registry.spec.ts
- tests/e2e/features/governance/governance.spec.ts
- ui/app/workspace/model-limits/views/modelLimitsEmptyState.tsx
- ui/app/workspace/config/views/pricingConfigView.tsx
- tests/e2e/features/model-limits/model-limits.spec.ts
- tests/e2e/features/dashboard/dashboard.spec.ts
- ui/app/workspace/governance/views/teamsTable.tsx
Radheshg04
force-pushed
the
02-27-feat_extend_e2e_ui_tests
branch
from
c34b8a3 to
0e8a703
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
ui/app/workspace/config/views/mcpView.tsx (1)
111-227:⚠️ Potential issue | 🟠 MajorAdd test IDs for the remaining interactive MCP controls.
This view now has partial test-id coverage, but the
mcp-tool-sync-intervalinput and the binding-levelSelectTriggerare still untagged. Please add stabledata-testidvalues so the page stays fully test-id addressable.As per coding guidelines "`ui/app/workspace/**/*.tsx`: UI workspace pages ... must include data-testid attributes on all interactive elements."🔧 Suggested update
<Input id="mcp-tool-sync-interval" + data-testid="mcp-tool-sync-interval-input" type="number" className="w-24" @@ - <Select value={localValues.mcp_code_mode_binding_level} onValueChange={handleCodeModeBindingLevelChange}> - <SelectTrigger id="mcp-binding-level" className="w-56"> + <Select value={localValues.mcp_code_mode_binding_level} onValueChange={handleCodeModeBindingLevelChange}> + <SelectTrigger id="mcp-binding-level" className="w-56" data-testid="mcp-binding-level-select-trigger">🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/config/views/mcpView.tsx` around lines 111 - 227, The mcp tool sync interval Input (id="mcp-tool-sync-interval") and the Code Mode Binding Level SelectTrigger (id="mcp-binding-level") lack stable test IDs; add data-testid attributes to both interactive controls (e.g., data-testid="mcp-tool-sync-interval-input" on the Input for mcp_tool_sync_interval and data-testid="mcp-binding-level-select-trigger" on the SelectTrigger used by mcp_code_mode_binding_level) so tests can reliably target them; ensure the names match the project's test-id naming convention and update any tests if needed.
♻️ Duplicate comments (3)
tests/e2e/features/observability/pages/observability.page.ts (2)
263-266:⚠️ Potential issue | 🟠 MajorCleanup still suppresses save failures.
await this.saveConfiguration().catch(() => {})hides persistence errors, so cleanup can silently fail and leak state into later tests.🔧 Proposed fix
if ((await this.isConnectorEnabled(connector)) && (await this.isToggleEnabled(connector))) { await this.toggleConnector(connector) - await this.saveConfiguration().catch(() => {}) + await this.saveConfiguration() }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/observability/pages/observability.page.ts` around lines 263 - 266, The cleanup code is swallowing save failures by calling await this.saveConfiguration().catch(() => {}); change it to surface errors instead of silencing them: remove the empty .catch and either await this.saveConfiguration() directly or catch and rethrow/log the error so failures fail the test run; update the block surrounding this.isConnectorEnabled, this.isToggleEnabled, toggleConnector and saveConfiguration to propagate or assert on saveConfiguration errors (e.g., await this.saveConfiguration() or catch(err) { /* log */ throw err }) so state leakage cannot be hidden.
52-55:⚠️ Potential issue | 🟠 MajorAvoid nondeterministic toggle fallback.
Falling back to
button[role="switch"]').first()can target the wrong switch when multiple toggles exist. This still breaks deterministic page-object behavior.🔧 Proposed fix
private static readonly CONNECTOR_TOGGLE_TESTIDS: Partial<Record<ObservabilityConnector, string>> = { otel: 'otel-connector-enable-toggle', prometheus: 'prometheus-connector-enable-toggle', + // add remaining connector testids as UI supports them + // maxim: 'maxim-connector-enable-toggle', + // datadog: 'datadog-connector-enable-toggle', + // bigquery: 'bigquery-connector-enable-toggle', + // newrelic: 'newrelic-connector-enable-toggle', } getConnectorToggle(connector: ObservabilityConnector): Locator { const testId = ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS[connector] - return testId ? this.page.getByTestId(testId) : this.page.locator('button[role="switch"]').first() + if (!testId) { + throw new Error(`Missing connector toggle testid mapping for: ${connector}`) + } + return this.page.getByTestId(testId) }#!/bin/bash set -euo pipefail FILE="tests/e2e/features/observability/pages/observability.page.ts" echo "1) Verify fallback selector usage:" rg -n "getConnectorToggle|button\\[role=\"switch\"\\]\\.first\\(" "$FILE" echo echo "2) Compare ObservabilityConnector union vs CONNECTOR_TOGGLE_TESTIDS coverage:" python - <<'PY' import re, pathlib p = pathlib.Path("tests/e2e/features/observability/pages/observability.page.ts") s = p.read_text() u = re.search(r"export type ObservabilityConnector\s*=\s*(.+)", s) connectors = re.findall(r"'([^']+)'", u.group(1)) m = re.search(r"CONNECTOR_TOGGLE_TESTIDS:[\s\S]*?=\s*\{([\s\S]*?)\n\s*\}", s) mapped = re.findall(r"\b([a-z]+)\s*:\s*'[^']+'", m.group(1)) print("connectors:", connectors) print("mapped toggles:", mapped) print("missing mappings:", sorted(set(connectors) - set(mapped))) PYAs per coding guidelines "
tests/e2e/**/*.ts: E2E test page objects must extend BasePage, use getByTestId() as the primary selector strategy, and import test/expect from ../../core/fixtures/base.fixture, never from@playwright/test."🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/observability/pages/observability.page.ts` around lines 52 - 55, The getConnectorToggle method currently falls back to this.page.locator('button[role="switch"]').first(), which is nondeterministic; update ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS to include a test-id entry for every member of the ObservabilityConnector union and remove the fallback path in getConnectorToggle (or make it throw a clear error when a connector is unmapped) so the method always uses this.page.getByTestId(testId); reference the getConnectorToggle function, ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS map, and the ObservabilityConnector type to locate and fix the missing mappings and ensure deterministic selection.tests/e2e/features/governance/pages/governance.page.ts (1)
191-193:⚠️ Potential issue | 🟠 MajorReuse
customerNameInputtest-id locator in edit flow.
editCustomerreverted to label-based selection; use the existingcustomerNameInputlocator for consistency and dialog isolation.As per coding guidelines "`tests/e2e/**/*.ts`: E2E test page objects must ... use getByTestId() as the primary selector strategy."🔧 Suggested fix
- if (updates.name) { - await this.page.getByLabel('Customer Name').clear() - await this.page.getByLabel('Customer Name').fill(updates.name) - } + if (updates.name) { + await this.customerNameInput.clear() + await this.customerNameInput.fill(updates.name) + }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/governance/pages/governance.page.ts` around lines 191 - 193, In edit flow inside the editCustomer method replace the label-based selections for the customer name with the existing customerNameInput test-id locator: use this.page.getByTestId('customerNameInput').clear() and .fill(updates.name) (or the equivalent variable name for the locator) so the page object consistently uses the customerNameInput locator and keeps the dialog-scoped selector rather than getByLabel('Customer Name').
🧹 Nitpick comments (14)
tests/e2e/features/dashboard/dashboard.spec.ts (2)
214-218: Consider a more specific assertion for state restoration.The test navigates with
volume_chart=linebut the assertion only verifies thatvolume_chart=exists in the URL, not that the value is specificallyline. If the intent is to verify state restoration, consider:- expect(url).toMatch(/volume_chart=/) + expect(url).toContain('volume_chart=line')If the value can be normalized to something other than
line, then the current assertion is acceptable but the comment should clarify why.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/dashboard/dashboard.spec.ts` around lines 214 - 218, The assertion for the restored chart state is too loose: replace the generic regex check of volume_chart= with an explicit check that the URL contains the expected value (e.g., expect(url).toContain('volume_chart=line')) so the test ensures the `volume_chart` parameter equals `line`, or if normalization can change the value, update the surrounding comment near the `const url = dashboardPage.page.url()` / `expect(url)` block to explain why only existence of `volume_chart=` is acceptable rather than asserting the exact value.
146-178: Consider adding skip annotations for conditional tests.Tests that conditionally execute assertions based on element visibility (lines 155-161, 171-177) may pass silently when the UI element isn't present. If these filters are enterprise-only features, consider using
test.skip()with a clear reason instead of silent conditional logic:test('should filter cost chart by model', async ({ dashboardPage }) => { await dashboardPage.waitForChartsToLoad() const costModelFilter = dashboardPage.costModelFilter const isVisible = await costModelFilter.isVisible().catch(() => false) if (!isVisible) { test.skip(true, 'Model filter not available in this configuration') return } // ... assertions })This provides better visibility in test reports about which tests were skipped and why.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/dashboard/dashboard.spec.ts` around lines 146 - 178, The conditional-visibility branches in the tests silently bypass assertions; instead, in the 'should filter cost chart by model' and 'should filter usage chart by model' tests use the Playwright test.skip mechanism when the filter elements are not present: after awaiting dashboardPage.waitForChartsToLoad() check costModelFilter.isVisible() (and usageModelFilter.isVisible()), and if false call test.skip(true, 'Model filter not available in this configuration') and return; otherwise proceed to call dashboardPage.filterCostChartByModel('all') / dashboardPage.filterUsageChartByModel('all') and assert via dashboardPage.getSelectedModel(...). This replaces the current if (isVisible) { ... } pattern with an explicit test.skip for clearer test reports.tests/e2e/features/virtual-keys/virtual-keys.spec.ts (2)
480-486: Defensive.catch(() => false)may mask real failures.The pattern
isVisible().catch(() => false)silently swallows errors. While this makes the test more resilient, it could hide actual issues (e.g., page object errors, element detachment). Consider logging or distinguishing between "element not visible" and "error occurred."- const isVisible = await providerSection.isVisible().catch(() => false) + const isVisible = await providerSection.isVisible().catch((error) => { + console.warn('Provider section visibility check failed:', error.message) + return false + })This same pattern appears on line 507.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/virtual-keys/virtual-keys.spec.ts` around lines 480 - 486, Replace the defensive .catch(() => false) around providerSection.isVisible() with an explicit await and a proper try/catch: call await providerSection.isVisible() (from virtualKeysPage.page.getByText(...)) directly, assert on the boolean when it returns, and in the catch block log the caught error (or use test.fail/throw to surface it) so real failures (e.g., element-detached or page errors) are not swallowed; apply the same change to the second occurrence of this pattern on the other line.
117-123: Consider usinggetByTestId()selector for consistency.The test uses
locator('#budgetMaxLimit')(ID selector) whereas the coding guidelines prefergetByTestId()as the primary selector strategy. If adata-testidattribute exists or can be added for this input, consider using it for consistency across the test suite.The verification logic itself is sound—waiting for animation, asserting the saved value, and closing the sheet.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/virtual-keys/virtual-keys.spec.ts` around lines 117 - 123, Replace the direct ID locator used in the test with the test-id selector for consistency: in the virtual-keys spec where you call virtualKeysPage.viewVirtualKey(vkData.name) then virtualKeysPage.waitForSheetAnimation(), change the budget input lookup that currently uses locator('#budgetMaxLimit') to use getByTestId('<appropriate-testid>') (e.g., 'budget-max-limit'); if the input lacks a data-testid attribute, add one in the component, then update the assertion to expect the value from virtualKeysPage.page.getByTestId('<appropriate-testid>') before calling virtualKeysPage.closeSheet().tests/e2e/features/mcp-registry/mcp-registry.spec.ts (1)
281-283: Optional: centralize allowed status literals to avoid drift.The same status set is asserted in multiple tests; extracting a module constant would reduce duplication and keep future updates consistent.
♻️ Suggested refactor
+const ALLOWED_CLIENT_STATUSES = ['connected', 'disconnected', 'connecting', 'error'] as const + test.describe('MCP Registry', () => { @@ - expect(['connected', 'disconnected', 'connecting', 'error']).toContain(status.toLowerCase()) + expect(ALLOWED_CLIENT_STATUSES).toContain(status.toLowerCase()) @@ - expect(['connected', 'disconnected', 'connecting', 'error']).toContain(status?.toLowerCase()) + expect(ALLOWED_CLIENT_STATUSES).toContain(status?.toLowerCase()) @@ - expect(['connected', 'disconnected', 'connecting', 'error']).toContain(status?.toLowerCase()) + expect(ALLOWED_CLIENT_STATUSES).toContain(status?.toLowerCase())🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/mcp-registry/mcp-registry.spec.ts` around lines 281 - 283, The test repeats the allowed status literals; extract a shared constant (e.g., ALLOWED_CLIENT_STATUSES) and use it in the assertion to avoid drift. Add the constant to a central test-utils or a shared constants module and update the spec to replace the inline array (used with mcpRegistryPage.getClientStatus and the status variable) with ALLOWED_CLIENT_STATUSES.map(s => s.toLowerCase()) or ensure comparison uses the shared set; update all other tests that assert the same statuses to reference this constant.tests/e2e/features/providers/pages/providers.page.ts (1)
638-655: Make pricing override actions tab-safe inside the page object.These helpers assume Governance tab is already active. Selecting the tab internally makes them less order-dependent and reduces flaky callsites.
♻️ Suggested hardening
async setPricingOverrides(json: string): Promise<void> { + await this.selectConfigTab('governance') await this.pricingOverridesJsonInput.clear() await this.pricingOverridesJsonInput.fill(json) } async resetPricingOverrides(): Promise<void> { + await this.selectConfigTab('governance') await this.pricingOverridesResetBtn.click() } async savePricingOverrides(): Promise<void> { + await this.selectConfigTab('governance') await this.pricingOverridesSaveBtn.click() await this.waitForSuccessToast() }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/providers/pages/providers.page.ts` around lines 638 - 655, The three helpers (setPricingOverrides, resetPricingOverrides, savePricingOverrides) assume the Governance tab is already active; make them tab-safe by ensuring they select/activate the Governance tab at the start of each method. For each method (setPricingOverrides, resetPricingOverrides, savePricingOverrides) add an initial await call to the page object's tab-selector (e.g. await this.selectGovernanceTab() or await this.governanceTab.click()) before interacting with pricingOverridesJsonInput, pricingOverridesResetBtn, or pricingOverridesSaveBtn so the actions work regardless of which tab is currently shown.tests/e2e/features/governance/pages/governance.page.ts (1)
6-27: Align config interfaces with implemented behavior.
TeamConfig/CustomerConfigexposerateLimitand budgetresetDuration, but create/edit methods currently ignore them. Either implement those fields or trim the interface to avoid misleading API surface.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/governance/pages/governance.page.ts` around lines 6 - 27, The TeamConfig and CustomerConfig interfaces declare budget.resetDuration and a rateLimit block that the create/edit flows don't actually handle; update the implementation so these fields are processed or remove them from the interfaces to avoid a misleading API. Concretely, either extend the createTeam/createCustomer and editTeam/editCustomer handlers to accept and persist budget.resetDuration and all rateLimit properties (tokenMaxLimit, tokenResetDuration, requestMaxLimit, requestResetDuration), mapping them into the same storage/DTOs used for budget and limits, or remove those properties from TeamConfig and CustomerConfig so the types reflect current behavior; modify the symbols TeamConfig, CustomerConfig and the create/edit methods (createTeam/createCustomer/editTeam/editCustomer) accordingly so the types and runtime behavior stay in sync.tests/e2e/features/model-limits/model-limits.spec.ts (1)
31-44: Consider using unique model identifiers to improve test isolation.Multiple tests use the same fixed
modelName: 'gpt-4o-mini'andprovider: 'openai'. If one test's cleanup fails, subsequent tests may encounter stale data. Since model limits are keyed by(modelName, provider), consider either:
- Using a unique suffix in the model name (e.g.,
gpt-4o-mini-${Date.now()})- Adding a pre-test check to delete any existing limit for that model
The current approach relies on cleanup always succeeding, which may be fragile in CI environments.
Also applies to: 46-62
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/model-limits/model-limits.spec.ts` around lines 31 - 44, The test 'should create a model limit with budget' (and the similar tests around lines 46-62) uses a fixed modelName 'gpt-4o-mini' causing possible cross-test interference; update the test to generate a unique modelName (e.g., append Date.now() or a UUID) when calling createModelLimitData, and/or call modelLimitsPage.deleteModelLimit(modelName, provider) (or a pre-test cleanup helper) before creating the limit; adjust references to createdLimits.push to store the generated modelName and provider so teardown still removes the correct entry and update modelLimitExists and createModelLimit calls to use the new unique modelName.tests/e2e/features/config/config.spec.ts (2)
73-129: Duplicate describe block names create confusion.There are two describe blocks named for pricing config:
- Lines 73-129:
'Pricing Config'with detailed tests for URL, force sync, and validation- Lines 476-493:
'Pricing Config Settings'with just a heading visibility testThis duplication is confusing and the second block (lines 476-493) appears to be a generic state-capture pattern that doesn't add value beyond what's already in the first block.
♻️ Consider consolidating or removing the duplicate
Either remove the
'Pricing Config Settings'block (lines 476-493) since it only verifies the heading which is already implicitly tested by the first block, or merge any unique tests into the primary'Pricing Config'block.Also applies to: 476-493
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/config/config.spec.ts` around lines 73 - 129, There are two test suites with overlapping names — describe('Pricing Config') and describe('Pricing Config Settings') — causing duplication; remove the redundant describe('Pricing Config Settings') block (or merge any unique tests from it into the existing describe('Pricing Config') suite) so all pricing-related tests live under the single describe('Pricing Config') suite, and ensure any unique assertions from the removed block (e.g., heading visibility) are retained in the main suite if needed.
123-128: URL validation test may fail if save button is already disabled.The test fills an invalid URL and clicks save without first verifying the save button is enabled. If the button is disabled (e.g., due to RBAC or no prior changes), the click may not trigger validation feedback.
♻️ Suggested guard
test('should validate URL format', async ({ configSettingsPage }) => { await configSettingsPage.pricingDatasheetUrlInput.fill('invalid-url-no-http') + + const isSaveEnabled = await configSettingsPage.pricingSaveBtn.isDisabled().then((d) => !d) + if (!isSaveEnabled) { + test.skip(true, 'Save button disabled - cannot test validation') + return + } + await configSettingsPage.pricingSaveBtn.click() await expect(configSettingsPage.page.getByText(/URL must start with http|valid URL/i)).toBeVisible() })🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/config/config.spec.ts` around lines 123 - 128, Test may click the save button when it's disabled so validation never runs; before calling configSettingsPage.pricingSaveBtn.click() assert the button is enabled (use expect(configSettingsPage.pricingSaveBtn).toBeEnabled() or check .isEnabled()) and only then click, or alternatively enable the button by making a visible change to the form (e.g., refilling the input) so that configSettingsPage.pricingSaveBtn becomes enabled before clicking; reference pricingDatasheetUrlInput and pricingSaveBtn to locate the elements.tests/e2e/features/providers/providers.spec.ts (2)
876-880: Redundant assertion in vLLM field check.Line 877's assertion
expect(urlVisible || modelVisible).toBe(true)is redundant because we only reach this branch whenurlVisible || modelVisibleis already true (from line 876's condition).♻️ Suggested simplification
const urlVisible = await vllmUrlInput.isVisible().catch(() => false) const modelVisible = await vllmModelInput.isVisible().catch(() => false) - if (urlVisible || modelVisible) { - expect(urlVisible || modelVisible).toBe(true) - } else { + if (!urlVisible && !modelVisible) { test.skip(true, 'vLLM key form fields not shown (provider may use standard key form)') } + // At least one vLLM-specific field is visible + expect(urlVisible || modelVisible).toBe(true) await providersPage.keyCancelBtn.click()🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/providers/providers.spec.ts` around lines 876 - 880, The if/else block around urlVisible and modelVisible contains a redundant assertion—remove the tautological expect(urlVisible || modelVisible).toBe(true) inside the if branch and instead invert the logic to skip when neither is visible: if (!(urlVisible || modelVisible)) { test.skip(true, 'vLLM key form fields not shown (provider may use standard key form)') } so the happy path simply proceeds without the needless assertion (referencing the urlVisible and modelVisible variables and the test.skip call).
368-378: Weak assertion provides no meaningful verification.The test checks
typeof modelsVisible === 'boolean'which is always true sinceisVisible().catch(() => false)returns a boolean. IfmodelsVisibleis true, asserting visibility again is redundant. If false, nothing is verified.Consider either:
- Removing this test if models section visibility is environment-dependent
- Making a deterministic assertion about expected state
♻️ Suggested improvement
test('should show provider models list', async ({ providersPage }) => { await providersPage.selectProvider('openai') - // Check for models section or tab - verify it exists and has deterministic state const modelsSection = providersPage.page.getByText(/Models/i).first() - const modelsVisible = await modelsSection.isVisible().catch(() => false) - - // Models section is either visible (when models tab/section exists) or not - // Assert we observed a concrete state rather than undefined - expect(typeof modelsVisible).toBe('boolean') - if (modelsVisible) { - await expect(modelsSection).toBeVisible() - } + // Models section should be visible for standard providers + await expect(modelsSection).toBeVisible() })🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/providers/providers.spec.ts` around lines 368 - 378, The current boolean-type assertion is meaningless; either remove this weak check block around modelsSection/modelsVisible or replace it with a deterministic presence+visibility check: use providersPage.page.getByText(/Models/i).first().count() to assert whether the element exists (e.g., expect(count).toBeGreaterThanOrEqual(0) or expect(count).toBe(1) depending on environment) and if count > 0 then assert visibility with await expect(modelsSection).toBeVisible(); update the modelsSection/modelsVisible logic accordingly so the test verifies a concrete expected condition instead of typeof modelsVisible === 'boolean'.tests/e2e/features/governance/governance.spec.ts (2)
77-80: Type assertion may fail if API response structure differs.The
expect(customer).toBeDefined()assertion on line 79 runs at test time, but the type assertion(customer as { id: string }).idon line 80 assumes a specific response shape. If the API returns a different structure (e.g.,customerIdinstead ofid), this will produce a runtime error after the assertion passes.Consider adding a more explicit type guard or using optional chaining with a fallback:
const customerId = customer?.id expect(customerId).toBeDefined()🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/governance/governance.spec.ts` around lines 77 - 80, The test assumes the returned customer object has an id field and then force-casts it; instead, make the extraction defensive: retrieve the id from the found customer using optional chaining or an explicit type guard (e.g., read customer?.id or check for 'id' in customer) and then assert that customerId is defined before using it; update the block around customersApi.getAll, the customer variable and the customerId extraction to perform the safe read and corresponding expect.
26-37: Consider optimizing cleanup navigation.The cleanup loop navigates to the customers page for each customer individually (line 28). If multiple customers were created, this adds unnecessary navigation overhead.
♻️ Suggested optimization
+ if (createdCustomers.length > 0) { + await governancePage.gotoCustomers() + } for (const name of [...createdCustomers]) { try { - await governancePage.gotoCustomers() const exists = await governancePage.customerExists(name)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/governance/governance.spec.ts` around lines 26 - 37, The cleanup loop currently calls governancePage.gotoCustomers() inside the for-loop for each name, causing repeated navigation; instead, call governancePage.gotoCustomers() once before iterating over createdCustomers (or call it once and only navigate again if a page action triggers navigation away), then loop through createdCustomers and use governancePage.customerExists(name) and governancePage.deleteCustomer(name) as before; update references to createdCustomers, governancePage.gotoCustomers(), governancePage.customerExists(), and governancePage.deleteCustomer() accordingly so navigation happens once and per-customer actions remain unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/e2e/features/governance/pages/governance.page.ts`:
- Around line 92-95: Replace the CSS id selector used to target the budget input
with a test-id-based locator: change the page.locator('#budgetMaxLimit') usage
(variable budgetInput) to use this.page.getByTestId(...) with the appropriate
data-testid for the max budget field, and fill it the same way; do the same for
the other occurrence referenced around lines 135-138 so both budget inputs
follow the getByTestId() selector contract used by the page object tests.
In `@tests/e2e/features/providers/pages/providers.page.ts`:
- Around line 211-225: Replace brittle structural selectors in getKeyWeight and
getKeyEnabledState with test-id based locators: inside getKeyWeight (which
currently uses keyRow.locator('td').nth(1)), use
keyRow.getByTestId('key-weight') (or the agreed data-testid) to read
textContent; inside getKeyEnabledState (which currently uses
keyRow.locator('button[role="switch"]') and reads data-state), use
keyRow.getByTestId('key-enabled-switch') (or the agreed test id) and read its
'data-state' (or a boolean attribute) the same way, falling back to safe
defaults as before; keep the existing getKeyRow usage to scope the lookups so
only locator strings change.
In `@tests/e2e/features/read-more-links/read-more-links.spec.ts`:
- Around line 65-66: The current RegExp built from DOCS_BASE only escapes the
first dot because String.replace was used without a global regex; update the
assertion in read-more-links.spec.ts (where popup.url() and DOCS_BASE are used)
to either escape all dots when constructing the RegExp (e.g., replace all '.'
via DOCS_BASE.replace(/\./g, '\\.') before creating new RegExp) or simplify the
check by asserting url.startsWith(`${DOCS_BASE}/`) instead of using RegExp;
apply the change to the expect(url)... line so the URL prefix check is robust.
---
Outside diff comments:
In `@ui/app/workspace/config/views/mcpView.tsx`:
- Around line 111-227: The mcp tool sync interval Input
(id="mcp-tool-sync-interval") and the Code Mode Binding Level SelectTrigger
(id="mcp-binding-level") lack stable test IDs; add data-testid attributes to
both interactive controls (e.g., data-testid="mcp-tool-sync-interval-input" on
the Input for mcp_tool_sync_interval and
data-testid="mcp-binding-level-select-trigger" on the SelectTrigger used by
mcp_code_mode_binding_level) so tests can reliably target them; ensure the names
match the project's test-id naming convention and update any tests if needed.
---
Duplicate comments:
In `@tests/e2e/features/governance/pages/governance.page.ts`:
- Around line 191-193: In edit flow inside the editCustomer method replace the
label-based selections for the customer name with the existing customerNameInput
test-id locator: use this.page.getByTestId('customerNameInput').clear() and
.fill(updates.name) (or the equivalent variable name for the locator) so the
page object consistently uses the customerNameInput locator and keeps the
dialog-scoped selector rather than getByLabel('Customer Name').
In `@tests/e2e/features/observability/pages/observability.page.ts`:
- Around line 263-266: The cleanup code is swallowing save failures by calling
await this.saveConfiguration().catch(() => {}); change it to surface errors
instead of silencing them: remove the empty .catch and either await
this.saveConfiguration() directly or catch and rethrow/log the error so failures
fail the test run; update the block surrounding this.isConnectorEnabled,
this.isToggleEnabled, toggleConnector and saveConfiguration to propagate or
assert on saveConfiguration errors (e.g., await this.saveConfiguration() or
catch(err) { /* log */ throw err }) so state leakage cannot be hidden.
- Around line 52-55: The getConnectorToggle method currently falls back to
this.page.locator('button[role="switch"]').first(), which is nondeterministic;
update ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS to include a test-id entry for
every member of the ObservabilityConnector union and remove the fallback path in
getConnectorToggle (or make it throw a clear error when a connector is unmapped)
so the method always uses this.page.getByTestId(testId); reference the
getConnectorToggle function, ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS map, and
the ObservabilityConnector type to locate and fix the missing mappings and
ensure deterministic selection.
---
Nitpick comments:
In `@tests/e2e/features/config/config.spec.ts`:
- Around line 73-129: There are two test suites with overlapping names —
describe('Pricing Config') and describe('Pricing Config Settings') — causing
duplication; remove the redundant describe('Pricing Config Settings') block (or
merge any unique tests from it into the existing describe('Pricing Config')
suite) so all pricing-related tests live under the single describe('Pricing
Config') suite, and ensure any unique assertions from the removed block (e.g.,
heading visibility) are retained in the main suite if needed.
- Around line 123-128: Test may click the save button when it's disabled so
validation never runs; before calling configSettingsPage.pricingSaveBtn.click()
assert the button is enabled (use
expect(configSettingsPage.pricingSaveBtn).toBeEnabled() or check .isEnabled())
and only then click, or alternatively enable the button by making a visible
change to the form (e.g., refilling the input) so that
configSettingsPage.pricingSaveBtn becomes enabled before clicking; reference
pricingDatasheetUrlInput and pricingSaveBtn to locate the elements.
In `@tests/e2e/features/dashboard/dashboard.spec.ts`:
- Around line 214-218: The assertion for the restored chart state is too loose:
replace the generic regex check of volume_chart= with an explicit check that the
URL contains the expected value (e.g.,
expect(url).toContain('volume_chart=line')) so the test ensures the
`volume_chart` parameter equals `line`, or if normalization can change the
value, update the surrounding comment near the `const url =
dashboardPage.page.url()` / `expect(url)` block to explain why only existence of
`volume_chart=` is acceptable rather than asserting the exact value.
- Around line 146-178: The conditional-visibility branches in the tests silently
bypass assertions; instead, in the 'should filter cost chart by model' and
'should filter usage chart by model' tests use the Playwright test.skip
mechanism when the filter elements are not present: after awaiting
dashboardPage.waitForChartsToLoad() check costModelFilter.isVisible() (and
usageModelFilter.isVisible()), and if false call test.skip(true, 'Model filter
not available in this configuration') and return; otherwise proceed to call
dashboardPage.filterCostChartByModel('all') /
dashboardPage.filterUsageChartByModel('all') and assert via
dashboardPage.getSelectedModel(...). This replaces the current if (isVisible) {
... } pattern with an explicit test.skip for clearer test reports.
In `@tests/e2e/features/governance/governance.spec.ts`:
- Around line 77-80: The test assumes the returned customer object has an id
field and then force-casts it; instead, make the extraction defensive: retrieve
the id from the found customer using optional chaining or an explicit type guard
(e.g., read customer?.id or check for 'id' in customer) and then assert that
customerId is defined before using it; update the block around
customersApi.getAll, the customer variable and the customerId extraction to
perform the safe read and corresponding expect.
- Around line 26-37: The cleanup loop currently calls
governancePage.gotoCustomers() inside the for-loop for each name, causing
repeated navigation; instead, call governancePage.gotoCustomers() once before
iterating over createdCustomers (or call it once and only navigate again if a
page action triggers navigation away), then loop through createdCustomers and
use governancePage.customerExists(name) and governancePage.deleteCustomer(name)
as before; update references to createdCustomers,
governancePage.gotoCustomers(), governancePage.customerExists(), and
governancePage.deleteCustomer() accordingly so navigation happens once and
per-customer actions remain unchanged.
In `@tests/e2e/features/governance/pages/governance.page.ts`:
- Around line 6-27: The TeamConfig and CustomerConfig interfaces declare
budget.resetDuration and a rateLimit block that the create/edit flows don't
actually handle; update the implementation so these fields are processed or
remove them from the interfaces to avoid a misleading API. Concretely, either
extend the createTeam/createCustomer and editTeam/editCustomer handlers to
accept and persist budget.resetDuration and all rateLimit properties
(tokenMaxLimit, tokenResetDuration, requestMaxLimit, requestResetDuration),
mapping them into the same storage/DTOs used for budget and limits, or remove
those properties from TeamConfig and CustomerConfig so the types reflect current
behavior; modify the symbols TeamConfig, CustomerConfig and the create/edit
methods (createTeam/createCustomer/editTeam/editCustomer) accordingly so the
types and runtime behavior stay in sync.
In `@tests/e2e/features/mcp-registry/mcp-registry.spec.ts`:
- Around line 281-283: The test repeats the allowed status literals; extract a
shared constant (e.g., ALLOWED_CLIENT_STATUSES) and use it in the assertion to
avoid drift. Add the constant to a central test-utils or a shared constants
module and update the spec to replace the inline array (used with
mcpRegistryPage.getClientStatus and the status variable) with
ALLOWED_CLIENT_STATUSES.map(s => s.toLowerCase()) or ensure comparison uses the
shared set; update all other tests that assert the same statuses to reference
this constant.
In `@tests/e2e/features/model-limits/model-limits.spec.ts`:
- Around line 31-44: The test 'should create a model limit with budget' (and the
similar tests around lines 46-62) uses a fixed modelName 'gpt-4o-mini' causing
possible cross-test interference; update the test to generate a unique modelName
(e.g., append Date.now() or a UUID) when calling createModelLimitData, and/or
call modelLimitsPage.deleteModelLimit(modelName, provider) (or a pre-test
cleanup helper) before creating the limit; adjust references to
createdLimits.push to store the generated modelName and provider so teardown
still removes the correct entry and update modelLimitExists and createModelLimit
calls to use the new unique modelName.
In `@tests/e2e/features/providers/pages/providers.page.ts`:
- Around line 638-655: The three helpers (setPricingOverrides,
resetPricingOverrides, savePricingOverrides) assume the Governance tab is
already active; make them tab-safe by ensuring they select/activate the
Governance tab at the start of each method. For each method
(setPricingOverrides, resetPricingOverrides, savePricingOverrides) add an
initial await call to the page object's tab-selector (e.g. await
this.selectGovernanceTab() or await this.governanceTab.click()) before
interacting with pricingOverridesJsonInput, pricingOverridesResetBtn, or
pricingOverridesSaveBtn so the actions work regardless of which tab is currently
shown.
In `@tests/e2e/features/providers/providers.spec.ts`:
- Around line 876-880: The if/else block around urlVisible and modelVisible
contains a redundant assertion—remove the tautological expect(urlVisible ||
modelVisible).toBe(true) inside the if branch and instead invert the logic to
skip when neither is visible: if (!(urlVisible || modelVisible)) {
test.skip(true, 'vLLM key form fields not shown (provider may use standard key
form)') } so the happy path simply proceeds without the needless assertion
(referencing the urlVisible and modelVisible variables and the test.skip call).
- Around line 368-378: The current boolean-type assertion is meaningless; either
remove this weak check block around modelsSection/modelsVisible or replace it
with a deterministic presence+visibility check: use
providersPage.page.getByText(/Models/i).first().count() to assert whether the
element exists (e.g., expect(count).toBeGreaterThanOrEqual(0) or
expect(count).toBe(1) depending on environment) and if count > 0 then assert
visibility with await expect(modelsSection).toBeVisible(); update the
modelsSection/modelsVisible logic accordingly so the test verifies a concrete
expected condition instead of typeof modelsVisible === 'boolean'.
In `@tests/e2e/features/virtual-keys/virtual-keys.spec.ts`:
- Around line 480-486: Replace the defensive .catch(() => false) around
providerSection.isVisible() with an explicit await and a proper try/catch: call
await providerSection.isVisible() (from virtualKeysPage.page.getByText(...))
directly, assert on the boolean when it returns, and in the catch block log the
caught error (or use test.fail/throw to surface it) so real failures (e.g.,
element-detached or page errors) are not swallowed; apply the same change to the
second occurrence of this pattern on the other line.
- Around line 117-123: Replace the direct ID locator used in the test with the
test-id selector for consistency: in the virtual-keys spec where you call
virtualKeysPage.viewVirtualKey(vkData.name) then
virtualKeysPage.waitForSheetAnimation(), change the budget input lookup that
currently uses locator('#budgetMaxLimit') to use
getByTestId('<appropriate-testid>') (e.g., 'budget-max-limit'); if the input
lacks a data-testid attribute, add one in the component, then update the
assertion to expect the value from
virtualKeysPage.page.getByTestId('<appropriate-testid>') before calling
virtualKeysPage.closeSheet().
ℹ️ Review info
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (44)
tests/e2e/core/fixtures/base.fixture.tstests/e2e/features/config/config.spec.tstests/e2e/features/config/pages/config-settings.page.tstests/e2e/features/dashboard/dashboard.spec.tstests/e2e/features/governance/governance.data.tstests/e2e/features/governance/governance.spec.tstests/e2e/features/governance/pages/governance.page.tstests/e2e/features/mcp-auth-config/mcp-auth-config.spec.tstests/e2e/features/mcp-auth-config/pages/mcp-auth-config.page.tstests/e2e/features/mcp-registry/mcp-registry.spec.tstests/e2e/features/mcp-registry/pages/mcp-registry.page.tstests/e2e/features/mcp-settings/mcp-settings.spec.tstests/e2e/features/mcp-settings/pages/mcp-settings.page.tstests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.tstests/e2e/features/mcp-tool-groups/pages/mcp-tool-groups.page.tstests/e2e/features/model-limits/model-limits.data.tstests/e2e/features/model-limits/model-limits.spec.tstests/e2e/features/model-limits/pages/model-limits.page.tstests/e2e/features/observability/observability.spec.tstests/e2e/features/observability/pages/observability.page.tstests/e2e/features/placeholders/placeholders.spec.tstests/e2e/features/plugins/plugins.spec.tstests/e2e/features/providers/pages/providers.page.tstests/e2e/features/providers/providers.spec.tstests/e2e/features/read-more-links/read-more-links.spec.tstests/e2e/features/routing-rules/pages/routing-rules.page.tstests/e2e/features/routing-rules/routing-rules.spec.tstests/e2e/features/virtual-keys/pages/virtual-keys.page.tstests/e2e/features/virtual-keys/virtual-keys.spec.tsui/app/workspace/config/views/clientSettingsView.tsxui/app/workspace/config/views/mcpView.tsxui/app/workspace/config/views/pricingConfigView.tsxui/app/workspace/config/views/securityView.tsxui/app/workspace/governance/views/customerDialog.tsxui/app/workspace/governance/views/teamDialog.tsxui/app/workspace/governance/views/teamsTable.tsxui/app/workspace/mcp-registry/views/mcpClientsTable.tsxui/app/workspace/mcp-registry/views/mcpServersEmptyState.tsxui/app/workspace/model-limits/views/modelLimitSheet.tsxui/app/workspace/model-limits/views/modelLimitsEmptyState.tsxui/app/workspace/model-limits/views/modelLimitsTable.tsxui/app/workspace/plugins/fragments/pluginFormFragments.tsxui/app/workspace/providers/views/providersEmptyState.tsxui/app/workspace/virtual-keys/views/virtualKeysTable.tsx
🚧 Files skipped from review as they are similar to previous changes (22)
- ui/app/workspace/governance/views/customerDialog.tsx
- ui/app/workspace/config/views/clientSettingsView.tsx
- tests/e2e/features/mcp-tool-groups/pages/mcp-tool-groups.page.ts
- ui/app/workspace/model-limits/views/modelLimitsEmptyState.tsx
- tests/e2e/features/routing-rules/routing-rules.spec.ts
- tests/e2e/features/model-limits/model-limits.data.ts
- tests/e2e/features/plugins/plugins.spec.ts
- ui/app/workspace/virtual-keys/views/virtualKeysTable.tsx
- tests/e2e/features/virtual-keys/pages/virtual-keys.page.ts
- ui/app/workspace/governance/views/teamDialog.tsx
- ui/app/workspace/mcp-registry/views/mcpClientsTable.tsx
- ui/app/workspace/providers/views/providersEmptyState.tsx
- tests/e2e/features/placeholders/placeholders.spec.ts
- tests/e2e/features/governance/governance.data.ts
- ui/app/workspace/model-limits/views/modelLimitsTable.tsx
- tests/e2e/features/routing-rules/pages/routing-rules.page.ts
- ui/app/workspace/model-limits/views/modelLimitSheet.tsx
- tests/e2e/features/model-limits/pages/model-limits.page.ts
- ui/app/workspace/plugins/fragments/pluginFormFragments.tsx
- tests/e2e/features/mcp-settings/pages/mcp-settings.page.ts
- ui/app/workspace/config/views/pricingConfigView.tsx
- ui/app/workspace/governance/views/teamsTable.tsx
Radheshg04
force-pushed
the
02-27-feat_extend_e2e_ui_tests
branch
from
0e8a703 to
bd26611
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 10
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
ui/app/workspace/config/views/pricingConfigView.tsx (1)
97-153:⚠️ Potential issue | 🟡 MinorAdd a test hook for the pricing sync interval input as well.
The form now has test IDs on most controls, but the
pricing-sync-intervalinput is still missing one.Suggested fix
<Input id="pricing-sync-interval" type="number" + data-testid="pricing-sync-interval-input" className={errors.pricing_sync_interval_hours ? "border-destructive" : ""}As per coding guidelines, "UI workspace pages ... must include data-testid attributes on all interactive elements."
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/config/views/pricingConfigView.tsx` around lines 97 - 153, The pricing sync interval input is missing a test hook; add a data-testid to the Input for the "pricing_sync_interval_hours" field (the Input with id="pricing-sync-interval" / register("pricing_sync_interval_hours")) so automated tests can target it (e.g., data-testid="pricing-sync-interval-input"); ensure the attribute is present on the same Input element that uses register("pricing_sync_interval_hours") and that any tests referencing pricing-sync-interval are updated accordingly.
♻️ Duplicate comments (1)
tests/e2e/features/governance/pages/governance.page.ts (1)
191-194:⚠️ Potential issue | 🟠 MajorUse the page object’s test-id locator in
editCustomer.Lines 192-193 reintroduce label-based lookup instead of the existing
customerNameInputtest-id locator, which weakens selector consistency and resilience.🔧 Suggested fix
- if (updates.name) { - await this.page.getByLabel('Customer Name').clear() - await this.page.getByLabel('Customer Name').fill(updates.name) + if (updates.name !== undefined) { + await this.customerNameInput.clear() + await this.customerNameInput.fill(updates.name) }As per coding guidelines "
tests/e2e/**/*.ts: E2E test page objects must ... use getByTestId() as the primary selector strategy."🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/governance/pages/governance.page.ts` around lines 191 - 194, The editCustomer flow is using label-based selectors for the customer name; replace the two calls to this.page.getByLabel('Customer Name').clear() and .fill(...) with the page object’s test-id locator this.customerNameInput.clear() and this.customerNameInput.fill(updates.name) so the page object’s customerNameInput (test-id) is the primary selector; update only the branches where updates.name is present to use customerNameInput to keep selector consistency.
🧹 Nitpick comments (9)
tests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.ts (1)
11-11: Use a stricter URL matcher to avoid accidental passes.Line 11 currently matches any URL containing
mcp-tool-groups; anchoring to the expected route is safer.Proposed change
- await expect(mcpToolGroupsPage.page).toHaveURL(/mcp-tool-groups/) + await expect(mcpToolGroupsPage.page).toHaveURL(/\/workspace\/mcp-tool-groups\/?$/)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.ts` at line 11, The URL assertion in the test uses a loose regex (/mcp-tool-groups/) which can match unintended routes; update the assertion on mcpToolGroupsPage.page in tests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.ts to use a stricter matcher that anchors the expected route (for example by matching the exact path or start/end anchors such as /^\/mcp-tool-groups\/?$/ or a string equality check) so the test only passes when the page is exactly the mcp-tool-groups route.tests/e2e/features/mcp-registry/mcp-registry.spec.ts (1)
281-284: Consider centralizing valid status values into one constant.The same allowlist appears in multiple tests; extracting one constant will reduce drift risk.
♻️ Suggested refactor
+const VALID_CLIENT_STATUSES = ['connected', 'disconnected', 'connecting', 'error'] as const + test.describe('MCP Registry', () => { @@ - expect(['connected', 'disconnected', 'connecting', 'error']).toContain(status?.toLowerCase()) + expect(VALID_CLIENT_STATUSES).toContain(status?.toLowerCase() as (typeof VALID_CLIENT_STATUSES)[number])🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/mcp-registry/mcp-registry.spec.ts` around lines 281 - 284, The test currently repeats the allowed status array inline; extract that array into a single exported constant (e.g., VALID_CLIENT_STATUSES) and use it in assertions instead of the inline ['connected','disconnected','connecting','error']; update the spec that calls mcpRegistryPage.getClientStatus to import or reference VALID_CLIENT_STATUSES and replace expect([...]).toContain(status?.toLowerCase()) with expect(VALID_CLIENT_STATUSES).toContain(status?.toLowerCase()); place the constant in a shared test utils module or at the top of this spec so other tests can import it to avoid duplication.tests/e2e/features/observability/observability.spec.ts (2)
241-255: BigQuery connector test coverage is minimal.Only the selection test exists for BigQuery. If BigQuery supports toggle/configuration like other connectors, consider adding similar tests for toggle behavior and configuration visibility for completeness.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/observability/observability.spec.ts` around lines 241 - 255, The BigQuery Connector e2e test only verifies selection; add tests mirroring other connectors to cover toggle and configuration visibility: expand the test.describe('BigQuery Connector') block to, after observabilityPage.selectConnector('bigquery'), assert toggle state via observabilityPage.isConnectorToggled('bigquery') or toggle it with observabilityPage.toggleConnector('bigquery') and check visibility of configuration fields with observabilityPage.isConnectorConfigVisible('bigquery') (or equivalent helper methods used by other connector tests) and assert expected behavior when toggled on/off.
21-22: Consider using a test-id for the providers header.The class selector
.text-muted-foregroundis fragile if styles change. A dedicateddata-testidon the sidebar section header would be more robust for this assertion.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/observability/observability.spec.ts` around lines 21 - 22, The test uses a fragile CSS selector in observability.spec.ts (observabilityPage.page.locator('.text-muted-foreground').filter({ hasText: 'Providers' }).first() assigned to providersHeader); change the app markup to add a stable data-testid (e.g., data-testid="sidebar-providers-header") on the Providers sidebar header and update the test to locate by that test-id (observabilityPage.page.locator('[data-testid="sidebar-providers-header"]') or equivalent) before asserting visibility so the test no longer depends on styling classes.tests/e2e/features/observability/pages/observability.page.ts (1)
1-1: Remove unusedexpectimport.The
expectfunction is imported from@playwright/testbut is not used anywhere in this page object. Page objects should delegate assertions to spec files. Remove the unused import to keep dependencies minimal.-import { Page, Locator, expect } from '@playwright/test' +import { Page, Locator } from '@playwright/test'🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/observability/pages/observability.page.ts` at line 1, Remove the unused "expect" import from the import statement in the Observability page object; update the import declaration that currently reads "import { Page, Locator, expect } from '@playwright/test'" to only import the used symbols (Page and Locator) so assertions remain in spec files and the page object doesn't include unused dependencies.tests/e2e/features/placeholders/placeholders.spec.ts (1)
6-7: Prefer test-id-first selectors for long-term test stability.These checks currently depend on visible copy/role. Where stable test IDs exist, switch primary lookup to
getByTestId()and keep text assertions as secondary intent checks.As per coding guidelines "
tests/e2e/**/*.ts: ... use getByTestId() as the primary selector strategy".Also applies to: 12-14, 59-61
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/placeholders/placeholders.spec.ts` around lines 6 - 7, Replace the brittle text/role selector usage with the test-id-first strategy: for each occurrence of page.getByText(/Prompt repository is coming soon/i) (and the similar checks at lines referenced 12-14 and 59-61), swap the primary lookup to page.getByTestId('<stable-test-id>') (use the existing stable test id for the prompt repository element) and then keep the visible/text check as a secondary assertion (e.g., assert the element is visible and that its text matches via toHaveText or expect(...).toContainText). Update the three places to use page.getByTestId(...) as the primary selector and keep the regex text assertions only as intent verification.tests/e2e/features/virtual-keys/virtual-keys.spec.ts (1)
121-122: Move raw ID selectors behind page-objectgetByTestIdlocators.Lines 121 and 166 use
page.locator('#...')directly in the spec. Per coding guidelines, tests should use page-object abstractions withgetByTestId()as the primary selector strategy to reduce brittleness and maintain consistency.Create getter properties in the page object and access them through
virtualKeysPageinstead:♻️ Suggested refactor
- const budgetInput = virtualKeysPage.page.locator('#budgetMaxLimit') + const budgetInput = virtualKeysPage.budgetMaxLimitInput await expect(budgetInput).toHaveValue(String(SAMPLE_BUDGETS.small.maxLimit)) - const tokenLimitInput = virtualKeysPage.page.locator('#tokenMaxLimit') + const tokenLimitInput = virtualKeysPage.tokenMaxLimitInput await expect(tokenLimitInput).toHaveValue(String(SAMPLE_RATE_LIMITS.tokenOnly.tokenMaxLimit))// tests/e2e/features/virtual-keys/pages/virtual-keys.page.ts get budgetMaxLimitInput() { return this.page.getByTestId('vk-budget-max-limit-input') } get tokenMaxLimitInput() { return this.page.getByTestId('vk-token-max-limit-input') }Also applies to: 166–167
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/virtual-keys/virtual-keys.spec.ts` around lines 121 - 122, Replace direct locators in the spec that use page.locator('#budgetMaxLimit') and the token input with page-object getters on virtualKeysPage; add getters named budgetMaxLimitInput and tokenMaxLimitInput in tests/e2e/features/virtual-keys/pages/virtual-keys.page.ts that return this.page.getByTestId('vk-budget-max-limit-input') and this.page.getByTestId('vk-token-max-limit-input') respectively, then update the spec to use virtualKeysPage.budgetMaxLimitInput and virtualKeysPage.tokenMaxLimitInput when asserting values against SAMPLE_BUDGETS.tests/e2e/features/governance/governance.spec.ts (1)
5-8: Consider serializing this spec while using shared module-level cleanup arrays.
createdTeams/createdCustomersare shared mutable state across tests; adding explicit serial mode here makes local runs safer too.Based on learnings, "ensure E2E tests run with CI=true so Playwright executes with workers: 1 ... This makes module-level cleanup state safe from parallel execution issues."
Also applies to: 108-109
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/governance/governance.spec.ts` around lines 5 - 8, The spec uses shared mutable arrays createdTeams and createdCustomers which makes it unsafe to run tests in parallel; update the suite to run serially by changing the suite declaration for "Governance - Teams" to a serialized describe (use test.describe.serial or equivalent) so tests execute one worker and avoid concurrent access to createdTeams/createdCustomers (also apply the same serialization to the other suite instance referenced near the other occurrence of createdTeams/createdCustomers).ui/app/workspace/governance/views/teamsTable.tsx (1)
166-166: Prefer immutable IDs indata-testidvalues instead of team names.Using
team.namein test hooks can cause flaky selectors when names change, include spaces/special chars, or collide.Suggested refactor
-<TableRow key={team.id} data-testid={`team-row-${team.name}`} className={cn("group transition-colors", isExhausted && "bg-red-500/5 hover:bg-red-500/10")}> +<TableRow key={team.id} data-testid={`team-row-${team.id}`} className={cn("group transition-colors", isExhausted && "bg-red-500/5 hover:bg-red-500/10")}> - data-testid={`team-edit-btn-${team.name}`} + data-testid={`team-edit-btn-${team.id}`} - data-testid={`team-delete-btn-${team.name}`} + data-testid={`team-delete-btn-${team.id}`}Also applies to: 316-316, 328-328
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/governance/views/teamsTable.tsx` at line 166, Replace fragile test hooks that use team.name with an immutable identifier: change the TableRow data-testid and any other test-id usages (e.g., the occurrences around the TableRow at lines showing team usage and the other occurrences near 316 and 328) to use team.id (or another stable unique field) instead of team.name; update the template strings that build `data-testid` to reference `team.id` (ensure it's stringified if necessary) so test selectors no longer depend on mutable/display names.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/e2e/features/config/config.spec.ts`:
- Around line 123-128: The URL validation test clicks the save button
unconditionally which fails if saving is disabled by RBAC; before filling and
clicking, check the save control (configSettingsPage.pricingSaveBtn) and
bail/skip the test when saving is disabled: e.g. query
pricingSaveBtn.isEnabled() or inspect its disabled attribute and call
test.skip(...) or return early if not enabled, then proceed to fill
pricingDatasheetUrlInput and click pricingSaveBtn only when enabled so the
assertion for the URL validation message runs in the correct context.
- Around line 81-89: The afterEach cleanup unconditionally calls
configSettingsPage.setPricingDatasheetUrl which fails in read-only/RBAC
environments; update the teardown to first navigate to 'pricing-config', then
check if the datasheet input/control is editable (e.g., via an isEditable or
isDisabled check on configSettingsPage.pricingDatasheetInput or by reusing the
existing pricingSaveBtn.isDisabled() logic) and only call
configSettingsPage.setPricingDatasheetUrl(originalPricingUrl) and
configSettingsPage.savePricingConfig() when the control is editable; ensure to
still call dismissToasts() conditionally after a successful save.
In `@tests/e2e/features/dashboard/dashboard.spec.ts`:
- Around line 214-218: The test currently only checks that "volume_chart=" is
present which allows wrong chart types to pass; update the assertion that
inspects the URL retrieved from dashboardPage.page.url() (variable url) to
assert the exact seeded value by either using
expect(url).toContain('volume_chart=line') or parse the URL (new URL(url)) and
assert searchParams.get('volume_chart') === 'line' so the test fails if the app
rewrites the chart type.
In `@tests/e2e/features/governance/governance.spec.ts`:
- Around line 94-105: The delete-team test (the test block using createTeamData,
governancePage.createTeam, governancePage.teamExists, and
governancePage.deleteTeam) creates a team but doesn't register it for cleanup;
modify the test to push the created team name into the shared cleanup array
before (or immediately after) calling governancePage.createTeam, and ensure the
suite has an afterEach hook that iterates that array and deletes any remaining
teams via governancePage.deleteTeam (or appropriate API) so stray resources are
removed if the delete assertion fails; apply the same pattern to the other test
that creates teams (the one around lines 158-169).
In `@tests/e2e/features/governance/pages/governance.page.ts`:
- Around line 6-27: The TeamConfig and CustomerConfig interfaces declare
budget.resetDuration and rateLimit fields that are not applied by the
create/edit flows; update the governance create/edit flows (the methods that
consume TeamConfig/CustomerConfig in this file — e.g., the
createTeam/createCustomer and editTeam/editCustomer interactions) to set
budget.resetDuration and each rateLimit subfield (tokenMaxLimit,
tokenResetDuration, requestMaxLimit, requestResetDuration) in the UI form, or if
those UI options are unsupported, remove those properties from TeamConfig and
CustomerConfig so the interfaces only expose supported inputs; ensure any test
helpers that build configs are updated to match the new shapes or updated flows
so no fields are silently ignored.
- Around line 119-121: getCustomerRow currently uses a partial-text filter which
can match wrong rows; replace the hasText + first approach with a deterministic
test-id selector (e.g. use
this.customersTable.getByTestId(`customer-row-${name}`) or
locator(`tr[data-testid="customer-row-${name}"]`) so it targets the exact row),
and ensure callers like editCustomer/deleteCustomer use getCustomerRow(name) so
actions operate on the uniquely identified row.
In `@tests/e2e/features/observability/pages/observability.page.ts`:
- Line 265: The call to saveConfiguration is silently swallowing errors via
.catch(() => {}); change it so failures are surfaced: remove the empty catch and
either await this.saveConfiguration() inside the surrounding try block (so the
outer try-catch handles errors) or replace the empty handler with .catch(err =>
{ /* log or rethrow */ throw err; }) so saveConfiguration errors are logged
and/or propagated instead of being ignored.
In `@tests/e2e/features/placeholders/placeholders.spec.ts`:
- Around line 15-17: Replace the synchronous popup.url() assertion with
Playwright's async URL waiter: after creating the popup via
Promise.all([page.waitForEvent('popup'), readMore.click()]) replace
expect(popup.url()).toMatch(...) with await
expect(popup).toHaveURL(/^https:\/\/docs\.getbifrost\.ai\/enterprise\/alert-channels(\?|$)/)
so the assertion polls until navigation stabilizes; keep the popup.close() call
afterwards unchanged.
In `@tests/e2e/features/routing-rules/routing-rules.spec.ts`:
- Around line 172-179: When count > 0 strengthen the populated-state assertions
in the routing-rules.spec by also asserting that routingRulesPage.emptyState is
not visible; update the branch that currently checks routingRulesPage.table to
include await expect(routingRulesPage.emptyState).not.toBeVisible() so the test
fails if both the table and the empty state are rendered. Locate the logic using
routingRulesPage.getRuleCount(), routingRulesPage.table and
routingRulesPage.emptyState and add the extra negative visibility check in the
populated branch.
In `@ui/app/workspace/governance/views/teamDialog.tsx`:
- Line 220: The DialogContent element in teamDialog.tsx uses a two-part
data-testid ("team-dialog"); update its data-testid to the required three-part
format "entity-element-qualifier" (apply to the DialogContent component
reference) so it matches the project's E2E convention—for example use a value
with entity "team", element "dialog" and an appropriate qualifier segment
(replace the existing data-testid on DialogContent accordingly).
---
Outside diff comments:
In `@ui/app/workspace/config/views/pricingConfigView.tsx`:
- Around line 97-153: The pricing sync interval input is missing a test hook;
add a data-testid to the Input for the "pricing_sync_interval_hours" field (the
Input with id="pricing-sync-interval" / register("pricing_sync_interval_hours"))
so automated tests can target it (e.g.,
data-testid="pricing-sync-interval-input"); ensure the attribute is present on
the same Input element that uses register("pricing_sync_interval_hours") and
that any tests referencing pricing-sync-interval are updated accordingly.
---
Duplicate comments:
In `@tests/e2e/features/governance/pages/governance.page.ts`:
- Around line 191-194: The editCustomer flow is using label-based selectors for
the customer name; replace the two calls to this.page.getByLabel('Customer
Name').clear() and .fill(...) with the page object’s test-id locator
this.customerNameInput.clear() and this.customerNameInput.fill(updates.name) so
the page object’s customerNameInput (test-id) is the primary selector; update
only the branches where updates.name is present to use customerNameInput to keep
selector consistency.
---
Nitpick comments:
In `@tests/e2e/features/governance/governance.spec.ts`:
- Around line 5-8: The spec uses shared mutable arrays createdTeams and
createdCustomers which makes it unsafe to run tests in parallel; update the
suite to run serially by changing the suite declaration for "Governance - Teams"
to a serialized describe (use test.describe.serial or equivalent) so tests
execute one worker and avoid concurrent access to createdTeams/createdCustomers
(also apply the same serialization to the other suite instance referenced near
the other occurrence of createdTeams/createdCustomers).
In `@tests/e2e/features/mcp-registry/mcp-registry.spec.ts`:
- Around line 281-284: The test currently repeats the allowed status array
inline; extract that array into a single exported constant (e.g.,
VALID_CLIENT_STATUSES) and use it in assertions instead of the inline
['connected','disconnected','connecting','error']; update the spec that calls
mcpRegistryPage.getClientStatus to import or reference VALID_CLIENT_STATUSES and
replace expect([...]).toContain(status?.toLowerCase()) with
expect(VALID_CLIENT_STATUSES).toContain(status?.toLowerCase()); place the
constant in a shared test utils module or at the top of this spec so other tests
can import it to avoid duplication.
In `@tests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.ts`:
- Line 11: The URL assertion in the test uses a loose regex (/mcp-tool-groups/)
which can match unintended routes; update the assertion on
mcpToolGroupsPage.page in
tests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.ts to use a stricter
matcher that anchors the expected route (for example by matching the exact path
or start/end anchors such as /^\/mcp-tool-groups\/?$/ or a string equality
check) so the test only passes when the page is exactly the mcp-tool-groups
route.
In `@tests/e2e/features/observability/observability.spec.ts`:
- Around line 241-255: The BigQuery Connector e2e test only verifies selection;
add tests mirroring other connectors to cover toggle and configuration
visibility: expand the test.describe('BigQuery Connector') block to, after
observabilityPage.selectConnector('bigquery'), assert toggle state via
observabilityPage.isConnectorToggled('bigquery') or toggle it with
observabilityPage.toggleConnector('bigquery') and check visibility of
configuration fields with observabilityPage.isConnectorConfigVisible('bigquery')
(or equivalent helper methods used by other connector tests) and assert expected
behavior when toggled on/off.
- Around line 21-22: The test uses a fragile CSS selector in
observability.spec.ts
(observabilityPage.page.locator('.text-muted-foreground').filter({ hasText:
'Providers' }).first() assigned to providersHeader); change the app markup to
add a stable data-testid (e.g., data-testid="sidebar-providers-header") on the
Providers sidebar header and update the test to locate by that test-id
(observabilityPage.page.locator('[data-testid="sidebar-providers-header"]') or
equivalent) before asserting visibility so the test no longer depends on styling
classes.
In `@tests/e2e/features/observability/pages/observability.page.ts`:
- Line 1: Remove the unused "expect" import from the import statement in the
Observability page object; update the import declaration that currently reads
"import { Page, Locator, expect } from '@playwright/test'" to only import the
used symbols (Page and Locator) so assertions remain in spec files and the page
object doesn't include unused dependencies.
In `@tests/e2e/features/placeholders/placeholders.spec.ts`:
- Around line 6-7: Replace the brittle text/role selector usage with the
test-id-first strategy: for each occurrence of page.getByText(/Prompt repository
is coming soon/i) (and the similar checks at lines referenced 12-14 and 59-61),
swap the primary lookup to page.getByTestId('<stable-test-id>') (use the
existing stable test id for the prompt repository element) and then keep the
visible/text check as a secondary assertion (e.g., assert the element is visible
and that its text matches via toHaveText or expect(...).toContainText). Update
the three places to use page.getByTestId(...) as the primary selector and keep
the regex text assertions only as intent verification.
In `@tests/e2e/features/virtual-keys/virtual-keys.spec.ts`:
- Around line 121-122: Replace direct locators in the spec that use
page.locator('#budgetMaxLimit') and the token input with page-object getters on
virtualKeysPage; add getters named budgetMaxLimitInput and tokenMaxLimitInput in
tests/e2e/features/virtual-keys/pages/virtual-keys.page.ts that return
this.page.getByTestId('vk-budget-max-limit-input') and
this.page.getByTestId('vk-token-max-limit-input') respectively, then update the
spec to use virtualKeysPage.budgetMaxLimitInput and
virtualKeysPage.tokenMaxLimitInput when asserting values against SAMPLE_BUDGETS.
In `@ui/app/workspace/governance/views/teamsTable.tsx`:
- Line 166: Replace fragile test hooks that use team.name with an immutable
identifier: change the TableRow data-testid and any other test-id usages (e.g.,
the occurrences around the TableRow at lines showing team usage and the other
occurrences near 316 and 328) to use team.id (or another stable unique field)
instead of team.name; update the template strings that build `data-testid` to
reference `team.id` (ensure it's stringified if necessary) so test selectors no
longer depend on mutable/display names.
ℹ️ Review info
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (45)
tests/e2e/core/fixtures/base.fixture.tstests/e2e/features/config/config.spec.tstests/e2e/features/config/pages/config-settings.page.tstests/e2e/features/dashboard/dashboard.spec.tstests/e2e/features/governance/governance.data.tstests/e2e/features/governance/governance.spec.tstests/e2e/features/governance/pages/governance.page.tstests/e2e/features/mcp-auth-config/mcp-auth-config.spec.tstests/e2e/features/mcp-auth-config/pages/mcp-auth-config.page.tstests/e2e/features/mcp-registry/mcp-registry.spec.tstests/e2e/features/mcp-registry/pages/mcp-registry.page.tstests/e2e/features/mcp-settings/mcp-settings.spec.tstests/e2e/features/mcp-settings/pages/mcp-settings.page.tstests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.tstests/e2e/features/mcp-tool-groups/pages/mcp-tool-groups.page.tstests/e2e/features/model-limits/model-limits.data.tstests/e2e/features/model-limits/model-limits.spec.tstests/e2e/features/model-limits/pages/model-limits.page.tstests/e2e/features/observability/observability.spec.tstests/e2e/features/observability/pages/observability.page.tstests/e2e/features/placeholders/placeholders.spec.tstests/e2e/features/plugins/plugins.spec.tstests/e2e/features/providers/pages/providers.page.tstests/e2e/features/providers/providers.spec.tstests/e2e/features/routing-rules/pages/routing-rules.page.tstests/e2e/features/routing-rules/routing-rules.spec.tstests/e2e/features/virtual-keys/pages/virtual-keys.page.tstests/e2e/features/virtual-keys/virtual-keys.spec.tsui/app/workspace/config/views/clientSettingsView.tsxui/app/workspace/config/views/mcpView.tsxui/app/workspace/config/views/pricingConfigView.tsxui/app/workspace/config/views/securityView.tsxui/app/workspace/governance/views/customerDialog.tsxui/app/workspace/governance/views/teamDialog.tsxui/app/workspace/governance/views/teamsTable.tsxui/app/workspace/mcp-registry/views/mcpClientsTable.tsxui/app/workspace/mcp-registry/views/mcpServersEmptyState.tsxui/app/workspace/model-limits/views/modelLimitSheet.tsxui/app/workspace/model-limits/views/modelLimitsEmptyState.tsxui/app/workspace/model-limits/views/modelLimitsTable.tsxui/app/workspace/plugins/fragments/pluginFormFragments.tsxui/app/workspace/providers/views/modelProviderKeysTableView.tsxui/app/workspace/providers/views/providersEmptyState.tsxui/app/workspace/virtual-keys/views/virtualKeysTable.tsxui/components/ui/numberAndSelect.tsx
🚧 Files skipped from review as they are similar to previous changes (22)
- ui/app/workspace/config/views/clientSettingsView.tsx
- tests/e2e/features/mcp-tool-groups/pages/mcp-tool-groups.page.ts
- ui/app/workspace/governance/views/customerDialog.tsx
- ui/app/workspace/mcp-registry/views/mcpServersEmptyState.tsx
- tests/e2e/features/governance/governance.data.ts
- ui/app/workspace/model-limits/views/modelLimitSheet.tsx
- tests/e2e/features/virtual-keys/pages/virtual-keys.page.ts
- tests/e2e/features/model-limits/model-limits.spec.ts
- tests/e2e/features/plugins/plugins.spec.ts
- ui/app/workspace/providers/views/providersEmptyState.tsx
- ui/app/workspace/config/views/mcpView.tsx
- ui/app/workspace/model-limits/views/modelLimitsTable.tsx
- tests/e2e/features/model-limits/pages/model-limits.page.ts
- tests/e2e/features/model-limits/model-limits.data.ts
- tests/e2e/features/mcp-auth-config/mcp-auth-config.spec.ts
- ui/app/workspace/config/views/securityView.tsx
- tests/e2e/core/fixtures/base.fixture.ts
- tests/e2e/features/mcp-settings/mcp-settings.spec.ts
- ui/app/workspace/model-limits/views/modelLimitsEmptyState.tsx
- tests/e2e/features/mcp-auth-config/pages/mcp-auth-config.page.ts
- ui/app/workspace/plugins/fragments/pluginFormFragments.tsx
- tests/e2e/features/routing-rules/pages/routing-rules.page.ts
Radheshg04
force-pushed
the
02-27-feat_extend_e2e_ui_tests
branch
from
bd26611 to
69a2041
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
ui/app/workspace/config/views/mcpView.tsx (1)
111-227:⚠️ Potential issue | 🟠 MajorAdd
data-testidto the remaining interactive controls in this workspace view.Line 164 (
mcp-tool-sync-intervalinput) and Line 184 (SelectTriggerfor binding level) are interactive but still missingdata-testid. This leaves the page partially instrumented for E2E.Proposed patch
<Input id="mcp-tool-sync-interval" + data-testid="mcp-tool-sync-interval-input" type="number" className="w-24" value={localValues.mcp_tool_sync_interval} onChange={(e) => handleToolSyncIntervalChange(e.target.value)} min="0" /> @@ <Select value={localValues.mcp_code_mode_binding_level} onValueChange={handleCodeModeBindingLevelChange}> - <SelectTrigger id="mcp-binding-level" className="w-56"> + <SelectTrigger id="mcp-binding-level" className="w-56" data-testid="mcp-binding-level-select-trigger"> <SelectValue placeholder="Select binding level" /> </SelectTrigger> <SelectContent> - <SelectItem value="server">Server-Level</SelectItem> - <SelectItem value="tool">Tool-Level</SelectItem> + <SelectItem value="server" data-testid="mcp-binding-level-server-option">Server-Level</SelectItem> + <SelectItem value="tool" data-testid="mcp-binding-level-tool-option">Tool-Level</SelectItem> </SelectContent> </Select>As per coding guidelines,
ui/app/workspace/**/*.tsxmust includedata-testidattributes on all interactive elements.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/config/views/mcpView.tsx` around lines 111 - 227, Add missing data-testid attributes to the interactive controls: add data-testid="mcp-tool-sync-interval-input" to the Input with id "mcp-tool-sync-interval" and add data-testid="mcp-binding-level-trigger" to the SelectTrigger with id "mcp-binding-level"; update the handlers/use of localValues (handleToolSyncIntervalChange, handleCodeModeBindingLevelChange) remain unchanged—just add the attributes so all interactive elements in this view are instrumented for E2E tests.
♻️ Duplicate comments (3)
tests/e2e/features/governance/pages/governance.page.ts (1)
6-15:⚠️ Potential issue | 🟠 Major
TeamConfig/CustomerConfigexpose inputs that are currently ignored.
budget.resetDurationandrateLimit.*are declared but not applied in create/edit flows, so caller intent is silently dropped. Please either wire these fields into the form interactions or remove them from the interfaces for now to avoid a misleading API.Suggested direction (narrow interface until supported)
export interface TeamConfig { name: string customerId?: string - budget?: { maxLimit: number; resetDuration?: string } - rateLimit?: { - tokenMaxLimit?: number - tokenResetDuration?: string - requestMaxLimit?: number - requestResetDuration?: string - } + budget?: { maxLimit: number } } export interface CustomerConfig { name: string - budget?: { maxLimit: number; resetDuration?: string } - rateLimit?: { - tokenMaxLimit?: number - tokenResetDuration?: string - requestMaxLimit?: number - requestResetDuration?: string - } + budget?: { maxLimit: number } }Also applies to: 21-27, 75-101, 128-144, 157-200
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/governance/pages/governance.page.ts` around lines 6 - 15, The TeamConfig and CustomerConfig interfaces declare budget.resetDuration and the rateLimit.* fields but those values are never used in the create/edit flows, which silently drops caller intent; either wire these fields into the form handling code that builds payloads for create/edit operations or remove them from the interfaces to narrow the public API. Locate the TeamConfig and CustomerConfig interface declarations (symbols: TeamConfig, CustomerConfig) and then either (A) add the missing plumbing in the createTeam/editTeam and createCustomer/editCustomer code paths so budget.resetDuration and rateLimit.{tokenMaxLimit,tokenResetDuration,requestMaxLimit,requestResetDuration} are read from the form/state and included in the outgoing payloads, or (B) remove those unused properties from the interfaces and any form bindings so the types reflect only supported inputs. Ensure consistency across all occurrences referenced in the review (the other interface blocks mirroring these fields) so the interfaces and form behavior stay in sync.tests/e2e/features/observability/pages/observability.page.ts (1)
52-55:⚠️ Potential issue | 🟠 MajorRemove nondeterministic toggle fallback for unmapped connectors.
Line 54 falls back to the first switch on the page, which can target the wrong connector and produce false state reads/toggles.
🔧 Suggested fix
getConnectorToggle(connector: ObservabilityConnector): Locator { const testId = ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS[connector] - return testId ? this.page.getByTestId(testId) : this.page.locator('button[role="switch"]').first() + if (!testId) { + throw new Error(`Missing connector toggle test id mapping for: ${connector}`) + } + return this.page.getByTestId(testId) }As per coding guidelines, "
tests/e2e/**/*.ts: E2E test page objects must ... use getByTestId() as the primary selector strategy."🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/observability/pages/observability.page.ts` around lines 52 - 55, The getConnectorToggle function currently falls back to the first switch when a connector key is not in ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS, causing nondeterministic selection; update getConnectorToggle (and any callers if needed) to use only this mapping and fail loudly when a testId is missing — e.g., remove the fallback branch and throw or assert with a clear message that ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS is missing an entry for the provided ObservabilityConnector so tests must add the proper testId before using getConnectorToggle.tests/e2e/features/providers/providers.spec.ts (1)
768-807:⚠️ Potential issue | 🟠 MajorGuarantee pricing-override restoration with
try/finally.Line 800 restoration currently runs only on the happy path. If the test fails/skips after mutation, workspace config can leak to subsequent tests.
🔧 Suggested fix
test('should save valid pricing overrides JSON', async ({ providersPage }) => { @@ - await providersPage.setPricingOverrides(validOverrides) - - const isSaveEnabled = await providersPage.pricingOverridesSaveBtn.isDisabled().then((d) => !d) - if (!isSaveEnabled) { - test.skip(true, 'Save button disabled (RBAC or no changes detected)') - return - } - - await providersPage.savePricingOverrides() - await providersPage.dismissToasts() - - // Restore original value for test isolation - await providersPage.setPricingOverrides(initialValue) - const restoreEnabled = await providersPage.pricingOverridesSaveBtn.isDisabled().then((d) => !d) - if (restoreEnabled) { - await providersPage.savePricingOverrides() - await providersPage.dismissToasts() - } + try { + await providersPage.setPricingOverrides(validOverrides) + + const isSaveEnabled = await providersPage.pricingOverridesSaveBtn.isDisabled().then((d) => !d) + if (!isSaveEnabled) { + test.skip(true, 'Save button disabled (RBAC or no changes detected)') + return + } + + await providersPage.savePricingOverrides() + await providersPage.dismissToasts() + } finally { + // Always restore original value for isolation + await providersPage.setPricingOverrides(initialValue) + const restoreEnabled = await providersPage.pricingOverridesSaveBtn.isDisabled().then((d) => !d) + if (restoreEnabled) { + await providersPage.savePricingOverrides() + await providersPage.dismissToasts() + } + } })🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/providers/providers.spec.ts` around lines 768 - 807, The test "should save valid pricing overrides JSON" mutates workspace config but only restores the original value on the happy path; wrap the mutation/assertion block in a try/finally so the original initialValue is always restored even on failure or skip. Concretely: capture initialValue via providersPage.pricingOverridesJsonInput.inputValue(), then perform setPricingOverrides(validOverrides), save/dismiss assertions using providersPage.setPricingOverrides, providersPage.savePricingOverrides, providersPage.dismissToasts inside the try, and in the finally always call providersPage.setPricingOverrides(initialValue) and, if the save button (providersPage.pricingOverridesSaveBtn) is enabled, call providersPage.savePricingOverrides() and providersPage.dismissToasts() to ensure cleanup.
🧹 Nitpick comments (3)
tests/e2e/features/mcp-settings/mcp-settings.spec.ts (1)
12-15: Use page object locators instead of directgetByLabel()calls.The test directly accesses
mcpSettingsPage.page.getByLabel(...)instead of using the locators already defined in theMCPSettingsPagepage object (maxAgentDepthInput,toolTimeoutInput). This bypasses the page object pattern and does not follow thegetByTestId()as primary selector strategy guideline.♻️ Proposed fix to use page object locators
test('should display MCP settings form fields', async ({ mcpSettingsPage }) => { - await expect(mcpSettingsPage.page.getByLabel('Max Agent Depth')).toBeVisible() - await expect(mcpSettingsPage.page.getByLabel('Tool Execution Timeout (seconds)')).toBeVisible() + await expect(mcpSettingsPage.maxAgentDepthInput).toBeVisible() + await expect(mcpSettingsPage.toolTimeoutInput).toBeVisible() })As per coding guidelines: "E2E test page objects must... use getByTestId() as the primary selector strategy".
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/mcp-settings/mcp-settings.spec.ts` around lines 12 - 15, The test is calling mcpSettingsPage.page.getByLabel(...) directly instead of using the MCPSettingsPage page object locators; update the assertions to use the page object's locators (maxAgentDepthInput and toolTimeoutInput) and assert visibility on those (e.g., await expect(mcpSettingsPage.maxAgentDepthInput).toBeVisible()), ensuring the page object uses getByTestId() as the primary selector strategy for those locators.tests/e2e/features/mcp-auth-config/mcp-auth-config.spec.ts (1)
11-11: Consider tightening the URL matcher
/mcp-auth-config/is broad and can match unintended URLs. Prefer anchoring to the expected workspace route for stronger routing verification.Suggested assertion tweak
- await expect(mcpAuthConfigPage.page).toHaveURL(/mcp-auth-config/) + await expect(mcpAuthConfigPage.page).toHaveURL(/\/workspace\/mcp-auth-config\/?$/)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/mcp-auth-config/mcp-auth-config.spec.ts` at line 11, The test uses a loose regex in the assertion await expect(mcpAuthConfigPage.page).toHaveURL(/mcp-auth-config/) which may match unintended routes; tighten it by changing the matcher on the toHaveURL call to an anchored pattern or exact path for the expected workspace route (for example use a full path string or a regex like /^\/workspaces\/[^\/]+\/mcp-auth-config\/?$/) so update the assertion on mcpAuthConfigPage.page.toHaveURL to the anchored/explicit matcher that reflects the intended workspace URL structure.tests/e2e/features/providers/providers.spec.ts (1)
368-377: Replace tautological assertion with a behavioral assertion.Line 374 (
expect(typeof modelsVisible).toBe('boolean')) is always true and doesn’t validate UI behavior. Prefer asserting a concrete expected state (visible section or explicit absence condition).🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/providers/providers.spec.ts` around lines 368 - 377, The current typeof check on modelsVisible is tautological; remove the line `expect(typeof modelsVisible).toBe('boolean')` and replace it with a concrete behavioral assertion: after computing modelsVisible from `modelsSection.isVisible()`, if modelsVisible is true assert `await expect(modelsSection).toBeVisible()`, otherwise assert `await expect(modelsSection).not.toBeVisible()` (or an explicit absence check such as verifying the locator count is 0) so the test verifies actual UI behavior using the `modelsSection` locator and the `modelsVisible` boolean.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/e2e/features/governance/pages/governance.page.ts`:
- Around line 191-194: Replace the non-deterministic label-based locators in the
updates.name block with the page object's test-id locator: instead of
this.page.getByLabel('Customer Name').clear() and .fill(updates.name), use
this.page.getByTestId('customer-name-input').clear() and
this.page.getByTestId('customer-name-input').fill(updates.name) so the customer
name edits rely on the dedicated customer-name-input test-id locator.
In `@tests/e2e/features/routing-rules/pages/routing-rules.page.ts`:
- Around line 593-597: The getRuleDescription function currently reads
textContent() of the entire first cell (function getRuleDescription -> const
firstCell = row.locator('td').first()), which returns both name and description;
change it to target the description span specifically: either update the UI to
add data-testid="routing-rule-description" to the description <span> and then
use getByTestId()/locator('[data-testid="routing-rule-description"]') on the row
to return only that span's textContent, or if you cannot add a test-id, select
the second span inside the first cell (e.g., firstCell.locator('span').nth(1))
or split the cell text by lines and return the description portion; update
getRuleDescription to use that targeted locator instead of the whole cell.
In `@ui/app/workspace/model-limits/views/modelLimitsEmptyState.tsx`:
- Line 40: The data-testid on the create button doesn't follow the required
"<entity>-<element>-<qualifier>" pattern; rename the attribute in
ModelLimitsEmptyState (the JSX element with
data-testid="create-model-limit-btn") to follow the convention (e.g.,
data-testid="model-limits-button-create"), update any imports/uses within that
component, and search and update all tests/e2e/ references to the old id so they
point to the new data-testid.
---
Outside diff comments:
In `@ui/app/workspace/config/views/mcpView.tsx`:
- Around line 111-227: Add missing data-testid attributes to the interactive
controls: add data-testid="mcp-tool-sync-interval-input" to the Input with id
"mcp-tool-sync-interval" and add data-testid="mcp-binding-level-trigger" to the
SelectTrigger with id "mcp-binding-level"; update the handlers/use of
localValues (handleToolSyncIntervalChange, handleCodeModeBindingLevelChange)
remain unchanged—just add the attributes so all interactive elements in this
view are instrumented for E2E tests.
---
Duplicate comments:
In `@tests/e2e/features/governance/pages/governance.page.ts`:
- Around line 6-15: The TeamConfig and CustomerConfig interfaces declare
budget.resetDuration and the rateLimit.* fields but those values are never used
in the create/edit flows, which silently drops caller intent; either wire these
fields into the form handling code that builds payloads for create/edit
operations or remove them from the interfaces to narrow the public API. Locate
the TeamConfig and CustomerConfig interface declarations (symbols: TeamConfig,
CustomerConfig) and then either (A) add the missing plumbing in the
createTeam/editTeam and createCustomer/editCustomer code paths so
budget.resetDuration and
rateLimit.{tokenMaxLimit,tokenResetDuration,requestMaxLimit,requestResetDuration}
are read from the form/state and included in the outgoing payloads, or (B)
remove those unused properties from the interfaces and any form bindings so the
types reflect only supported inputs. Ensure consistency across all occurrences
referenced in the review (the other interface blocks mirroring these fields) so
the interfaces and form behavior stay in sync.
In `@tests/e2e/features/observability/pages/observability.page.ts`:
- Around line 52-55: The getConnectorToggle function currently falls back to the
first switch when a connector key is not in
ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS, causing nondeterministic selection;
update getConnectorToggle (and any callers if needed) to use only this mapping
and fail loudly when a testId is missing — e.g., remove the fallback branch and
throw or assert with a clear message that
ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS is missing an entry for the provided
ObservabilityConnector so tests must add the proper testId before using
getConnectorToggle.
In `@tests/e2e/features/providers/providers.spec.ts`:
- Around line 768-807: The test "should save valid pricing overrides JSON"
mutates workspace config but only restores the original value on the happy path;
wrap the mutation/assertion block in a try/finally so the original initialValue
is always restored even on failure or skip. Concretely: capture initialValue via
providersPage.pricingOverridesJsonInput.inputValue(), then perform
setPricingOverrides(validOverrides), save/dismiss assertions using
providersPage.setPricingOverrides, providersPage.savePricingOverrides,
providersPage.dismissToasts inside the try, and in the finally always call
providersPage.setPricingOverrides(initialValue) and, if the save button
(providersPage.pricingOverridesSaveBtn) is enabled, call
providersPage.savePricingOverrides() and providersPage.dismissToasts() to ensure
cleanup.
---
Nitpick comments:
In `@tests/e2e/features/mcp-auth-config/mcp-auth-config.spec.ts`:
- Line 11: The test uses a loose regex in the assertion await
expect(mcpAuthConfigPage.page).toHaveURL(/mcp-auth-config/) which may match
unintended routes; tighten it by changing the matcher on the toHaveURL call to
an anchored pattern or exact path for the expected workspace route (for example
use a full path string or a regex like
/^\/workspaces\/[^\/]+\/mcp-auth-config\/?$/) so update the assertion on
mcpAuthConfigPage.page.toHaveURL to the anchored/explicit matcher that reflects
the intended workspace URL structure.
In `@tests/e2e/features/mcp-settings/mcp-settings.spec.ts`:
- Around line 12-15: The test is calling mcpSettingsPage.page.getByLabel(...)
directly instead of using the MCPSettingsPage page object locators; update the
assertions to use the page object's locators (maxAgentDepthInput and
toolTimeoutInput) and assert visibility on those (e.g., await
expect(mcpSettingsPage.maxAgentDepthInput).toBeVisible()), ensuring the page
object uses getByTestId() as the primary selector strategy for those locators.
In `@tests/e2e/features/providers/providers.spec.ts`:
- Around line 368-377: The current typeof check on modelsVisible is
tautological; remove the line `expect(typeof modelsVisible).toBe('boolean')` and
replace it with a concrete behavioral assertion: after computing modelsVisible
from `modelsSection.isVisible()`, if modelsVisible is true assert `await
expect(modelsSection).toBeVisible()`, otherwise assert `await
expect(modelsSection).not.toBeVisible()` (or an explicit absence check such as
verifying the locator count is 0) so the test verifies actual UI behavior using
the `modelsSection` locator and the `modelsVisible` boolean.
ℹ️ Review info
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (47)
.github/workflows/e2e-tests.ymltests/e2e/core/fixtures/base.fixture.tstests/e2e/features/config/config.spec.tstests/e2e/features/config/pages/config-settings.page.tstests/e2e/features/dashboard/dashboard.spec.tstests/e2e/features/governance/governance.data.tstests/e2e/features/governance/governance.spec.tstests/e2e/features/governance/pages/governance.page.tstests/e2e/features/mcp-auth-config/mcp-auth-config.spec.tstests/e2e/features/mcp-auth-config/pages/mcp-auth-config.page.tstests/e2e/features/mcp-registry/mcp-registry.spec.tstests/e2e/features/mcp-registry/pages/mcp-registry.page.tstests/e2e/features/mcp-settings/mcp-settings.spec.tstests/e2e/features/mcp-settings/pages/mcp-settings.page.tstests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.tstests/e2e/features/mcp-tool-groups/pages/mcp-tool-groups.page.tstests/e2e/features/model-limits/model-limits.data.tstests/e2e/features/model-limits/model-limits.spec.tstests/e2e/features/model-limits/pages/model-limits.page.tstests/e2e/features/observability/observability.spec.tstests/e2e/features/observability/pages/observability.page.tstests/e2e/features/placeholders/placeholders.spec.tstests/e2e/features/plugins/plugins.spec.tstests/e2e/features/providers/pages/providers.page.tstests/e2e/features/providers/providers.spec.tstests/e2e/features/routing-rules/pages/routing-rules.page.tstests/e2e/features/routing-rules/routing-rules.spec.tstests/e2e/features/virtual-keys/pages/virtual-keys.page.tstests/e2e/features/virtual-keys/virtual-keys.spec.tsui/app/workspace/config/views/clientSettingsView.tsxui/app/workspace/config/views/mcpView.tsxui/app/workspace/config/views/pricingConfigView.tsxui/app/workspace/config/views/securityView.tsxui/app/workspace/governance/views/customerDialog.tsxui/app/workspace/governance/views/customerTable.tsxui/app/workspace/governance/views/teamDialog.tsxui/app/workspace/governance/views/teamsTable.tsxui/app/workspace/mcp-registry/views/mcpClientsTable.tsxui/app/workspace/mcp-registry/views/mcpServersEmptyState.tsxui/app/workspace/model-limits/views/modelLimitSheet.tsxui/app/workspace/model-limits/views/modelLimitsEmptyState.tsxui/app/workspace/model-limits/views/modelLimitsTable.tsxui/app/workspace/plugins/fragments/pluginFormFragments.tsxui/app/workspace/providers/views/modelProviderKeysTableView.tsxui/app/workspace/providers/views/providersEmptyState.tsxui/app/workspace/virtual-keys/views/virtualKeysTable.tsxui/components/ui/numberAndSelect.tsx
✅ Files skipped from review due to trivial changes (1)
- .github/workflows/e2e-tests.yml
🚧 Files skipped from review as they are similar to previous changes (24)
- ui/app/workspace/governance/views/teamsTable.tsx
- tests/e2e/features/mcp-auth-config/pages/mcp-auth-config.page.ts
- tests/e2e/features/routing-rules/routing-rules.spec.ts
- ui/app/workspace/governance/views/teamDialog.tsx
- tests/e2e/core/fixtures/base.fixture.ts
- tests/e2e/features/mcp-settings/pages/mcp-settings.page.ts
- tests/e2e/features/governance/governance.data.ts
- ui/app/workspace/mcp-registry/views/mcpServersEmptyState.tsx
- tests/e2e/features/model-limits/pages/model-limits.page.ts
- tests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.ts
- tests/e2e/features/model-limits/model-limits.spec.ts
- ui/app/workspace/virtual-keys/views/virtualKeysTable.tsx
- ui/app/workspace/model-limits/views/modelLimitSheet.tsx
- ui/app/workspace/mcp-registry/views/mcpClientsTable.tsx
- tests/e2e/features/mcp-registry/pages/mcp-registry.page.ts
- ui/app/workspace/model-limits/views/modelLimitsTable.tsx
- ui/app/workspace/governance/views/customerDialog.tsx
- tests/e2e/features/virtual-keys/virtual-keys.spec.ts
- ui/components/ui/numberAndSelect.tsx
- ui/app/workspace/config/views/pricingConfigView.tsx
- tests/e2e/features/mcp-tool-groups/pages/mcp-tool-groups.page.ts
- ui/app/workspace/plugins/fragments/pluginFormFragments.tsx
- tests/e2e/features/mcp-registry/mcp-registry.spec.ts
- tests/e2e/features/placeholders/placeholders.spec.ts
Radheshg04
force-pushed
the
02-27-feat_extend_e2e_ui_tests
branch
3 times, most recently
from
6672cda to
b47e66b
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 10
♻️ Duplicate comments (3)
tests/e2e/features/config/config.spec.ts (1)
125-133:⚠️ Potential issue | 🟠 MajorGuard editability/save before filling invalid URL.
Line 126 fills the input before RBAC/read-only checks. In restricted environments this can fail before
test.skip(...)is reached.🔧 Suggested fix
test('should validate URL format', async ({ configSettingsPage }) => { - await configSettingsPage.pricingDatasheetUrlInput.fill('invalid-url-no-http') - const canSave = await configSettingsPage.pricingSaveBtn.isDisabled().then((d) => !d) - if (!canSave) { - test.skip(true, 'Save button disabled (RBAC)') + const canEdit = await configSettingsPage.pricingDatasheetUrlInput.isEditable().catch(() => false) + const canSave = await configSettingsPage.pricingSaveBtn.isDisabled().then((d) => !d) + if (!canEdit || !canSave) { + test.skip(true, 'Pricing URL validation unavailable (RBAC/read-only)') return } + await configSettingsPage.pricingDatasheetUrlInput.fill('invalid-url-no-http') await configSettingsPage.pricingSaveBtn.click() await expect(configSettingsPage.page.getByText(/URL must start with http|valid URL/i)).toBeVisible() })🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/config/config.spec.ts` around lines 125 - 133, The test fills pricingDatasheetUrlInput before checking save/editability which can fail under RBAC; instead, first determine editability by checking pricingSaveBtn (or pricingDatasheetUrlInput.isEnabled()) and call test.skip(...) if not editable, then perform pricingDatasheetUrlInput.fill('invalid-url-no-http') and proceed to click pricingSaveBtn; update the test block around configSettingsPage.pricingDatasheetUrlInput and configSettingsPage.pricingSaveBtn so the RBAC guard runs before any input.fill operations.tests/e2e/features/observability/pages/observability.page.ts (1)
52-63:⚠️ Potential issue | 🟠 MajorFallback toggle/delete locators are still nondeterministic and can mask missing coverage.
For unmapped connectors,
button[role="switch"]').first()can target the wrong control, and the no-match delete locator silently hides missing test IDs. This can corruptgetCurrentState()anddisableAllConnectors()behavior for non-OTel/Prometheus connectors.Suggested hard-fail fix
getConnectorToggle(connector: ObservabilityConnector): Locator { const testId = ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS[connector] - return testId ? this.page.getByTestId(testId) : this.page.locator('button[role="switch"]').first() + if (!testId) { + throw new Error(`Missing connector toggle testid mapping for: ${connector}`) + } + return this.getByTestId(testId) } getConnectorDeleteBtn(connector: ObservabilityConnector): Locator { const testId = ObservabilityPage.CONNECTOR_DELETE_TESTIDS[connector] - return testId ? this.page.getByTestId(testId) : this.page.locator('[data-testid="connector-delete-unused"]') + if (!testId) { + throw new Error(`Missing connector delete-button testid mapping for: ${connector}`) + } + return this.getByTestId(testId) }As per coding guidelines "
tests/e2e/**/*.ts: E2E test page objects must extend BasePage, use getByTestId() as the primary selector strategy, and import test/expect from ../../core/fixtures/base.fixture, never from `@playwright/test``."🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/observability/pages/observability.page.ts` around lines 52 - 63, The fallback locators in getConnectorToggle and getConnectorDeleteBtn make tests nondeterministic for unmapped connectors; instead of returning a broad or no-match locator, change both functions to hard-fail when the connector key is not present in ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS / ObservabilityPage.CONNECTOR_DELETE_TESTIDS by throwing a descriptive Error that names the missing connector and the expected testId; this forces callers (e.g., getCurrentState, disableAllConnectors) to only use mapped connectors and keeps selectors exclusively via getByTestId.tests/e2e/features/providers/providers.spec.ts (1)
779-809:⚠️ Potential issue | 🟠 MajorGuarantee pricing-override restoration with
finally.At Line 803+, restoration runs only on the happy path. If save/dismiss fails earlier, workspace-level overrides can leak into later tests in the stack.
🔧 Suggested fix
await providersPage.selectConfigTab('governance') const initialValue = await providersPage.pricingOverridesJsonInput.inputValue() @@ - await providersPage.setPricingOverrides(validOverrides) - - const isSaveEnabled = await providersPage.pricingOverridesSaveBtn.isDisabled().then((d) => !d) - if (!isSaveEnabled) { - test.skip(true, 'Save button disabled (RBAC or no changes detected)') - return - } - - await providersPage.savePricingOverrides() - await providersPage.dismissToasts() - - // Restore original value for test isolation - await providersPage.setPricingOverrides(initialValue) - const restoreEnabled = await providersPage.pricingOverridesSaveBtn.isDisabled().then((d) => !d) - if (restoreEnabled) { - await providersPage.savePricingOverrides() - await providersPage.dismissToasts() - } + try { + await providersPage.setPricingOverrides(validOverrides) + + const isSaveEnabled = await providersPage.pricingOverridesSaveBtn.isDisabled().then((d) => !d) + if (!isSaveEnabled) { + test.skip(true, 'Save button disabled (RBAC or no changes detected)') + return + } + + await providersPage.savePricingOverrides() + await providersPage.dismissToasts() + } finally { + // Restore original value for test isolation + await providersPage.setPricingOverrides(initialValue) + const restoreEnabled = await providersPage.pricingOverridesSaveBtn.isDisabled().then((d) => !d) + if (restoreEnabled) { + await providersPage.savePricingOverrides() + await providersPage.dismissToasts() + } + }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/providers/providers.spec.ts` around lines 779 - 809, The test currently restores pricing overrides only on the happy path; wrap the mutation and save/dismiss steps in a try/finally so restoration always runs: perform providersPage.setPricingOverrides(validOverrides), check/save/dismiss inside a try block, and in finally read the original value (initialValue from providersPage.pricingOverridesJsonInput.inputValue()), call providersPage.setPricingOverrides(initialValue) and conditionally save/dismiss using providersPage.pricingOverridesSaveBtn and providersPage.savePricingOverrides()/providersPage.dismissToasts() to guarantee cleanup even if save or dismiss throws; reference providersPage.selectConfigTab, setPricingOverrides, pricingOverridesJsonInput, pricingOverridesSaveBtn, savePricingOverrides, and dismissToasts to locate the changes.
🧹 Nitpick comments (9)
ui/app/workspace/providers/fragments/debuggingFormFragment.tsx (1)
66-66: LGTM! The data-testid follows the naming convention.The
provider-config-debugging-contenttest ID correctly uses the<entity>-<element>-<qualifier>pattern.Since this PR extends e2e test coverage, consider also adding
data-testidattributes to the interactive elements within this form (the twoSwitchcomponents and the submitButton) for more granular test assertions. As per coding guidelines: "UI workspace pages... must include data-testid attributes on all interactive elements.",
♻️ Suggested data-testid additions for interactive elements
<Switch size="md" checked={field.value} disabled={!hasUpdateProviderAccess} + data-testid="provider-config-send-back-raw-request-switch" onCheckedChange={(checked) => { field.onChange(checked); form.trigger("send_back_raw_request"); }} /><Switch size="md" checked={field.value} disabled={!hasUpdateProviderAccess} + data-testid="provider-config-send-back-raw-response-switch" onCheckedChange={(checked) => { field.onChange(checked); form.trigger("send_back_raw_response"); }} /><Button type="submit" disabled={!form.formState.isDirty || !form.formState.isValid || !hasUpdateProviderAccess || isUpdatingProvider} isLoading={isUpdatingProvider} + data-testid="provider-config-debugging-save-btn" > Save Debugging Configuration </Button>🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/providers/fragments/debuggingFormFragment.tsx` at line 66, Add data-testid attributes to the interactive controls inside the debugging form so e2e tests can target them: add data-testid="provider-config-debugging-switch-<name>" to each Switch (use the internal prop/name for each switch to replace <name>) and data-testid="provider-config-debugging-submit" to the form submit Button; update the Switch and Button elements referenced in this file (the Switch components and the submit Button used with form.handleSubmit/onSubmit) to include those attributes following the <entity>-<element>-<qualifier> pattern.tests/e2e/features/mcp-settings/mcp-settings.spec.ts (1)
12-15: Use page object locators instead of direct label selectors.The
MCPSettingsPagealready definesmaxAgentDepthInputandtoolTimeoutInputlocators (per the relevant code snippets). Using these would be more consistent with the test-id-first strategy and avoid bypassing the page object abstraction.Suggested change
test('should display MCP settings form fields', async ({ mcpSettingsPage }) => { - await expect(mcpSettingsPage.page.getByLabel('Max Agent Depth')).toBeVisible() - await expect(mcpSettingsPage.page.getByLabel('Tool Execution Timeout (seconds)')).toBeVisible() + await expect(mcpSettingsPage.maxAgentDepthInput).toBeVisible() + await expect(mcpSettingsPage.toolTimeoutInput).toBeVisible() })🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/mcp-settings/mcp-settings.spec.ts` around lines 12 - 15, Replace direct label-based selectors in the test block for 'should display MCP settings form fields' with the MCPSettingsPage's locators: use mcpSettingsPage.maxAgentDepthInput and mcpSettingsPage.toolTimeoutInput instead of mcpSettingsPage.page.getByLabel(...), then assert visibility on those locators (e.g., await expect(mcpSettingsPage.maxAgentDepthInput).toBeVisible()). This keeps the page-object abstraction and the test-id-first strategy intact while leaving the test name and flow unchanged.tests/e2e/features/model-limits/pages/model-limits.page.ts (2)
57-60: Use test-id selector for budget input instead of CSS ID.Lines 58 and 75 use
page.locator('#modelBudgetMaxLimit')which is a CSS ID selector. Per coding guidelines, page objects should usegetByTestId()as the primary selector strategy. Consider adding adata-testidattribute to the budget input in the UI and updating the locator.Suggested change
if (config.budget?.maxLimit !== undefined) { - const budgetInput = this.page.locator('#modelBudgetMaxLimit') + const budgetInput = this.page.getByTestId('model-budget-max-limit-input') await budgetInput.fill(String(config.budget.maxLimit)) }And similarly for
editModelLimit:if (updates.budget?.maxLimit !== undefined) { - const budgetInput = this.page.locator('#modelBudgetMaxLimit') + const budgetInput = this.page.getByTestId('model-budget-max-limit-input') await budgetInput.clear() await budgetInput.fill(String(updates.budget.maxLimit)) }As per coding guidelines: "E2E test page objects must... use getByTestId() as the primary selector strategy."
Also applies to: 74-78
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/model-limits/pages/model-limits.page.ts` around lines 57 - 60, Replace usage of CSS ID selectors with test-id selectors: update the locator in model-limits.page.ts that currently uses this.page.locator('#modelBudgetMaxLimit') to use this.page.getByTestId('modelBudgetMaxLimit') (and do the same for the editModelLimit-related locator around lines 74-78). Add a matching data-testid="modelBudgetMaxLimit" attribute to the budget input in the UI if it doesn't exist, and update any variables (budgetInput, editModelLimit) to reference the new getByTestId locators so the page object follows the project's getByTestId selector guideline.
48-55: Consider adding test-id selectors for provider combobox and model input.Line 49 uses
[role="combobox"]which could match multiple comboboxes if the sheet has more than one. Line 52 usesgetByPlaceholder()which is less stable than test-id selectors. Addingdata-testidattributes to these form elements inmodelLimitSheet.tsxwould improve test reliability.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/model-limits/pages/model-limits.page.ts` around lines 48 - 55, The tests use fragile generic selectors: fillSelect(this.page, '[role="combobox"]', ...) and this.sheet.getByPlaceholder(/Search for a model/i) which can match multiple elements; update the UI to add stable data-testid attributes on the provider combobox and the model search input in modelLimitSheet.tsx (e.g., data-testid="provider-combobox" and data-testid="model-search-input"), then update the test to use those testids (referencing the fillSelect call and the modelInput retrieval) so fillSelect targets the specific provider combobox and modelInput uses getByTestId instead of getByPlaceholder to improve reliability.tests/e2e/features/governance/pages/governance.page.ts (1)
103-110: Consider adding a test-id for the delete confirmation dialog.Line 107 uses
page.locator('[role="alertdialog"]')which can match any alert dialog on the page. While role-based selectors work for standard dialogs, adding a specificdata-testidto the delete confirmation dialog in the UI would make this more deterministic.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/governance/pages/governance.page.ts` around lines 103 - 110, The deleteTeam method uses a generic locator for the confirmation dialog which can match any alertdialog; update the UI to add a specific data-testid (e.g. data-testid="team-delete-dialog") on the delete confirmation dialog, then change deleteTeam to target that test id (use this.page.getByTestId('team-delete-dialog') or locator('[data-testid="team-delete-dialog"]') before clicking the Delete button) while keeping the existing team-delete-btn-{name} trigger and waitForSuccessToast call intact so the flow remains deterministic.tests/e2e/features/config/config.spec.ts (1)
393-406: Also guard toggle interaction on enabled state (not just visibility).A visible-but-disabled switch will still fail on click in some RBAC setups. Add an enabled/editable check before toggling.
🔧 Suggested hardening
test('should toggle enforce auth on inference', async ({ configSettingsPage }) => { const isVisible = await configSettingsPage.enforceAuthOnInferenceSwitch.isVisible().catch(() => false) - if (!isVisible) { - test.skip(true, 'Enforce auth on inference not available') + const isEnabled = await configSettingsPage.enforceAuthOnInferenceSwitch.isEnabled().catch(() => false) + if (!isVisible || !isEnabled) { + test.skip(true, 'Enforce auth on inference not available or not editable') return } const initialState = await configSettingsPage.getSwitchState(configSettingsPage.enforceAuthOnInferenceSwitch) await configSettingsPage.toggleEnforceAuthOnInference()🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/config/config.spec.ts` around lines 393 - 406, The test currently only checks visibility before interacting with enforceAuthOnInferenceSwitch, but a visible switch can be disabled and clicking it will fail; before calling getSwitchState/toggleEnforceAuthOnInference, check that configSettingsPage.enforceAuthOnInferenceSwitch.isEnabled().catch(() => false) and if false call test.skip(true, 'Enforce auth on inference not enabled for this role') and return; keep the existing calls to getSwitchState, toggleEnforceAuthOnInference, hasPendingChanges, and saveSettings unchanged otherwise.tests/e2e/features/plugins/pages/plugins.page.ts (1)
57-59: Prefer test-id locators for the primary readiness/count paths.Lines 57-59 and Lines 123-124 still route through
createBtn/pluginList, which are role/CSS/icon selectors. Since this page now has stable test IDs, switching primary checks togetByTestId()will make these flows less brittle.As per coding guidelines "
tests/e2e/**/*.ts: E2E test page objects must ... use getByTestId() as the primary selector strategy".Also applies to: 116-125
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/plugins/pages/plugins.page.ts` around lines 57 - 59, Replace brittle role/CSS/icon-based readiness and count checks that use createBtn and pluginList with test-id based locators: call this.page.getByTestId('plugins-empty-state') for the empty-state readiness path and this.page.getByTestId('plugins-list') (or the stable test-id for the list/count) for the list readiness/count path, and use the same chaining (.first().waitFor({...}) or .waitFor({...})) currently applied to createBtn/pluginList; update both usages that reference createBtn and pluginList so the primary selector is getByTestId() while preserving existing .or(), .first(), and .waitFor() logic.tests/e2e/features/observability/observability.spec.ts (2)
175-177: Prefer connector-id assertions over UI text labels for selected-tab checks.
toContain('Prometheus')/toContain('BigQuery')is copy-dependent. Using active tabdata-testid(or connector id) is more stable for E2E assertions.Also applies to: 252-254
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/observability/observability.spec.ts` around lines 175 - 177, Replace fragile UI-label assertions that rely on visible text in observabilityPage.getSelectedConnector() with checks against the connector's stable identifier/data-testid: update the test to retrieve the selected tab's data-testid or connector-id (via observabilityPage.getSelectedConnector() or add a new observabilityPage.getSelectedConnectorId() helper) and assert equality/containment against the expected id (e.g., "prometheus" or "bigquery") instead of asserting toContain('Prometheus'); apply the same change to the other assertion locations currently using toContain('BigQuery').
81-93: Delete-button tests can pass by skipping without establishing the “configured” precondition.Both tests skip when the delete button is absent, but neither test actively configures the connector first. This can lead to low-signal coverage in CI.
Suggested pattern
- const deleteBtn = observabilityPage.getConnectorDeleteBtn('otel') - const isVisible = await deleteBtn.isVisible().catch(() => false) - - if (!isVisible) { - test.skip(true, 'OTel delete button not visible (connector may not be configured)') - return - } + if ( + !(await observabilityPage.isConnectorEnabled('otel')) && + (await observabilityPage.isToggleEnabled('otel')) + ) { + await observabilityPage.toggleConnector('otel') + await observabilityPage.saveConfiguration() + } + const deleteBtn = observabilityPage.getConnectorDeleteBtn('otel') await expect(deleteBtn).toBeVisible()Also applies to: 219-238
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/observability/observability.spec.ts` around lines 81 - 93, The test currently checks for an OTel delete button and skips if it's not visible without first ensuring the connector is configured; update the test to programmatically establish the "configured" precondition by calling a setup helper (e.g., observabilityPage.configureConnector('otel') or a fixture method) before calling observabilityPage.selectConnector('otel') and getConnectorDeleteBtn('otel'), then assert the configuration succeeded and only use test.skip if configuration fails; reference observabilityPage.selectConnector, observabilityPage.configureConnector (or create that helper), getConnectorDeleteBtn('otel'), and test.skip when implementing this change.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/workflows/e2e-tests.yml:
- Around line 4-5: The workflow currently triggers on push.branches =
["02-27-feat_extend_e2e_ui_tests"]; update the trigger so E2E runs on the main
integration branch by changing push.branches to ["main"] and optionally add a
pull_request trigger (add a top-level pull_request key) so the workflow runs for
PRs as well; look for the push, branches and pull_request keys in the workflow
to make this change.
In `@tests/e2e/features/config/config.spec.ts`:
- Around line 49-51: Replace brittle label/CSS selectors in the spec with the
page-object test-id locators: update assertions that call
configSettingsPage.page.getByLabel(...) and any locator('#...') checks to use
the page object’s getByTestId(...) methods (e.g.,
configSettingsPage.getByTestId('max-agent-depth') and
configSettingsPage.getByTestId('tool-execution-timeout')). Ensure the
configSettingsPage class extends BasePage and exposes getByTestId wrappers, and
that the spec imports test/expect from ../../core/fixtures/base.fixture (not
`@playwright/test`). Apply the same change to the other assertions referenced (the
ones currently using getByLabel/locator).
In `@tests/e2e/features/mcp-registry/mcp-registry.spec.ts`:
- Around line 275-283: The test currently accepts 'error' as a valid
post-reconnect status; update the assertion in the reconnect test (after calling
mcpRegistryPage.reconnectClient and retrieving status via
mcpRegistryPage.getClientStatus) to reject 'error' — e.g., assert that status is
truthy and that status.toLowerCase() is one of the allowed states excluding
'error' (use the existing variables/methods clientExists, getClientStatus,
reconnectClient and the local status variable to locate the check and remove
'error' from the allowed values or tighten to require 'connected').
In `@tests/e2e/features/observability/pages/observability.page.ts`:
- Around line 143-145: In toggleConnector, after performing await
toggle.click(), wait for the element's data-state attribute to change to the
expected post-click value before returning; locate the toggle element referenced
by the toggle variable inside the toggleConnector function and use a wait for
the attribute transition (data-state) or an equivalent wait-for-selector
targeting the new state so subsequent assertions don't race the UI update.
In `@tests/e2e/features/plugins/plugins.spec.ts`:
- Around line 240-251: The current branch decision uses an immediate
pluginsPage.sheet.isVisible() which can race with UI transitions; instead wait
explicitly for either the inline error locator
(pluginsPage.sheet.locator('[role="alert"], .text-destructive')) or the toast
locator (pluginsPage.page.locator('[data-sonner-toast][data-type="error"]')) to
become visible (or timeout) and then branch based on which became visible;
ensure you still call pluginsPage.cancelPlugin() when the inline error path is
observed. Use those unique locators and pluginsPage.cancelPlugin to locate and
implement the fix.
In `@tests/e2e/features/providers/pages/providers.page.ts`:
- Around line 428-438: setPerformanceConfig() is currently trying to toggle
"Send Back Raw Request/Response" while remaining on the performance tab, but
those controls now live under the Debugging tab (selectors renamed to
getRawRequestSwitch / getRawResponseSwitch) causing lookup failures; update
setPerformanceConfig to first navigate to or select the Debugging tab (or accept
a tab parameter) before interacting with the raw request/response switches, and
use the new helpers getRawRequestSwitch() and getRawResponseSwitch() to perform
the toggles; apply the same change to the other occurrences referenced around
the 484-517 region so all interactions with these switches target the Debugging
tab.
- Around line 398-407: Add data-testid attributes to the debugging UI controls
in debuggingFormFragment.tsx: add
data-testid="provider-config-debugging-save-btn" to the Save button and
data-testid="send-back-raw-request-switch" and
data-testid="send-back-raw-response-switch" to the respective switch inputs;
then update the page object methods getConfigSaveBtn, getRawRequestSwitch, and
getRawResponseSwitch to use getByTestId('provider-config-debugging-save-btn'),
getByTestId('send-back-raw-request-switch'), and
getByTestId('send-back-raw-response-switch') respectively instead of
getByRole/getByLabel/DOM traversal so tests follow the test-id selector
guideline.
In `@tests/e2e/features/providers/providers.spec.ts`:
- Around line 371-380: The test currently uses expect(typeof
modelsVisible).toBe('boolean') which is a no-op; replace that with a concrete
assertion on the observed state by asserting modelsVisible is true (or
explicitly asserting visibility when present). Locate
providersPage.page.getByText(/Models/i).first() (modelsSection) and the
modelsVisible variable and change the check to expect(modelsVisible).toBe(true)
(or, if you want to allow both states, assert via the branch: if (modelsVisible)
await expect(modelsSection).toBeVisible(); else await
expect(modelsSection).not.toBeVisible()), ensuring the test fails when the
models section never appears.
- Around line 881-885: The test currently asserts (urlVisible || modelVisible)
which allows only one field to be visible; change the check to require both
fields by asserting (urlVisible && modelVisible) in the same test block and
update the skip branch to trigger when either is missing (use the same
urlVisible and modelVisible variables), so the test name “fields” correctly
requires both vLLM form fields to be shown.
In `@tests/e2e/features/virtual-keys/virtual-keys.spec.ts`:
- Around line 401-409: The test currently calls
virtualKeysPage.cleanupAllVirtualKeys(), which is a global destructive cleanup;
instead, stop calling cleanupAllVirtualKeys() inside the test flow and replace
with a scoped pattern: create keys with unique names using Date.now(), push
created key IDs into a local array (e.g., createdVirtualKeys) whenever you call
the factory, and only delete those tracked keys in an afterEach hook; also
change the pre-test check around virtualKeysPage.table /
virtualKeysPage.emptyState to branch (skip destructive deletion) when hadTable
is true and assert accordingly rather than wiping all keys. Ensure references to
virtualKeysPage.table, virtualKeysPage.emptyState,
virtualKeysPage.cleanupAllVirtualKeys, and the new createdVirtualKeys tracking
array are updated so cleanup is scoped.
---
Duplicate comments:
In `@tests/e2e/features/config/config.spec.ts`:
- Around line 125-133: The test fills pricingDatasheetUrlInput before checking
save/editability which can fail under RBAC; instead, first determine editability
by checking pricingSaveBtn (or pricingDatasheetUrlInput.isEnabled()) and call
test.skip(...) if not editable, then perform
pricingDatasheetUrlInput.fill('invalid-url-no-http') and proceed to click
pricingSaveBtn; update the test block around
configSettingsPage.pricingDatasheetUrlInput and
configSettingsPage.pricingSaveBtn so the RBAC guard runs before any input.fill
operations.
In `@tests/e2e/features/observability/pages/observability.page.ts`:
- Around line 52-63: The fallback locators in getConnectorToggle and
getConnectorDeleteBtn make tests nondeterministic for unmapped connectors;
instead of returning a broad or no-match locator, change both functions to
hard-fail when the connector key is not present in
ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS /
ObservabilityPage.CONNECTOR_DELETE_TESTIDS by throwing a descriptive Error that
names the missing connector and the expected testId; this forces callers (e.g.,
getCurrentState, disableAllConnectors) to only use mapped connectors and keeps
selectors exclusively via getByTestId.
In `@tests/e2e/features/providers/providers.spec.ts`:
- Around line 779-809: The test currently restores pricing overrides only on the
happy path; wrap the mutation and save/dismiss steps in a try/finally so
restoration always runs: perform
providersPage.setPricingOverrides(validOverrides), check/save/dismiss inside a
try block, and in finally read the original value (initialValue from
providersPage.pricingOverridesJsonInput.inputValue()), call
providersPage.setPricingOverrides(initialValue) and conditionally save/dismiss
using providersPage.pricingOverridesSaveBtn and
providersPage.savePricingOverrides()/providersPage.dismissToasts() to guarantee
cleanup even if save or dismiss throws; reference providersPage.selectConfigTab,
setPricingOverrides, pricingOverridesJsonInput, pricingOverridesSaveBtn,
savePricingOverrides, and dismissToasts to locate the changes.
---
Nitpick comments:
In `@tests/e2e/features/config/config.spec.ts`:
- Around line 393-406: The test currently only checks visibility before
interacting with enforceAuthOnInferenceSwitch, but a visible switch can be
disabled and clicking it will fail; before calling
getSwitchState/toggleEnforceAuthOnInference, check that
configSettingsPage.enforceAuthOnInferenceSwitch.isEnabled().catch(() => false)
and if false call test.skip(true, 'Enforce auth on inference not enabled for
this role') and return; keep the existing calls to getSwitchState,
toggleEnforceAuthOnInference, hasPendingChanges, and saveSettings unchanged
otherwise.
In `@tests/e2e/features/governance/pages/governance.page.ts`:
- Around line 103-110: The deleteTeam method uses a generic locator for the
confirmation dialog which can match any alertdialog; update the UI to add a
specific data-testid (e.g. data-testid="team-delete-dialog") on the delete
confirmation dialog, then change deleteTeam to target that test id (use
this.page.getByTestId('team-delete-dialog') or
locator('[data-testid="team-delete-dialog"]') before clicking the Delete button)
while keeping the existing team-delete-btn-{name} trigger and
waitForSuccessToast call intact so the flow remains deterministic.
In `@tests/e2e/features/mcp-settings/mcp-settings.spec.ts`:
- Around line 12-15: Replace direct label-based selectors in the test block for
'should display MCP settings form fields' with the MCPSettingsPage's locators:
use mcpSettingsPage.maxAgentDepthInput and mcpSettingsPage.toolTimeoutInput
instead of mcpSettingsPage.page.getByLabel(...), then assert visibility on those
locators (e.g., await expect(mcpSettingsPage.maxAgentDepthInput).toBeVisible()).
This keeps the page-object abstraction and the test-id-first strategy intact
while leaving the test name and flow unchanged.
In `@tests/e2e/features/model-limits/pages/model-limits.page.ts`:
- Around line 57-60: Replace usage of CSS ID selectors with test-id selectors:
update the locator in model-limits.page.ts that currently uses
this.page.locator('#modelBudgetMaxLimit') to use
this.page.getByTestId('modelBudgetMaxLimit') (and do the same for the
editModelLimit-related locator around lines 74-78). Add a matching
data-testid="modelBudgetMaxLimit" attribute to the budget input in the UI if it
doesn't exist, and update any variables (budgetInput, editModelLimit) to
reference the new getByTestId locators so the page object follows the project's
getByTestId selector guideline.
- Around line 48-55: The tests use fragile generic selectors:
fillSelect(this.page, '[role="combobox"]', ...) and
this.sheet.getByPlaceholder(/Search for a model/i) which can match multiple
elements; update the UI to add stable data-testid attributes on the provider
combobox and the model search input in modelLimitSheet.tsx (e.g.,
data-testid="provider-combobox" and data-testid="model-search-input"), then
update the test to use those testids (referencing the fillSelect call and the
modelInput retrieval) so fillSelect targets the specific provider combobox and
modelInput uses getByTestId instead of getByPlaceholder to improve reliability.
In `@tests/e2e/features/observability/observability.spec.ts`:
- Around line 175-177: Replace fragile UI-label assertions that rely on visible
text in observabilityPage.getSelectedConnector() with checks against the
connector's stable identifier/data-testid: update the test to retrieve the
selected tab's data-testid or connector-id (via
observabilityPage.getSelectedConnector() or add a new
observabilityPage.getSelectedConnectorId() helper) and assert
equality/containment against the expected id (e.g., "prometheus" or "bigquery")
instead of asserting toContain('Prometheus'); apply the same change to the other
assertion locations currently using toContain('BigQuery').
- Around line 81-93: The test currently checks for an OTel delete button and
skips if it's not visible without first ensuring the connector is configured;
update the test to programmatically establish the "configured" precondition by
calling a setup helper (e.g., observabilityPage.configureConnector('otel') or a
fixture method) before calling observabilityPage.selectConnector('otel') and
getConnectorDeleteBtn('otel'), then assert the configuration succeeded and only
use test.skip if configuration fails; reference
observabilityPage.selectConnector, observabilityPage.configureConnector (or
create that helper), getConnectorDeleteBtn('otel'), and test.skip when
implementing this change.
In `@tests/e2e/features/plugins/pages/plugins.page.ts`:
- Around line 57-59: Replace brittle role/CSS/icon-based readiness and count
checks that use createBtn and pluginList with test-id based locators: call
this.page.getByTestId('plugins-empty-state') for the empty-state readiness path
and this.page.getByTestId('plugins-list') (or the stable test-id for the
list/count) for the list readiness/count path, and use the same chaining
(.first().waitFor({...}) or .waitFor({...})) currently applied to
createBtn/pluginList; update both usages that reference createBtn and pluginList
so the primary selector is getByTestId() while preserving existing .or(),
.first(), and .waitFor() logic.
In `@ui/app/workspace/providers/fragments/debuggingFormFragment.tsx`:
- Line 66: Add data-testid attributes to the interactive controls inside the
debugging form so e2e tests can target them: add
data-testid="provider-config-debugging-switch-<name>" to each Switch (use the
internal prop/name for each switch to replace <name>) and
data-testid="provider-config-debugging-submit" to the form submit Button; update
the Switch and Button elements referenced in this file (the Switch components
and the submit Button used with form.handleSubmit/onSubmit) to include those
attributes following the <entity>-<element>-<qualifier> pattern.
ℹ️ Review info
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (54)
.github/workflows/e2e-tests.ymltests/e2e/core/fixtures/base.fixture.tstests/e2e/features/config/config.spec.tstests/e2e/features/config/pages/config-settings.page.tstests/e2e/features/dashboard/dashboard.spec.tstests/e2e/features/governance/governance.data.tstests/e2e/features/governance/governance.spec.tstests/e2e/features/governance/pages/governance.page.tstests/e2e/features/mcp-auth-config/mcp-auth-config.spec.tstests/e2e/features/mcp-auth-config/pages/mcp-auth-config.page.tstests/e2e/features/mcp-registry/mcp-registry.spec.tstests/e2e/features/mcp-registry/pages/mcp-registry.page.tstests/e2e/features/mcp-settings/mcp-settings.spec.tstests/e2e/features/mcp-settings/pages/mcp-settings.page.tstests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.tstests/e2e/features/mcp-tool-groups/pages/mcp-tool-groups.page.tstests/e2e/features/model-limits/model-limits.data.tstests/e2e/features/model-limits/model-limits.spec.tstests/e2e/features/model-limits/pages/model-limits.page.tstests/e2e/features/observability/observability.spec.tstests/e2e/features/observability/pages/observability.page.tstests/e2e/features/placeholders/placeholders.spec.tstests/e2e/features/plugins/pages/plugins.page.tstests/e2e/features/plugins/plugins.spec.tstests/e2e/features/providers/pages/providers.page.tstests/e2e/features/providers/providers.spec.tstests/e2e/features/routing-rules/pages/routing-rules.page.tstests/e2e/features/routing-rules/routing-rules.spec.tstests/e2e/features/virtual-keys/pages/virtual-keys.page.tstests/e2e/features/virtual-keys/virtual-keys.spec.tsui/app/workspace/config/views/clientSettingsView.tsxui/app/workspace/config/views/mcpView.tsxui/app/workspace/config/views/pricingConfigView.tsxui/app/workspace/config/views/securityView.tsxui/app/workspace/governance/views/customerDialog.tsxui/app/workspace/governance/views/customerTable.tsxui/app/workspace/governance/views/teamDialog.tsxui/app/workspace/governance/views/teamsTable.tsxui/app/workspace/mcp-registry/views/mcpClientsTable.tsxui/app/workspace/mcp-registry/views/mcpServersEmptyState.tsxui/app/workspace/model-limits/views/modelLimitSheet.tsxui/app/workspace/model-limits/views/modelLimitsEmptyState.tsxui/app/workspace/model-limits/views/modelLimitsTable.tsxui/app/workspace/plugins/fragments/pluginFormFragments.tsxui/app/workspace/plugins/views/pluginsEmptyState.tsxui/app/workspace/providers/fragments/debuggingFormFragment.tsxui/app/workspace/providers/page.tsxui/app/workspace/providers/views/modelProviderKeysTableView.tsxui/app/workspace/providers/views/providersEmptyState.tsxui/app/workspace/routing-rules/views/routingRulesEmptyState.tsxui/app/workspace/routing-rules/views/routingRulesTable.tsxui/app/workspace/virtual-keys/views/virtualKeysEmptyState.tsxui/app/workspace/virtual-keys/views/virtualKeysTable.tsxui/components/ui/numberAndSelect.tsx
✅ Files skipped from review due to trivial changes (1)
- ui/app/workspace/routing-rules/views/routingRulesEmptyState.tsx
🚧 Files skipped from review as they are similar to previous changes (25)
- tests/e2e/features/routing-rules/routing-rules.spec.ts
- ui/app/workspace/virtual-keys/views/virtualKeysTable.tsx
- ui/app/workspace/model-limits/views/modelLimitsTable.tsx
- tests/e2e/features/mcp-auth-config/pages/mcp-auth-config.page.ts
- tests/e2e/features/mcp-tool-groups/pages/mcp-tool-groups.page.ts
- ui/app/workspace/config/views/mcpView.tsx
- ui/components/ui/numberAndSelect.tsx
- tests/e2e/features/model-limits/model-limits.data.ts
- tests/e2e/features/mcp-settings/pages/mcp-settings.page.ts
- ui/app/workspace/mcp-registry/views/mcpServersEmptyState.tsx
- ui/app/workspace/governance/views/teamDialog.tsx
- ui/app/workspace/config/views/securityView.tsx
- ui/app/workspace/mcp-registry/views/mcpClientsTable.tsx
- ui/app/workspace/providers/views/providersEmptyState.tsx
- ui/app/workspace/plugins/fragments/pluginFormFragments.tsx
- ui/app/workspace/config/views/clientSettingsView.tsx
- ui/app/workspace/model-limits/views/modelLimitsEmptyState.tsx
- ui/app/workspace/governance/views/teamsTable.tsx
- ui/app/workspace/config/views/pricingConfigView.tsx
- ui/app/workspace/model-limits/views/modelLimitSheet.tsx
- ui/app/workspace/governance/views/customerTable.tsx
- ui/app/workspace/governance/views/customerDialog.tsx
- tests/e2e/features/model-limits/model-limits.spec.ts
- tests/e2e/features/config/pages/config-settings.page.ts
- tests/e2e/features/mcp-auth-config/mcp-auth-config.spec.ts
Radheshg04
force-pushed
the
02-27-feat_extend_e2e_ui_tests
branch
from
b47e66b to
2ef6161
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 6
♻️ Duplicate comments (2)
tests/e2e/features/observability/pages/observability.page.ts (1)
52-55:⚠️ Potential issue | 🟠 Major
getConnectorTogglefallback is still nondeterministic for non-mapped connectors.Line 54 falls back to the first switch in the view, which can target the wrong control when a panel has multiple switches. With new Maxim/Datadog/BigQuery toggle tests, this can produce false passes or flaky failures.
Suggested fix
getConnectorToggle(connector: ObservabilityConnector): Locator { const testId = ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS[connector] - return testId ? this.page.getByTestId(testId) : this.page.locator('button[role="switch"]').first() + if (!testId) { + throw new Error(`Missing toggle test id mapping for connector: ${connector}`) + } + return this.page.getByTestId(testId) }Run this read-only check to confirm connector toggle APIs are used for connectors missing testid mappings:
#!/bin/bash set -euo pipefail python - <<'PY' import re, pathlib page_file = pathlib.Path("tests/e2e/features/observability/pages/observability.page.ts") spec_file = pathlib.Path("tests/e2e/features/observability/observability.spec.ts") page = page_file.read_text() spec = spec_file.read_text() m = re.search(r'CONNECTOR_TOGGLE_TESTIDS:[\s\S]*?=\s*\{([\s\S]*?)\n\s*\}', page) mapped = set(re.findall(r"\b([a-z]+)\s*:", m.group(1))) if m else set() used = set(re.findall(r"(?:isToggleEnabled|isConnectorEnabled|toggleConnector)\('([a-z]+)'\)", spec)) print("mapped_toggle_testids:", sorted(mapped)) print("toggle_apis_used_in_spec:", sorted(used)) print("used_but_unmapped:", sorted(used - mapped)) PYAs per coding guidelines "
tests/e2e/**/*.ts: E2E test page objects must extend BasePage, use getByTestId() as the primary selector strategy..."🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/observability/pages/observability.page.ts` around lines 52 - 55, getConnectorToggle currently falls back to this.page.locator('button[role="switch"]').first(), which is nondeterministic and causes flaky tests for connectors not present in CONNECTOR_TOGGLE_TESTIDS; update getConnectorToggle to throw or assert when a connector key is unmapped (using CONNECTOR_TOGGLE_TESTIDS) and require tests to call the connector-specific toggle APIs (isToggleEnabled, isConnectorEnabled, toggleConnector) for unmapped connectors, or alternatively implement a deterministic selector lookup (e.g., locate by data-connector attribute) inside getConnectorToggle to uniquely select the intended switch for the given connector; change the function getConnectorToggle and ensure CONNECTOR_TOGGLE_TESTIDS is the source of truth so tests in observability.spec.ts use mapped IDs instead of relying on the first() fallback.tests/e2e/features/providers/pages/providers.page.ts (1)
388-397:⚠️ Potential issue | 🟠 MajorDebugging/config selectors are still not test-id-first in the page object.
getConfigSaveBtn,getRawRequestSwitch, andgetRawResponseSwitchstill rely on role/label traversal instead of dedicatedgetByTestId()locators, so this remains brittle against text/layout changes.As per coding guidelines,
tests/e2e/**/*.ts: “E2E test page objects must extend BasePage, use getByTestId() as the primary selector strategy…”Also applies to: 420-429
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/providers/pages/providers.page.ts` around lines 388 - 397, The page object must use test-id-first selectors and extend BasePage: update getConfigSaveBtn, getRawRequestSwitch, and getRawResponseSwitch to use this.page.getByTestId(...) as the primary locator (map each configType and switch to a stable test id like config-save-{type}, raw-request-switch, raw-response-switch) and only fall back to role/label if the test id is missing; also ensure the class extends BasePage if it doesn't already so the shared this.page/getByTestId utilities are available.
🧹 Nitpick comments (10)
tests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.ts (1)
11-11: Match the URL pattern to/workspace/route like other E2E assertions.The current regex
/mcp-tool-groups/is too loose and will match unintended URLs. Align with the established pattern in the codebase by using/\/workspace\/mcp-tool-groups/.Suggested diff
- await expect(mcpToolGroupsPage.page).toHaveURL(/mcp-tool-groups/) + await expect(mcpToolGroupsPage.page).toHaveURL(/\/workspace\/mcp-tool-groups/)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.ts` at line 11, The URL assertion uses a too-broad regex; update the expect call on mcpToolGroupsPage.page to use the stricter pattern that matches the workspace route (replace the current /mcp-tool-groups/ regex with /\/workspace\/mcp-tool-groups/), i.e. modify the toHaveURL invocation to assert against the workspace-prefixed regex so it only matches /workspace/mcp-tool-groups URLs.tests/e2e/features/plugins/pages/plugins.page.ts (1)
57-59: Prefer a test-id-first create button locator in page readiness checks.
goto()now correctly waits onplugins-empty-state, butcreateBtnis still primarily text/icon-driven. UsegetByTestId('plugins-button-install-new')as the primary selector to match the page-object selector guideline and reduce brittleness.♻️ Suggested locator adjustment
- this.createBtn = page.getByRole('button').filter({ - hasText: /Install New Plugin/i - }).or( - page.locator('button').filter({ has: page.locator('svg.lucide-plus') }).filter({ hasText: /Install/i }) - ) + this.createBtn = page.getByTestId('plugins-button-install-new').or( + page.getByRole('button', { name: /Install New Plugin/i }) + )As per coding guidelines, "E2E test page objects must ... use getByTestId() as the primary selector strategy."
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/plugins/pages/plugins.page.ts` around lines 57 - 59, Update the page readiness wait to prefer the test-id-based create button locator: replace the current expression that starts with this.createBtn.or(...) so the primary selector is this.page.getByTestId('plugins-button-install-new') (e.g., this.page.getByTestId('plugins-button-install-new').or(this.page.getByTestId('plugins-empty-state')).first().waitFor({ state: 'visible', timeout: 10000 })); this ensures the primary selector follows the page-object guideline while still falling back to 'plugins-empty-state' for readiness checks and keeps the existing waitFor timeout and semantics.tests/e2e/features/plugins/plugins.spec.ts (1)
240-252: Consider extracting shared inline-error vs toast-error assertion flow.Both blocks implement the same wait/branch/assert pattern. A helper would reduce duplication and keep future error-handling tweaks consistent.
🧩 Optional DRY refactor
+ async function assertInlineOrToastError(pluginsPage: PluginsPage): Promise<boolean> { + const inlineError = pluginsPage.sheet.locator('[role="alert"], .text-destructive').first() + const errorToast = pluginsPage.page.locator('[data-sonner-toast][data-type="error"]').first() + await inlineError.or(errorToast).waitFor({ state: 'visible', timeout: 10000 }) + const hasInlineError = await inlineError.isVisible().catch(() => false) + if (hasInlineError) { + await expect(pluginsPage.sheet).toBeVisible() + await expect(inlineError).toBeVisible() + } else { + await expect(errorToast).toBeVisible() + } + return hasInlineError + }Also applies to: 341-353
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/plugins/plugins.spec.ts` around lines 240 - 252, Extract the duplicated "inline-error vs toast-error" wait/branch/assert flow into a reusable helper (e.g., assertInlineOrToastError or handleErrorToastOrInline) and replace both occurrences with calls to it; the helper should accept the page/sheet/context and perform: compute inlineError = pluginsPage.sheet.locator('[role="alert"], .text-destructive').first(), errorToast = pluginsPage.page.locator('[data-sonner-toast][data-type="error"]').first(), await inlineError.or(errorToast).waitFor({ state: 'visible', timeout: 10000 }), check inline visibility with inlineError.isVisible().catch(() => false), assert sheet and inlineError visibility and call pluginsPage.cancelPlugin() when inline error exists, otherwise assert errorToast visibility — then call this helper in place of the duplicated blocks referencing pluginsPage.sheet, inlineError, errorToast, and pluginsPage.cancelPlugin.tests/e2e/features/placeholders/placeholders.spec.ts (2)
9-18: Refactor this spec into a Placeholder page object for selector consistency and lower duplication.This file repeats navigation + assertions across many tests. A dedicated page object (extending
BasePage) with stablegetByTestId()locators would make flows easier to maintain and less brittle as UI copy changes.As per coding guidelines:
tests/e2e/**/*.ts: “E2E test page objects must extend BasePage, use getByTestId() as the primary selector strategy, and import test/expect from ../../core/fixtures/base.fixture, never from@playwright/test.”Also applies to: 20-77
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/placeholders/placeholders.spec.ts` around lines 9 - 18, The spec duplicates navigation and assertions; refactor by creating a PlaceholderPage class that extends BasePage and exposes stable getByTestId() locators and convenience methods (e.g., loadAlertChannels(), readMoreButton(), openReadMorePopup()) that encapsulate page.goto('/workspace/alert-channels'), waitForLoadState, visibility checks, and popup handling; update the test to import test/expect from ../../core/fixtures/base.fixture, instantiate PlaceholderPage(page), call PlaceholderPage.loadAlertChannels() and PlaceholderPage.openReadMorePopup(), and replace direct calls to page.getByText / page.getByRole with the page object’s getByTestId-based selectors (e.g., getByTestId('placeholder-read-more') and getByTestId('placeholder-heading')) to centralize selectors and reduce duplication.
11-11: ReplacewaitForLoadState('networkidle')with assertion-driven readiness checks.Playwright explicitly discourages
waitForLoadState('networkidle')because it waits for 500+ ms of network inactivity—unreliable for modern apps with background traffic, polling, or WebSocket connections. The subsequent assertions (toHaveURL(),toBeVisible()) are stronger, more specific readiness signals and provide better test reliability.These waits can be safely removed at lines 11, 22, 28, 34, 40, 46, 52, 58, 69, 75 since assertions immediately follow and will auto-wait until the required condition is met.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/placeholders/placeholders.spec.ts` at line 11, Replace occurrences of page.waitForLoadState('networkidle') with assertion-driven readiness checks: remove the explicit wait calls and rely on the subsequent assertions (e.g., expect(page).toHaveURL(...) and expect(locator).toBeVisible(...)) which auto-wait; update any tests that currently call page.waitForLoadState('networkidle') to simply perform the following assertions (toHaveURL, toBeVisible, etc.) in place so they become the gating condition for readiness and remove flakiness from background network activity.tests/e2e/features/mcp-settings/mcp-settings.spec.ts (1)
13-15: Use page-object locators here instead of direct label selectors.Line 13 and Line 14 should assert through
mcpSettingsPagelocators (test-id-backed) to keep selector strategy consistent and less brittle.Suggested fix
- await expect(mcpSettingsPage.page.getByLabel('Max Agent Depth')).toBeVisible() - await expect(mcpSettingsPage.page.getByLabel('Tool Execution Timeout (seconds)')).toBeVisible() + await expect(mcpSettingsPage.maxAgentDepthInput).toBeVisible() + await expect(mcpSettingsPage.toolTimeoutInput).toBeVisible()As per coding guidelines,
tests/e2e/**/*.ts: “E2E test page objects must extend BasePage, use getByTestId() as the primary selector strategy…”🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/mcp-settings/mcp-settings.spec.ts` around lines 13 - 15, The assertions currently use direct label selectors (mcpSettingsPage.page.getByLabel('Max Agent Depth') and mcpSettingsPage.page.getByLabel('Tool Execution Timeout (seconds)')) which violates the page-object test-id strategy; update these to use the mcpSettingsPage page-object locators (e.g., use mcpSettingsPage.getByTestId('max-agent-depth') or the page-object properties like mcpSettingsPage.maxAgentDepth and mcpSettingsPage.toolExecutionTimeout) and assert visibility via those locators so tests rely on test-id-backed selectors instead of labels.ui/app/workspace/governance/views/customerTable.tsx (1)
168-168: Prefer a stable qualifier for the row test-id (usecustomer.idinstead of raw name).Name-based test IDs are prone to collisions and special-character issues. A stable id-based qualifier is safer for E2E selectors.
Suggested fix
- data-testid={`customer-row-${customer.name}`} + data-testid={`customer-row-${customer.id}`}As per coding guidelines,
ui/app/**/*.{tsx,ts}: if you rename/remove adata-testid, “search tests/e2e/ for references.”🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/governance/views/customerTable.tsx` at line 168, The data-testid for table rows currently uses the unstable customer.name; update the test-id generation to use the stable unique identifier customer.id (e.g., change `data-testid={`customer-row-${customer.name}`}` to use `customer.id`) so selectors won't break on duplicates or special characters, and after renaming/removing this data-testid search the e2e tests for references and update any selectors accordingly; locate this change in the component that renders customer rows (CustomerTable / the row render function where `data-testid={`customer-row-...`}` is defined).tests/e2e/features/virtual-keys/virtual-keys.spec.ts (1)
121-123: Prefer page-object test-id locators over direct#idselectors in spec code.These checks currently bypass the page object (
virtualKeysPage.page.locator('#...')). Consider exposing these fields onVirtualKeysPagewithgetByTestId()and asserting through the page object for consistency and lower selector brittleness.As per coding guidelines,
tests/e2e/**/*.ts: “E2E test page objects must extend BasePage, use getByTestId() as the primary selector strategy…”Also applies to: 166-168
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/virtual-keys/virtual-keys.spec.ts` around lines 121 - 123, The spec is directly using page.locator('#budgetMaxLimit') instead of the VirtualKeysPage page-object; update VirtualKeysPage to expose a locator/accessor for the budget input using getByTestId (e.g., add a getter like budgetMaxLimit or getBudgetMaxLimit that returns this.getByTestId('budgetMaxLimit')), then change the test to assert via virtualKeysPage.budgetMaxLimit (or virtualKeysPage.getBudgetMaxLimit()) and keep the await virtualKeysPage.closeSheet() call; also apply the same pattern for the other instances flagged (lines ~166-168) and keep using SAMPLE_BUDGETS.small.maxLimit for the expected value.tests/e2e/features/governance/governance.spec.ts (1)
26-37: Consider navigating to customers page once before the cleanup loop.The cleanup loop navigates to
/workspace/governance/customerson every iteration (line 28). Moving the navigation outside the loop would be more efficient.♻️ Suggested improvement
createdTeams.length = 0 + if (createdCustomers.length > 0) { + await governancePage.gotoCustomers() + } for (const name of [...createdCustomers]) { try { - await governancePage.gotoCustomers() const exists = await governancePage.customerExists(name) if (exists) { await governancePage.deleteCustomer(name) } } catch (e) { console.error(`[CLEANUP] Failed to delete customer ${name}:`, e) } }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/governance/governance.spec.ts` around lines 26 - 37, The loop currently calls governancePage.gotoCustomers() on every iteration which is inefficient; move the single navigation call outside the for loop so you navigate once before iterating over createdCustomers, then inside the loop call governancePage.customerExists(name) and governancePage.deleteCustomer(name) as before; keep the try/catch around the per-item delete to handle errors, and ensure createdCustomers.length = 0 remains after the loop.tests/e2e/features/governance/pages/governance.page.ts (1)
4-4: Remove unused importfillSelect.The
fillSelectutility is imported but never used in this file.♻️ Suggested fix
-import { fillSelect, waitForNetworkIdle } from '../../../core/utils/test-helpers' +import { waitForNetworkIdle } from '../../../core/utils/test-helpers'🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/governance/pages/governance.page.ts` at line 4, The import statement in governance.page.ts includes an unused symbol `fillSelect` (imported alongside `waitForNetworkIdle`); remove `fillSelect` from the import to eliminate the unused import warning and keep only `waitForNetworkIdle` in the import list so the file imports only symbols it actually uses.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/e2e/features/dashboard/dashboard.spec.ts`:
- Around line 233-243: The chart-content selector is too generic and can match
decorative SVG/icons causing false positives; update the selectors used by
volumeChartContent, tokenChartContent, costChartContent, and modelChartContent
to target the actual chart drawing surface (e.g., a specific chart container
class or SVG structure such as an SVG with a chart-group child like "svg > g" or
a canvas inside the chart canvas wrapper) or add an additional assertion that
each matched element has drawing content (non-zero width/height, has child <g>
elements, or pixel data for canvas) before using .count() so the assertions on
volumeCount, tokenCount, costCount, and modelCount only pass when real chart
surfaces rendered.
In `@tests/e2e/features/model-limits/pages/model-limits.page.ts`:
- Around line 55-61: The test currently clicks the first dropdown option using
modelInput / firstOption and captures selectedModelName, which ignores
config.modelName and causes nondeterministic selection; update the selection
logic to use config.modelName: open the model multiselect (modelInput), wait for
options, locate the option whose text equals or includes config.modelName
(instead of using first()), click that matching option, set selectedModelName
from that matched option, and if no match is found throw or fail the test so
selection is deterministic (adjust references to modelInput, firstOption, and
selectedModelName accordingly).
In `@tests/e2e/features/placeholders/placeholders.spec.ts`:
- Line 23: The current URL assertions use unanchored regex in await
expect(page).toHaveURL which allows longer/deeper routes to pass; update each
call (the await expect(page).toHaveURL invocations in placeholders.spec.ts) to
anchor the end and optionally allow query params, e.g. change
/\/workspace\/guardrails/ to /\/workspace\/guardrails(?:\?.*)?$/ so the
assertion matches the exact route (with optional query string). Apply this
anchoring pattern to each similar assertion in the file (all other await
expect(page).toHaveURL calls).
In `@tests/e2e/features/routing-rules/routing-rules.spec.ts`:
- Around line 7-8: Replace the fixed SEED_RULE_NAME constant with a unique,
timestamped name (use Date.now()) to avoid collisions; update
tests/e2e/features/routing-rules/routing-rules.spec.ts to generate
SEED_RULE_NAME dynamically (e.g., append Date.now()) and ensure created rule
names are recorded in a tracking array and removed in an afterEach cleanup hook
so parallel/previous runs cannot delete non-test data; locate references to
SEED_RULE_NAME in the file and adjust setup/teardown to use the tracked names.
- Around line 33-40: The worker-scoped beforeAll/afterAll hooks are using the
test-scoped fixture routingRulesPage (which depends on page) and will fail at
runtime; move the seed creation logic currently in test.beforeAll (and cleanup
in test.afterAll) into test.beforeEach and test.afterEach respectively so they
can access routingRulesPage, or alternatively implement seeding using a
worker-scoped API/browser fixture instead of routingRulesPage; also change the
shared SEED_RULE_NAME to include a unique suffix (e.g., append Date.now()) to
match other test factories and avoid collisions when running tests in parallel.
In `@ui/app/workspace/model-limits/views/modelLimitsTable.tsx`:
- Line 154: The data-testid for the TableRow in modelLimitsTable.tsx uses raw
config.model_name and config.provider which may include spaces/special chars;
normalize these fragments before interpolation by creating a slugified,
lowercase identifier (e.g., replace non-alphanumeric characters with hyphens,
collapse duplicate hyphens, trim hyphens) and use a deterministic fallback like
"all" when provider is empty so the data-testid follows the
<entity>-<element>-<qualifier> convention; update the TableRow data-testid
construction (and the other occurrences noted around lines 303 and 316) to use
the normalized model and provider fragments.
---
Duplicate comments:
In `@tests/e2e/features/observability/pages/observability.page.ts`:
- Around line 52-55: getConnectorToggle currently falls back to
this.page.locator('button[role="switch"]').first(), which is nondeterministic
and causes flaky tests for connectors not present in CONNECTOR_TOGGLE_TESTIDS;
update getConnectorToggle to throw or assert when a connector key is unmapped
(using CONNECTOR_TOGGLE_TESTIDS) and require tests to call the
connector-specific toggle APIs (isToggleEnabled, isConnectorEnabled,
toggleConnector) for unmapped connectors, or alternatively implement a
deterministic selector lookup (e.g., locate by data-connector attribute) inside
getConnectorToggle to uniquely select the intended switch for the given
connector; change the function getConnectorToggle and ensure
CONNECTOR_TOGGLE_TESTIDS is the source of truth so tests in
observability.spec.ts use mapped IDs instead of relying on the first() fallback.
In `@tests/e2e/features/providers/pages/providers.page.ts`:
- Around line 388-397: The page object must use test-id-first selectors and
extend BasePage: update getConfigSaveBtn, getRawRequestSwitch, and
getRawResponseSwitch to use this.page.getByTestId(...) as the primary locator
(map each configType and switch to a stable test id like config-save-{type},
raw-request-switch, raw-response-switch) and only fall back to role/label if the
test id is missing; also ensure the class extends BasePage if it doesn't already
so the shared this.page/getByTestId utilities are available.
---
Nitpick comments:
In `@tests/e2e/features/governance/governance.spec.ts`:
- Around line 26-37: The loop currently calls governancePage.gotoCustomers() on
every iteration which is inefficient; move the single navigation call outside
the for loop so you navigate once before iterating over createdCustomers, then
inside the loop call governancePage.customerExists(name) and
governancePage.deleteCustomer(name) as before; keep the try/catch around the
per-item delete to handle errors, and ensure createdCustomers.length = 0 remains
after the loop.
In `@tests/e2e/features/governance/pages/governance.page.ts`:
- Line 4: The import statement in governance.page.ts includes an unused symbol
`fillSelect` (imported alongside `waitForNetworkIdle`); remove `fillSelect` from
the import to eliminate the unused import warning and keep only
`waitForNetworkIdle` in the import list so the file imports only symbols it
actually uses.
In `@tests/e2e/features/mcp-settings/mcp-settings.spec.ts`:
- Around line 13-15: The assertions currently use direct label selectors
(mcpSettingsPage.page.getByLabel('Max Agent Depth') and
mcpSettingsPage.page.getByLabel('Tool Execution Timeout (seconds)')) which
violates the page-object test-id strategy; update these to use the
mcpSettingsPage page-object locators (e.g., use
mcpSettingsPage.getByTestId('max-agent-depth') or the page-object properties
like mcpSettingsPage.maxAgentDepth and mcpSettingsPage.toolExecutionTimeout) and
assert visibility via those locators so tests rely on test-id-backed selectors
instead of labels.
In `@tests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.ts`:
- Line 11: The URL assertion uses a too-broad regex; update the expect call on
mcpToolGroupsPage.page to use the stricter pattern that matches the workspace
route (replace the current /mcp-tool-groups/ regex with
/\/workspace\/mcp-tool-groups/), i.e. modify the toHaveURL invocation to assert
against the workspace-prefixed regex so it only matches
/workspace/mcp-tool-groups URLs.
In `@tests/e2e/features/placeholders/placeholders.spec.ts`:
- Around line 9-18: The spec duplicates navigation and assertions; refactor by
creating a PlaceholderPage class that extends BasePage and exposes stable
getByTestId() locators and convenience methods (e.g., loadAlertChannels(),
readMoreButton(), openReadMorePopup()) that encapsulate
page.goto('/workspace/alert-channels'), waitForLoadState, visibility checks, and
popup handling; update the test to import test/expect from
../../core/fixtures/base.fixture, instantiate PlaceholderPage(page), call
PlaceholderPage.loadAlertChannels() and PlaceholderPage.openReadMorePopup(), and
replace direct calls to page.getByText / page.getByRole with the page object’s
getByTestId-based selectors (e.g., getByTestId('placeholder-read-more') and
getByTestId('placeholder-heading')) to centralize selectors and reduce
duplication.
- Line 11: Replace occurrences of page.waitForLoadState('networkidle') with
assertion-driven readiness checks: remove the explicit wait calls and rely on
the subsequent assertions (e.g., expect(page).toHaveURL(...) and
expect(locator).toBeVisible(...)) which auto-wait; update any tests that
currently call page.waitForLoadState('networkidle') to simply perform the
following assertions (toHaveURL, toBeVisible, etc.) in place so they become the
gating condition for readiness and remove flakiness from background network
activity.
In `@tests/e2e/features/plugins/pages/plugins.page.ts`:
- Around line 57-59: Update the page readiness wait to prefer the test-id-based
create button locator: replace the current expression that starts with
this.createBtn.or(...) so the primary selector is
this.page.getByTestId('plugins-button-install-new') (e.g.,
this.page.getByTestId('plugins-button-install-new').or(this.page.getByTestId('plugins-empty-state')).first().waitFor({
state: 'visible', timeout: 10000 })); this ensures the primary selector follows
the page-object guideline while still falling back to 'plugins-empty-state' for
readiness checks and keeps the existing waitFor timeout and semantics.
In `@tests/e2e/features/plugins/plugins.spec.ts`:
- Around line 240-252: Extract the duplicated "inline-error vs toast-error"
wait/branch/assert flow into a reusable helper (e.g., assertInlineOrToastError
or handleErrorToastOrInline) and replace both occurrences with calls to it; the
helper should accept the page/sheet/context and perform: compute inlineError =
pluginsPage.sheet.locator('[role="alert"], .text-destructive').first(),
errorToast =
pluginsPage.page.locator('[data-sonner-toast][data-type="error"]').first(),
await inlineError.or(errorToast).waitFor({ state: 'visible', timeout: 10000 }),
check inline visibility with inlineError.isVisible().catch(() => false), assert
sheet and inlineError visibility and call pluginsPage.cancelPlugin() when inline
error exists, otherwise assert errorToast visibility — then call this helper in
place of the duplicated blocks referencing pluginsPage.sheet, inlineError,
errorToast, and pluginsPage.cancelPlugin.
In `@tests/e2e/features/virtual-keys/virtual-keys.spec.ts`:
- Around line 121-123: The spec is directly using
page.locator('#budgetMaxLimit') instead of the VirtualKeysPage page-object;
update VirtualKeysPage to expose a locator/accessor for the budget input using
getByTestId (e.g., add a getter like budgetMaxLimit or getBudgetMaxLimit that
returns this.getByTestId('budgetMaxLimit')), then change the test to assert via
virtualKeysPage.budgetMaxLimit (or virtualKeysPage.getBudgetMaxLimit()) and keep
the await virtualKeysPage.closeSheet() call; also apply the same pattern for the
other instances flagged (lines ~166-168) and keep using
SAMPLE_BUDGETS.small.maxLimit for the expected value.
In `@ui/app/workspace/governance/views/customerTable.tsx`:
- Line 168: The data-testid for table rows currently uses the unstable
customer.name; update the test-id generation to use the stable unique identifier
customer.id (e.g., change `data-testid={`customer-row-${customer.name}`}` to use
`customer.id`) so selectors won't break on duplicates or special characters, and
after renaming/removing this data-testid search the e2e tests for references and
update any selectors accordingly; locate this change in the component that
renders customer rows (CustomerTable / the row render function where
`data-testid={`customer-row-...`}` is defined).
ℹ️ Review info
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (54)
.github/workflows/e2e-tests.ymltests/e2e/core/fixtures/base.fixture.tstests/e2e/features/config/config.spec.tstests/e2e/features/config/pages/config-settings.page.tstests/e2e/features/dashboard/dashboard.spec.tstests/e2e/features/governance/governance.data.tstests/e2e/features/governance/governance.spec.tstests/e2e/features/governance/pages/governance.page.tstests/e2e/features/mcp-auth-config/mcp-auth-config.spec.tstests/e2e/features/mcp-auth-config/pages/mcp-auth-config.page.tstests/e2e/features/mcp-registry/mcp-registry.spec.tstests/e2e/features/mcp-registry/pages/mcp-registry.page.tstests/e2e/features/mcp-settings/mcp-settings.spec.tstests/e2e/features/mcp-settings/pages/mcp-settings.page.tstests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.tstests/e2e/features/mcp-tool-groups/pages/mcp-tool-groups.page.tstests/e2e/features/model-limits/model-limits.data.tstests/e2e/features/model-limits/model-limits.spec.tstests/e2e/features/model-limits/pages/model-limits.page.tstests/e2e/features/observability/observability.spec.tstests/e2e/features/observability/pages/observability.page.tstests/e2e/features/placeholders/placeholders.spec.tstests/e2e/features/plugins/pages/plugins.page.tstests/e2e/features/plugins/plugins.spec.tstests/e2e/features/providers/pages/providers.page.tstests/e2e/features/providers/providers.spec.tstests/e2e/features/routing-rules/pages/routing-rules.page.tstests/e2e/features/routing-rules/routing-rules.spec.tstests/e2e/features/virtual-keys/pages/virtual-keys.page.tstests/e2e/features/virtual-keys/virtual-keys.spec.tsui/app/workspace/config/views/clientSettingsView.tsxui/app/workspace/config/views/mcpView.tsxui/app/workspace/config/views/pricingConfigView.tsxui/app/workspace/config/views/securityView.tsxui/app/workspace/governance/views/customerDialog.tsxui/app/workspace/governance/views/customerTable.tsxui/app/workspace/governance/views/teamDialog.tsxui/app/workspace/governance/views/teamsTable.tsxui/app/workspace/mcp-registry/views/mcpClientsTable.tsxui/app/workspace/mcp-registry/views/mcpServersEmptyState.tsxui/app/workspace/model-limits/views/modelLimitSheet.tsxui/app/workspace/model-limits/views/modelLimitsEmptyState.tsxui/app/workspace/model-limits/views/modelLimitsTable.tsxui/app/workspace/plugins/fragments/pluginFormFragments.tsxui/app/workspace/plugins/views/pluginsEmptyState.tsxui/app/workspace/providers/fragments/debuggingFormFragment.tsxui/app/workspace/providers/page.tsxui/app/workspace/providers/views/modelProviderKeysTableView.tsxui/app/workspace/providers/views/providersEmptyState.tsxui/app/workspace/routing-rules/views/routingRulesEmptyState.tsxui/app/workspace/routing-rules/views/routingRulesTable.tsxui/app/workspace/virtual-keys/views/virtualKeysEmptyState.tsxui/app/workspace/virtual-keys/views/virtualKeysTable.tsxui/components/ui/numberAndSelect.tsx
🚧 Files skipped from review as they are similar to previous changes (25)
- ui/app/workspace/model-limits/views/modelLimitSheet.tsx
- ui/app/workspace/mcp-registry/views/mcpServersEmptyState.tsx
- ui/app/workspace/routing-rules/views/routingRulesEmptyState.tsx
- tests/e2e/features/mcp-auth-config/pages/mcp-auth-config.page.ts
- ui/app/workspace/model-limits/views/modelLimitsEmptyState.tsx
- ui/components/ui/numberAndSelect.tsx
- ui/app/workspace/providers/page.tsx
- tests/e2e/features/model-limits/model-limits.data.ts
- tests/e2e/core/fixtures/base.fixture.ts
- tests/e2e/features/governance/governance.data.ts
- ui/app/workspace/governance/views/customerDialog.tsx
- ui/app/workspace/config/views/securityView.tsx
- ui/app/workspace/mcp-registry/views/mcpClientsTable.tsx
- tests/e2e/features/mcp-registry/mcp-registry.spec.ts
- tests/e2e/features/mcp-auth-config/mcp-auth-config.spec.ts
- .github/workflows/e2e-tests.yml
- tests/e2e/features/mcp-settings/pages/mcp-settings.page.ts
- ui/app/workspace/providers/views/modelProviderKeysTableView.tsx
- tests/e2e/features/virtual-keys/pages/virtual-keys.page.ts
- ui/app/workspace/plugins/fragments/pluginFormFragments.tsx
- ui/app/workspace/governance/views/teamDialog.tsx
- ui/app/workspace/config/views/pricingConfigView.tsx
- tests/e2e/features/mcp-tool-groups/pages/mcp-tool-groups.page.ts
- tests/e2e/features/model-limits/model-limits.spec.ts
- tests/e2e/features/routing-rules/pages/routing-rules.page.ts
Radheshg04
force-pushed
the
02-27-feat_extend_e2e_ui_tests
branch
from
2ef6161 to
bc61c65
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
tests/e2e/features/providers/pages/providers.page.ts (1)
184-195:⚠️ Potential issue | 🟠 MajorScope the confirmation click to the alert dialog.
Line 194 uses a global
Deletebutton locator; this can hit the wrong button when both panel and dialog actions are present.🔧 Suggested fix
- await this.page.getByRole('button', { name: 'Delete' }).click() + const confirmBtn = this.page + .locator('[role="alertdialog"]') + .getByRole('button', { name: /^Delete$/ }) + await confirmBtn.waitFor({ state: 'visible', timeout: 5000 }) + await confirmBtn.click()🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/providers/pages/providers.page.ts` around lines 184 - 195, The confirmation click in deleteProvider currently uses a global getByRole('button', { name: 'Delete' }) which can target the wrong button; scope the confirmation to the alert/dialog before clicking (e.g., find the dialog via this.page.getByRole('dialog') or a dialog-specific locator and then call .getByRole('button', { name: 'Delete' }) on that dialog), keeping the rest of deleteProvider (selectProvider and deleteBtn) unchanged.tests/e2e/features/providers/providers.spec.ts (1)
468-499:⚠️ Potential issue | 🟠 MajorMove debugging config restoration into a
finallyblock.On Lines 479-486, if the test fails after toggling and before the restore block, debugging flags can leak into later tests/suites.
🔧 Suggested fix
test('should toggle and save raw request/response', async ({ providersPage }) => { await providersPage.selectConfigTab('debugging') @@ - // Toggle both switches - await rawRequestSwitch.click() - await rawResponseSwitch.click() - - // Save and verify success - const saveBtn = providersPage.getConfigSaveBtn('debugging') - await expect(saveBtn).toBeEnabled() - await providersPage.saveDebuggingConfig() - - // Restore original states - const currentRawRequest = await rawRequestSwitch.getAttribute('data-state') === 'checked' - const currentRawResponse = await rawResponseSwitch.getAttribute('data-state') === 'checked' - - if (currentRawRequest !== originalRawRequest) { - await rawRequestSwitch.click() - } - if (currentRawResponse !== originalRawResponse) { - await rawResponseSwitch.click() - } - - await providersPage.saveDebuggingConfig() + try { + // Toggle both switches + await rawRequestSwitch.click() + await rawResponseSwitch.click() + + // Save and verify success + const saveBtn = providersPage.getConfigSaveBtn('debugging') + await expect(saveBtn).toBeEnabled() + await providersPage.saveDebuggingConfig() + } finally { + // Always restore original states + const currentRawRequest = await rawRequestSwitch.getAttribute('data-state') === 'checked' + const currentRawResponse = await rawResponseSwitch.getAttribute('data-state') === 'checked' + + if (currentRawRequest !== originalRawRequest) { + await rawRequestSwitch.click() + } + if (currentRawResponse !== originalRawResponse) { + await rawResponseSwitch.click() + } + + const restoreSaveBtn = providersPage.getConfigSaveBtn('debugging') + if (await restoreSaveBtn.isEnabled()) { + await providersPage.saveDebuggingConfig() + } + } })🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/providers/providers.spec.ts` around lines 468 - 499, The test "should toggle and save raw request/response" currently toggles switches returned by providersPage.getRawRequestSwitch() and providersPage.getRawResponseSwitch() and only restores their original states at the end of the happy path; move the restoration logic into a try...finally so that after toggling and calling providersPage.saveDebuggingConfig() you always revert to the captured original states (using the same getRawRequestSwitch/getRawResponseSwitch and comparing data-state === 'checked') and call providersPage.saveDebuggingConfig() inside the finally block; ensure the save button check via providersPage.getConfigSaveBtn('debugging') still gates the initial save and that the final restore still clicks the switches only when current !== original.
♻️ Duplicate comments (2)
tests/e2e/features/governance/pages/governance.page.ts (1)
6-27:⚠️ Potential issue | 🟠 MajorConfig interfaces still advertise fields that current flows ignore.
budget.resetDurationandrateLimit.*are accepted inTeamConfig/CustomerConfig, but create/edit methods do not apply them. That silently drops caller intent and can produce false confidence in coverage.🔧 Minimal safe option until UI support lands
export interface TeamConfig { name: string customerId?: string - budget?: { maxLimit: number; resetDuration?: string } - rateLimit?: { - tokenMaxLimit?: number - tokenResetDuration?: string - requestMaxLimit?: number - requestResetDuration?: string - } + budget?: { maxLimit: number } } export interface CustomerConfig { name: string - budget?: { maxLimit: number; resetDuration?: string } - rateLimit?: { - tokenMaxLimit?: number - tokenResetDuration?: string - requestMaxLimit?: number - requestResetDuration?: string - } + budget?: { maxLimit: number } }Also applies to: 75-206
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/governance/pages/governance.page.ts` around lines 6 - 27, The TeamConfig and CustomerConfig interfaces advertise fields (budget.resetDuration and rateLimit.*) that are not applied by the create/edit flows; either remove those fields from the public interfaces or explicitly mark them unsupported and enforce that in the create/edit handlers (createTeam, editTeam, createCustomer, editCustomer) to avoid silently dropping caller intent: update TeamConfig and CustomerConfig to remove budget.resetDuration and all rateLimit.* properties (or add clear comments/Deprecated types), and add validation in the create/edit methods to reject or ignore those properties with a clear error/log so callers cannot assume they are applied.tests/e2e/features/providers/pages/providers.page.ts (1)
400-409:⚠️ Potential issue | 🟠 MajorUse test-id selectors for debugging save/switch controls in the page object.
getConfigSaveBtn('debugging'),getRawRequestSwitch(), andgetRawResponseSwitch()still depend on text/DOM structure, which is brittle.As per coding guidelines, "`tests/e2e/**/*.ts`: E2E test page objects must ... use getByTestId() as the primary selector strategy."🔧 Suggested refactor
getConfigSaveBtn(configType: 'network' | 'proxy' | 'performance' | 'governance' | 'debugging'): Locator { + if (configType === 'debugging') { + return this.page.getByTestId('provider-config-debugging-save-btn') + } const buttonNames: Record<string, string> = { network: 'Save Network Configuration', proxy: 'Save Proxy Configuration', performance: 'Save Performance Configuration', governance: 'Save Governance Configuration', - debugging: 'Save Debugging Configuration', } return this.page.getByRole('button', { name: buttonNames[configType] }) } getRawRequestSwitch(): Locator { - return this.page.getByLabel('Send Back Raw Request').locator('..').locator('button[role="switch"]') + return this.page.getByTestId('send-back-raw-request-switch') } getRawResponseSwitch(): Locator { - return this.page.getByLabel('Send Back Raw Response').locator('..').locator('button[role="switch"]') + return this.page.getByTestId('send-back-raw-response-switch') }Also applies to: 432-441
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/providers/pages/providers.page.ts` around lines 400 - 409, getConfigSaveBtn currently relies on visible button text which is brittle; change it to use test-id selectors instead (e.g., replace page.getByRole(..., { name: buttonNames[configType] }) with page.getByTestId('save-config-' + configType) or equivalent test ids), and update the tests and component markup to add those data-testid attributes for debugging/save controls. Similarly update getRawRequestSwitch() and getRawResponseSwitch() to use page.getByTestId('raw-request-switch') and page.getByTestId('raw-response-switch') (or the agreed test-id names) so all selectors in the page object use getByTestId as the primary strategy. Ensure buttonNames mapping is removed or kept only for backwards mapping if needed and that test fixtures/components include the matching data-testid attributes.
🧹 Nitpick comments (7)
ui/app/workspace/model-limits/views/modelLimitsTable.tsx (1)
130-157: Consider precomputing test-id fragments once per row.You currently normalize
model_name/providermultiple times. Computing once improves readability and avoids drift.Optional refactor
{modelConfigs?.map((config) => { + const modelPart = toTestIdPart(config.model_name); + const providerPart = toTestIdPart(config.provider || "all"); const isBudgetExhausted = config.budget?.max_limit && config.budget.max_limit > 0 && config.budget.current_usage >= config.budget.max_limit; @@ - <TableRow key={config.id} data-testid={`model-limit-row-${toTestIdPart(config.model_name)}-${toTestIdPart(config.provider || "all")}`} className={cn("group transition-colors", isExhausted && "bg-red-500/5 hover:bg-red-500/10")}> + <TableRow key={config.id} data-testid={`model-limit-row-${modelPart}-${providerPart}`} className={cn("group transition-colors", isExhausted && "bg-red-500/5 hover:bg-red-500/10")}> @@ - data-testid={`model-limit-button-edit-${toTestIdPart(config.model_name)}-${toTestIdPart(config.provider || "all")}`} + data-testid={`model-limit-button-edit-${modelPart}-${providerPart}`} @@ - data-testid={`model-limit-button-delete-${toTestIdPart(config.model_name)}-${toTestIdPart(config.provider || "all")}`} + data-testid={`model-limit-button-delete-${modelPart}-${providerPart}`}Also applies to: 306-319
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/model-limits/views/modelLimitsTable.tsx` around lines 130 - 157, Precompute the normalized test-id fragments once per row to avoid repeated calls to toTestIdPart and improve readability: inside the map over modelConfigs (in the block using modelConfigs?.map and returning <TableRow key={config.id} ...>), create local constants like modelTestId = toTestIdPart(config.model_name) and providerTestId = toTestIdPart(config.provider || "all") and then use those constants in the data-testid string (`model-limit-row-${modelTestId}-${providerTestId}`) and anywhere else in that row (also apply the same change to the similar code at lines 306-319); keep all existing logic (isBudgetExhausted, isRateLimitExhausted, percentages) unchanged.tests/e2e/features/routing-rules/routing-rules.spec.ts (1)
32-55: Consolidate duplicated nested hook work.The outer hooks already handle
goto()andcreatedRulescleanup. Duplicating them in the seeded describe adds extra navigation and maintenance drift risk; keep inner hooks focused on seed create/delete only.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/routing-rules/routing-rules.spec.ts` around lines 32 - 55, The inner describe duplicates outer hook work (navigation and createdRules cleanup); remove the redundant routingRulesPage.goto() call and any loop or clearing of createdRules from the inner test.beforeEach/test.afterEach so those hooks only create and delete the seeded rule. Specifically, keep using routingRulesPage.createRoutingRule(...) to create the seed (using seedRuleName) in the inner beforeEach, and in the inner afterEach only check routingRulesPage.ruleExists(seedRuleName) and call routingRulesPage.deleteRoutingRule(seedRuleName) as needed, leaving outer test.beforeEach/test.afterEach to manage navigation (routingRulesPage.goto()) and the createdRules array cleanup. Ensure seedRuleName is reset to null after deletion.ui/app/workspace/governance/views/customerDialog.tsx (1)
213-213: Use a 3-part test-id for the dialog container.Line 213 uses
customer-dialog, which misses the<entity>-<element>-<qualifier>convention. Prefer something likecustomer-dialog-contentand update any E2E references accordingly.As per coding guidelines,
ui/app/**/*.{tsx,ts}: E2E test data-testid attributes must follow the conventiondata-testid="<entity>-<element>-<qualifier>".🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/governance/views/customerDialog.tsx` at line 213, Replace the non-conforming test id on the DialogContent element: change the data-testid value currently set on DialogContent ("customer-dialog") to follow the entity-element-qualifier pattern (for example "customer-dialog-content"), update any E2E / test references that assert or query "customer-dialog" to the new value, and ensure the modified DialogContent JSX (component DialogContent) uses the new test id string everywhere it appears.tests/e2e/features/observability/observability.spec.ts (1)
5-16:originalStateis captured but never used for restoration.The
originalStatevariable is assigned inbeforeEach(Line 10) but theafterEachhook simply disables all connectors rather than restoring to the original state. Either useoriginalStateto selectively restore, or remove the unused variable.Option 1: Remove unused variable
test.describe('Observability', () => { - let originalState: ObservabilityState - test.beforeEach(async ({ observabilityPage }) => { await observabilityPage.goto() - // Capture original state for restoration - originalState = await observabilityPage.getCurrentState() })Option 2: Actually restore original state
test.afterEach(async ({ observabilityPage }) => { - // Restore original state - disable all connectors that weren't enabled before - await observabilityPage.disableAllConnectors() + // Restore connectors to their original state + await observabilityPage.restoreState(originalState) })This would require adding a
restoreStatemethod toObservabilityPage.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/observability/observability.spec.ts` around lines 5 - 16, The test captures originalState (type ObservabilityState) via observabilityPage.getCurrentState but never uses it; either remove the unused originalState variable or restore the state in afterEach by adding and invoking a restore method. Fix option A: delete the originalState variable and its assignment in test.beforeEach and keep await observabilityPage.disableAllConnectors() in test.afterEach. Fix option B: implement observabilityPage.restoreState(state: ObservabilityState) (or restoreState on ObservabilityPage) and replace await observabilityPage.disableAllConnectors() in test.afterEach with await observabilityPage.restoreState(originalState) so the test returns to the captured state.tests/e2e/features/observability/pages/observability.page.ts (1)
1-3: Page object importsexpectfrom@playwright/testinstead of the base fixture.The coding guidelines specify imports should come from
../../core/fixtures/base.fixture. While page objects legitimately needPageandLocatortypes from Playwright, theexpectused on Line 147 could be imported from the base fixture for consistency.However, since this is a page object (not a spec file) and the
expectis used for internal assertions within helper methods, this is acceptable. Consider aligning imports in a follow-up if desired.As per coding guidelines: "E2E test page objects must... import test/expect from ../../core/fixtures/base.fixture, never from
@playwright/test."🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/observability/pages/observability.page.ts` around lines 1 - 3, The file imports expect from '@playwright/test' instead of the shared fixture; change the import so Page and Locator still come from '@playwright/test' but expect is imported from ../../core/fixtures/base.fixture (replace the existing expect import), keeping all usages of expect (e.g., in the Observability page helper methods where expect is called) unchanged so the page object uses the base.fixture expect for consistency with E2E guidelines.tests/e2e/features/virtual-keys/virtual-keys.spec.ts (1)
121-123: Use page-object test-id locators for budget/rate-limit field assertions.These direct
#...selectors in spec code are brittle and bypass the page object abstraction. Expose these fields onVirtualKeysPagewithgetByTestId()and assert via those locators.♻️ Proposed refactor
- const budgetInput = virtualKeysPage.page.locator('#budgetMaxLimit') + const budgetInput = virtualKeysPage.budgetMaxLimitInput await expect(budgetInput).toHaveValue(String(SAMPLE_BUDGETS.small.maxLimit)) - const tokenLimitInput = virtualKeysPage.page.locator('#tokenMaxLimit') + const tokenLimitInput = virtualKeysPage.tokenMaxLimitInput await expect(tokenLimitInput).toHaveValue(String(SAMPLE_RATE_LIMITS.tokenOnly.tokenMaxLimit))// tests/e2e/features/virtual-keys/pages/virtual-keys.page.ts readonly budgetMaxLimitInput = this.page.getByTestId('budget-max-limit-input') readonly tokenMaxLimitInput = this.page.getByTestId('token-max-limit-input')As per coding guidelines
tests/e2e/**/*.ts: “E2E test page objects must ... use getByTestId() as the primary selector strategy.”Also applies to: 166-168
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/virtual-keys/virtual-keys.spec.ts` around lines 121 - 123, The spec uses brittle ID selectors (`#budgetMaxLimit`) instead of the page-object test-id locators; update VirtualKeysPage to expose readonly locators (e.g., budgetMaxLimitInput and tokenMaxLimitInput) using this.page.getByTestId('budget-max-limit-input') and getByTestId('token-max-limit-input'), then change the assertions in virtual-keys.spec.ts to use virtualKeysPage.budgetMaxLimitInput and virtualKeysPage.tokenMaxLimitInput with toHaveValue(String(...)) and similarly replace the other direct #... usages (lines noted around 166-168) with the new page-object properties.tests/e2e/features/governance/pages/governance.page.ts (1)
97-98: Scope create/save button lookups to the active dialog.Using global
this.page.getByRole(...)here is less deterministic. Preferthis.teamDialog/this.customerDialogscoped locators (or dialog-specific test IDs) for create/save actions.As per coding guidelines
tests/e2e/**/*.ts: “E2E test page objects must ... use getByTestId() as the primary selector strategy.”Also applies to: 140-141, 184-185, 202-203
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/governance/pages/governance.page.ts` around lines 97 - 98, The test uses a global locator this.page.getByRole(...) (saveBtn) which is nondeterministic; change the lookup to be scoped to the appropriate dialog (e.g., this.teamDialog or this.customerDialog) and switch to getByTestId() per guidelines so the Create/Save button is located like this.teamDialog.getByTestId(...) (or this.customerDialog.getByTestId(...)) before clicking; update the analogous occurrences around lines referenced (save/create at 140-141, 184-185, 202-203) to use dialog-scoped getByTestId locators instead of this.page.getByRole.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/e2e/features/model-limits/pages/model-limits.page.ts`:
- Around line 48-50: Update the JSDoc for the method that creates a model limit
to reflect current behavior: replace "selects the first model" with a note that
it selects config.modelName deterministically and state that it returns the
selected model name (config.modelName) for use in exists/edit/delete; reference
the function/method that performs the creation (createModelLimit or the create
method in model-limits.page.ts) so the comment accurately matches the
implementation.
- Around line 57-65: The selection is too broad and flaky: replace the generic
selectors used around fillSelect and the model multiselect (the combobox/select
logic and the modelInput/getByPlaceholder + targetOption logic) with stable
test-id based selectors (use this.sheet.getByTestId for the provider combobox
and the model list container), perform exact option matching rather than global
role lookup, and remove the use of `.first()` so you query options scoped to the
specific combobox/list; update calls that reference fillSelect, modelInput, and
targetOption to use getByTestId-scoped queries and exact name matching for
deterministic selection.
In `@ui/app/workspace/model-limits/views/modelLimitsTable.tsx`:
- Line 118: The data-testid on the table container in modelLimitsTable.tsx uses
a 2-part id; update the div with data-testid="model-limits-table" to a 3-part id
following the convention (for example data-testid="model-limits-table-main") and
update any tests that reference the old id to the new three-part value so
selectors remain in sync; locate the container div in the ModelLimitsTable
component (the <div ... data-testid="model-limits-table">) and replace the value
accordingly.
---
Outside diff comments:
In `@tests/e2e/features/providers/pages/providers.page.ts`:
- Around line 184-195: The confirmation click in deleteProvider currently uses a
global getByRole('button', { name: 'Delete' }) which can target the wrong
button; scope the confirmation to the alert/dialog before clicking (e.g., find
the dialog via this.page.getByRole('dialog') or a dialog-specific locator and
then call .getByRole('button', { name: 'Delete' }) on that dialog), keeping the
rest of deleteProvider (selectProvider and deleteBtn) unchanged.
In `@tests/e2e/features/providers/providers.spec.ts`:
- Around line 468-499: The test "should toggle and save raw request/response"
currently toggles switches returned by providersPage.getRawRequestSwitch() and
providersPage.getRawResponseSwitch() and only restores their original states at
the end of the happy path; move the restoration logic into a try...finally so
that after toggling and calling providersPage.saveDebuggingConfig() you always
revert to the captured original states (using the same
getRawRequestSwitch/getRawResponseSwitch and comparing data-state === 'checked')
and call providersPage.saveDebuggingConfig() inside the finally block; ensure
the save button check via providersPage.getConfigSaveBtn('debugging') still
gates the initial save and that the final restore still clicks the switches only
when current !== original.
---
Duplicate comments:
In `@tests/e2e/features/governance/pages/governance.page.ts`:
- Around line 6-27: The TeamConfig and CustomerConfig interfaces advertise
fields (budget.resetDuration and rateLimit.*) that are not applied by the
create/edit flows; either remove those fields from the public interfaces or
explicitly mark them unsupported and enforce that in the create/edit handlers
(createTeam, editTeam, createCustomer, editCustomer) to avoid silently dropping
caller intent: update TeamConfig and CustomerConfig to remove
budget.resetDuration and all rateLimit.* properties (or add clear
comments/Deprecated types), and add validation in the create/edit methods to
reject or ignore those properties with a clear error/log so callers cannot
assume they are applied.
In `@tests/e2e/features/providers/pages/providers.page.ts`:
- Around line 400-409: getConfigSaveBtn currently relies on visible button text
which is brittle; change it to use test-id selectors instead (e.g., replace
page.getByRole(..., { name: buttonNames[configType] }) with
page.getByTestId('save-config-' + configType) or equivalent test ids), and
update the tests and component markup to add those data-testid attributes for
debugging/save controls. Similarly update getRawRequestSwitch() and
getRawResponseSwitch() to use page.getByTestId('raw-request-switch') and
page.getByTestId('raw-response-switch') (or the agreed test-id names) so all
selectors in the page object use getByTestId as the primary strategy. Ensure
buttonNames mapping is removed or kept only for backwards mapping if needed and
that test fixtures/components include the matching data-testid attributes.
---
Nitpick comments:
In `@tests/e2e/features/governance/pages/governance.page.ts`:
- Around line 97-98: The test uses a global locator this.page.getByRole(...)
(saveBtn) which is nondeterministic; change the lookup to be scoped to the
appropriate dialog (e.g., this.teamDialog or this.customerDialog) and switch to
getByTestId() per guidelines so the Create/Save button is located like
this.teamDialog.getByTestId(...) (or this.customerDialog.getByTestId(...))
before clicking; update the analogous occurrences around lines referenced
(save/create at 140-141, 184-185, 202-203) to use dialog-scoped getByTestId
locators instead of this.page.getByRole.
In `@tests/e2e/features/observability/observability.spec.ts`:
- Around line 5-16: The test captures originalState (type ObservabilityState)
via observabilityPage.getCurrentState but never uses it; either remove the
unused originalState variable or restore the state in afterEach by adding and
invoking a restore method. Fix option A: delete the originalState variable and
its assignment in test.beforeEach and keep await
observabilityPage.disableAllConnectors() in test.afterEach. Fix option B:
implement observabilityPage.restoreState(state: ObservabilityState) (or
restoreState on ObservabilityPage) and replace await
observabilityPage.disableAllConnectors() in test.afterEach with await
observabilityPage.restoreState(originalState) so the test returns to the
captured state.
In `@tests/e2e/features/observability/pages/observability.page.ts`:
- Around line 1-3: The file imports expect from '@playwright/test' instead of
the shared fixture; change the import so Page and Locator still come from
'@playwright/test' but expect is imported from ../../core/fixtures/base.fixture
(replace the existing expect import), keeping all usages of expect (e.g., in the
Observability page helper methods where expect is called) unchanged so the page
object uses the base.fixture expect for consistency with E2E guidelines.
In `@tests/e2e/features/routing-rules/routing-rules.spec.ts`:
- Around line 32-55: The inner describe duplicates outer hook work (navigation
and createdRules cleanup); remove the redundant routingRulesPage.goto() call and
any loop or clearing of createdRules from the inner
test.beforeEach/test.afterEach so those hooks only create and delete the seeded
rule. Specifically, keep using routingRulesPage.createRoutingRule(...) to create
the seed (using seedRuleName) in the inner beforeEach, and in the inner
afterEach only check routingRulesPage.ruleExists(seedRuleName) and call
routingRulesPage.deleteRoutingRule(seedRuleName) as needed, leaving outer
test.beforeEach/test.afterEach to manage navigation (routingRulesPage.goto())
and the createdRules array cleanup. Ensure seedRuleName is reset to null after
deletion.
In `@tests/e2e/features/virtual-keys/virtual-keys.spec.ts`:
- Around line 121-123: The spec uses brittle ID selectors (`#budgetMaxLimit`)
instead of the page-object test-id locators; update VirtualKeysPage to expose
readonly locators (e.g., budgetMaxLimitInput and tokenMaxLimitInput) using
this.page.getByTestId('budget-max-limit-input') and
getByTestId('token-max-limit-input'), then change the assertions in
virtual-keys.spec.ts to use virtualKeysPage.budgetMaxLimitInput and
virtualKeysPage.tokenMaxLimitInput with toHaveValue(String(...)) and similarly
replace the other direct #... usages (lines noted around 166-168) with the new
page-object properties.
In `@ui/app/workspace/governance/views/customerDialog.tsx`:
- Line 213: Replace the non-conforming test id on the DialogContent element:
change the data-testid value currently set on DialogContent ("customer-dialog")
to follow the entity-element-qualifier pattern (for example
"customer-dialog-content"), update any E2E / test references that assert or
query "customer-dialog" to the new value, and ensure the modified DialogContent
JSX (component DialogContent) uses the new test id string everywhere it appears.
In `@ui/app/workspace/model-limits/views/modelLimitsTable.tsx`:
- Around line 130-157: Precompute the normalized test-id fragments once per row
to avoid repeated calls to toTestIdPart and improve readability: inside the map
over modelConfigs (in the block using modelConfigs?.map and returning <TableRow
key={config.id} ...>), create local constants like modelTestId =
toTestIdPart(config.model_name) and providerTestId =
toTestIdPart(config.provider || "all") and then use those constants in the
data-testid string (`model-limit-row-${modelTestId}-${providerTestId}`) and
anywhere else in that row (also apply the same change to the similar code at
lines 306-319); keep all existing logic (isBudgetExhausted,
isRateLimitExhausted, percentages) unchanged.
ℹ️ Review info
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (53)
.github/workflows/e2e-tests.ymltests/e2e/core/fixtures/base.fixture.tstests/e2e/features/config/config.spec.tstests/e2e/features/config/pages/config-settings.page.tstests/e2e/features/dashboard/dashboard.spec.tstests/e2e/features/governance/governance.data.tstests/e2e/features/governance/governance.spec.tstests/e2e/features/governance/pages/governance.page.tstests/e2e/features/mcp-auth-config/mcp-auth-config.spec.tstests/e2e/features/mcp-auth-config/pages/mcp-auth-config.page.tstests/e2e/features/mcp-registry/mcp-registry.spec.tstests/e2e/features/mcp-registry/pages/mcp-registry.page.tstests/e2e/features/mcp-settings/mcp-settings.spec.tstests/e2e/features/mcp-settings/pages/mcp-settings.page.tstests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.tstests/e2e/features/mcp-tool-groups/pages/mcp-tool-groups.page.tstests/e2e/features/model-limits/model-limits.data.tstests/e2e/features/model-limits/model-limits.spec.tstests/e2e/features/model-limits/pages/model-limits.page.tstests/e2e/features/observability/observability.spec.tstests/e2e/features/observability/pages/observability.page.tstests/e2e/features/placeholders/placeholders.spec.tstests/e2e/features/plugins/pages/plugins.page.tstests/e2e/features/plugins/plugins.spec.tstests/e2e/features/providers/pages/providers.page.tstests/e2e/features/providers/providers.spec.tstests/e2e/features/routing-rules/pages/routing-rules.page.tstests/e2e/features/routing-rules/routing-rules.spec.tstests/e2e/features/virtual-keys/pages/virtual-keys.page.tstests/e2e/features/virtual-keys/virtual-keys.spec.tsui/app/workspace/config/views/clientSettingsView.tsxui/app/workspace/config/views/mcpView.tsxui/app/workspace/config/views/pricingConfigView.tsxui/app/workspace/config/views/securityView.tsxui/app/workspace/governance/views/customerDialog.tsxui/app/workspace/governance/views/customerTable.tsxui/app/workspace/governance/views/teamDialog.tsxui/app/workspace/governance/views/teamsTable.tsxui/app/workspace/mcp-registry/views/mcpClientsTable.tsxui/app/workspace/mcp-registry/views/mcpServersEmptyState.tsxui/app/workspace/model-limits/views/modelLimitSheet.tsxui/app/workspace/model-limits/views/modelLimitsEmptyState.tsxui/app/workspace/model-limits/views/modelLimitsTable.tsxui/app/workspace/plugins/fragments/pluginFormFragments.tsxui/app/workspace/plugins/views/pluginsEmptyState.tsxui/app/workspace/providers/fragments/debuggingFormFragment.tsxui/app/workspace/providers/views/modelProviderKeysTableView.tsxui/app/workspace/providers/views/providersEmptyState.tsxui/app/workspace/routing-rules/views/routingRulesEmptyState.tsxui/app/workspace/routing-rules/views/routingRulesTable.tsxui/app/workspace/virtual-keys/views/virtualKeysEmptyState.tsxui/app/workspace/virtual-keys/views/virtualKeysTable.tsxui/components/ui/numberAndSelect.tsx
🚧 Files skipped from review as they are similar to previous changes (25)
- tests/e2e/features/mcp-auth-config/mcp-auth-config.spec.ts
- ui/app/workspace/config/views/clientSettingsView.tsx
- ui/app/workspace/virtual-keys/views/virtualKeysEmptyState.tsx
- tests/e2e/features/plugins/plugins.spec.ts
- tests/e2e/features/model-limits/model-limits.spec.ts
- ui/app/workspace/providers/views/modelProviderKeysTableView.tsx
- ui/app/workspace/providers/fragments/debuggingFormFragment.tsx
- ui/app/workspace/plugins/views/pluginsEmptyState.tsx
- ui/components/ui/numberAndSelect.tsx
- tests/e2e/features/mcp-settings/pages/mcp-settings.page.ts
- tests/e2e/features/mcp-tool-groups/pages/mcp-tool-groups.page.ts
- ui/app/workspace/model-limits/views/modelLimitsEmptyState.tsx
- ui/app/workspace/config/views/pricingConfigView.tsx
- ui/app/workspace/governance/views/customerTable.tsx
- ui/app/workspace/virtual-keys/views/virtualKeysTable.tsx
- tests/e2e/features/virtual-keys/pages/virtual-keys.page.ts
- ui/app/workspace/governance/views/teamDialog.tsx
- ui/app/workspace/governance/views/teamsTable.tsx
- ui/app/workspace/routing-rules/views/routingRulesEmptyState.tsx
- .github/workflows/e2e-tests.yml
- ui/app/workspace/mcp-registry/views/mcpServersEmptyState.tsx
- ui/app/workspace/mcp-registry/views/mcpClientsTable.tsx
- ui/app/workspace/model-limits/views/modelLimitSheet.tsx
- ui/app/workspace/routing-rules/views/routingRulesTable.tsx
- ui/app/workspace/providers/views/providersEmptyState.tsx
Radheshg04
force-pushed
the
02-27-feat_extend_e2e_ui_tests
branch
3 times, most recently
from
98f8fcb to
53a87f2
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 7
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
ui/app/workspace/config/views/mcpView.tsx (1)
163-170:⚠️ Potential issue | 🟡 MinorAdd a missing
data-testidon Tool Sync Interval input.Line 163’s interactive
Inputis the only MCP settings control in this section without a test selector.Suggested patch
<Input id="mcp-tool-sync-interval" + data-testid="mcp-tool-sync-interval-input" type="number" className="w-24" value={localValues.mcp_tool_sync_interval}As per coding guidelines,
ui/app/workspace/**/*.tsx“must include data-testid attributes on all interactive elements.”🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/config/views/mcpView.tsx` around lines 163 - 170, Add a data-testid to the Input for the Tool Sync Interval so automated tests can target it; locate the Input with id "mcp-tool-sync-interval" (the one bound to value localValues.mcp_tool_sync_interval and onChange handleToolSyncIntervalChange) and add a data-testid attribute (e.g. data-testid="mcp-tool-sync-interval") to the element consistent with other MCP settings controls.tests/e2e/features/virtual-keys/virtual-keys.spec.ts (1)
253-261:⚠️ Potential issue | 🟠 MajorTrack the original VK before rename to avoid cleanup leaks on failure.
If
editVirtualKeyfails, only the updated name is tracked and the original key is left behind.Suggested fix
- await virtualKeysPage.createVirtualKey(vkData) - - // Now edit it - const updatedName = `${originalName} Updated` - managementVKs.push(updatedName) // Track the updated name for cleanup + await virtualKeysPage.createVirtualKey(vkData) + managementVKs.push(originalName) + + // Now edit it + const updatedName = `${originalName} Updated` await virtualKeysPage.editVirtualKey(originalName, { name: updatedName, }) + + const originalIdx = managementVKs.indexOf(originalName) + if (originalIdx >= 0) managementVKs.splice(originalIdx, 1) + managementVKs.push(updatedName)As per coding guidelines,
tests/e2e/**/*.tsrequires tracking created resources and cleaning them up inafterEach.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/virtual-keys/virtual-keys.spec.ts` around lines 253 - 261, The test currently only pushes the updated name into managementVKs before calling virtualKeysPage.editVirtualKey, which leaves the original key untracked if the edit fails; after creating the VK with virtualKeysPage.createVirtualKey(vkData) immediately push originalName into managementVKs to ensure the created resource is tracked, then after a successful edit replace that entry with updatedName (e.g., find the index of originalName in managementVKs and set it to updatedName) so cleanup in afterEach will remove the correct key whether the rename succeeds or not.tests/e2e/features/providers/pages/providers.page.ts (1)
193-204:⚠️ Potential issue | 🟠 MajorScope the destructive confirm click to the alert dialog.
page.getByRole('button', { name: 'Delete' })is global and can click the wrong Delete button if multiple are present.Suggested fix
- // Confirm deletion in dialog - await this.page.getByRole('button', { name: 'Delete' }).click() + // Confirm deletion in dialog (scoped) + const confirmBtn = this.page.locator('[role="alertdialog"]').getByRole('button', { name: /^Delete$/ }) + await confirmBtn.waitFor({ state: 'visible', timeout: 5000 }) + await confirmBtn.click()🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/providers/pages/providers.page.ts` around lines 193 - 204, In deleteProvider, the final confirmation click uses a global locator this.page.getByRole('button', { name: 'Delete' }) which may hit the wrong button; scope the click to the confirmation dialog by locating the dialog/alertdialog first (e.g., use this.page.getByRole('alertdialog' or 'dialog') or a dialog test id) and then call getByRole('button', { name: 'Delete' }) on that dialog locator so the click targets the dialog's Delete button rather than any other Delete button on the page.
♻️ Duplicate comments (3)
tests/e2e/features/observability/pages/observability.page.ts (1)
52-55:⚠️ Potential issue | 🟠 MajorAvoid nondeterministic toggle fallback on Line 54.
Using
.first()for unknown connectors can target the wrong switch and make connector state checks flaky. Keep this method strictly connector-scoped via mapped test ids only.Suggested fix
getConnectorToggle(connector: ObservabilityConnector): Locator { const testId = ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS[connector] - return testId ? this.page.getByTestId(testId) : this.page.locator('button[role="switch"]').first() + if (!testId) { + throw new Error(`Missing toggle test id mapping for connector: ${connector}`) + } + return this.page.getByTestId(testId) }As per coding guidelines "
tests/e2e/**/*.ts: E2E test page objects must extend BasePage, use getByTestId() as the primary selector strategy..."🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/observability/pages/observability.page.ts` around lines 52 - 55, The getConnectorToggle method currently falls back to this.page.locator('button[role="switch"]').first() when ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS[connector] is missing, which causes nondeterministic test behavior; update getConnectorToggle (in ObservabilityPage) to only use this.page.getByTestId(testId) and remove the .first() fallback—if testId is undefined, throw a clear error (or assert) indicating the connector mapping is missing so tests fail fast and remain connector-scoped.tests/e2e/features/providers/pages/providers.page.ts (1)
409-418:⚠️ Potential issue | 🟠 MajorDebugging controls still don’t use test-id-first selectors (already flagged).
getConfigSaveBtn('debugging')and raw request/response switch locators still rely on role/label traversal rather thangetByTestId().As per coding guidelines,
tests/e2e/**/*.tspage objects must usegetByTestId()as the primary selector strategy.Also applies to: 438-450
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/providers/pages/providers.page.ts` around lines 409 - 418, The getConfigSaveBtn function currently uses role/label mapping; change it to use test-id-first selectors by replacing the role-based lookup in getConfigSaveBtn(configType) with this.page.getByTestId(...) using stable test IDs for each config type (e.g., test ids like 'save-network-config', 'save-proxy-config', 'save-performance-config', 'save-governance-config', 'save-debugging-config'); also update the raw request/response switch locators referenced in the same page object (the other locators mentioned around the request/response switch) to use this.page.getByTestId() as primary selectors and fall back to role/label only if the test-id is missing.tests/e2e/features/model-limits/pages/model-limits.page.ts (1)
47-51:⚠️ Potential issue | 🟡 MinorJSDoc describes outdated behavior.
The documentation states "picks the first model in the search dropdown" but should describe deterministic selection using
config.modelName(once the fix above is applied).Proposed fix
/** - * Create a model limit via the sheet: selects provider, picks the first model - * in the search dropdown, fills budget and rate limit, then saves. Returns - * the selected model name for use in exists/edit/delete. + * Create a model limit via the sheet: selects provider, searches for and selects + * the model matching config.modelName, fills budget and rate limit, then saves. + * Returns the selected model name for use in exists/edit/delete. */🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/model-limits/pages/model-limits.page.ts` around lines 47 - 51, Update the JSDoc block above the function that creates a model limit to reflect current deterministic behavior: replace the phrase "picks the first model in the search dropdown" with a description that the sheet selects the model specified by config.modelName (i.e., deterministic selection using config.modelName), and adjust the summary to mention provider selection, filling budget/rate limits, saving, and returning the selected model name for use in exists/edit/delete; ensure the comment references config.modelName explicitly.
🧹 Nitpick comments (7)
ui/app/workspace/routing-rules/views/routingRulesView.tsx (1)
65-67: Align thisdata-testidwith the project naming convention.
create-routing-rule-btnis action-first; the UI guideline expects<entity>-<element>-<qualifier>. Prefer something likerouting-rule-create-btn(and update E2E/page-object references accordingly).Suggested patch
- data-testid="create-routing-rule-btn" + data-testid="routing-rule-create-btn"As per coding guidelines,
ui/app/**/*.{tsx,ts}: “E2E test data-testid attributes must follow the convention 'data-testid="--"'. If you rename or remove a data-testid, search tests/e2e/ for references.”🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/routing-rules/views/routingRulesView.tsx` around lines 65 - 67, Update the data-testid on the Button in routingRulesView.tsx from "create-routing-rule-btn" to follow the entity-element-qualifier convention, e.g. "routing-rule-create-btn"; locate the Button with the onClick handler handleCreateNew and change its data-testid attribute, then search and update any E2E/page-object references in tests/e2e that use the old test id.tests/e2e/features/config/config.spec.ts (4)
418-421: Assertion doesn't verify actual visibility.
expect(isVisible).toBeDefined()always passes since a boolean is always defined. This doesn't actually verify whether the rate limiting section is visible.♻️ Suggested fix
test('should display rate limiting section', async ({ configSettingsPage }) => { const isVisible = await configSettingsPage.isRateLimitingSectionVisible() - expect(isVisible).toBeDefined() + if (!isVisible) { + test.skip(true, 'Rate limiting section not available') + return + } + expect(isVisible).toBe(true) })🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/config/config.spec.ts` around lines 418 - 421, The test "should display rate limiting section" currently asserts with expect(isVisible).toBeDefined(), which doesn't verify visibility; update the assertion to assert the boolean truthiness of isVisible (e.g., expect(isVisible).toBe(true) or expect(isVisible).toBeTruthy()) so the call to configSettingsPage.isRateLimitingSectionVisible() is actually validated; keep the test name and the isRateLimitingSectionVisible() helper unchanged.
318-360: Addtest.skip()when log retention input is unavailable.These tests silently pass without executing assertions when
isVisibleis false. This masks environment issues and produces misleading test reports where tests appear to pass but didn't actually verify anything.♻️ Suggested fix
test('should change log retention days', async ({ configSettingsPage }) => { const retentionInput = configSettingsPage.logRetentionDaysInput const isVisible = await retentionInput.isVisible().catch(() => false) - if (isVisible) { + if (!isVisible) { + test.skip(true, 'Log retention input not available') + return + } + const originalValue = await retentionInput.inputValue() // ... rest of test logic - } })Apply the same pattern to
should save and persist log retention days.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/config/config.spec.ts` around lines 318 - 360, Both tests ('should change log retention days' and 'should save and persist log retention days') currently no-op when logRetentionDaysInput.isVisible() is false; update each test to call test.skip() when the input is not visible (e.g., if (!isVisible) test.skip('log retention input not available in this environment')), then proceed with the existing logic that uses configSettingsPage.logRetentionDaysInput, configSettingsPage.hasPendingChanges(), configSettingsPage.saveSettings(), and configSettingsPage.goto('logging') so the test either explicitly skips or runs assertions as intended.
483-500: Consider consolidating with existing Pricing Config block.Lines 73-136 already contain a comprehensive
Pricing Configtest block with detailed tests including view visibility (line 94). ThisPricing Config Settingsblock adds state capture/restore overhead but only verifies the heading is visible, which provides no additional coverage.Consider either removing this block or expanding it with distinct test cases not covered by the existing
Pricing Configblock.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/config/config.spec.ts` around lines 483 - 500, The new test.describe block named 'Pricing Config Settings' duplicates existing 'Pricing Config' coverage and only checks the heading while adding state capture/restore via originalState, configSettingsPage.getCurrentSettings, and configSettingsPage.restoreSettings; remove this redundant test.describe block entirely or replace it with additional distinct assertions (e.g., interact with specific pricing toggles or save/restore behavior) so it provides unique coverage beyond the existing Pricing Config tests.
442-480: Addtest.skip()when worker pool input is unavailable.Same issue as the log retention tests—these silently pass without assertions when the input is not visible.
♻️ Suggested fix
test('should change worker pool size', async ({ configSettingsPage }) => { const workerPoolInput = configSettingsPage.workerPoolSizeInput const isVisible = await workerPoolInput.isVisible().catch(() => false) - if (isVisible) { + if (!isVisible) { + test.skip(true, 'Worker pool size input not available') + return + } + const originalValue = await workerPoolInput.inputValue() // ... rest of test logic - } })Apply the same pattern to
should save and persist worker pool size.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/config/config.spec.ts` around lines 442 - 480, For both tests "should change worker pool size" and "should save and persist worker pool size" replace the current conditional that wraps the whole test body with an explicit runtime skip when the input is not visible: call const isVisible = await configSettingsPage.workerPoolSizeInput.isVisible().catch(() => false) then invoke test.skip(!isVisible, 'worker pool input not visible') (and return) so the test is marked skipped instead of silently passing; keep the remainder of the existing test logic (reading originalValue, computing newValue, clearing/filling, and in the persist test calling configSettingsPage.saveSettings() and reload) unchanged.tests/e2e/features/observability/observability.spec.ts (1)
177-179: Avoid hard-coded provider display names in assertions.Use
ProviderLabelsas the source of truth for connector labels instead of literal strings ('Prometheus','BigQuery') to prevent drift when UI labels change.Based on learnings "In E2E tests, use the authoritative ProviderLabels constant exported from ui/lib/constants/logs.ts as the source of truth for provider display names... Do not hard-code strings."
Also applies to: 254-256
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/observability/observability.spec.ts` around lines 177 - 179, The test is asserting against hard-coded provider display names; replace literal strings with the authoritative ProviderLabels constants: import ProviderLabels from 'ui/lib/constants/logs' (or the correct named export) and change expect(selected).toContain('Prometheus') to expect(selected).toContain(ProviderLabels.PROMETHEUS) (and similarly replace 'BigQuery' at the other occurrence with ProviderLabels.BIGQUERY); locate the assertions around observabilityPage.getSelectedConnector() in observability.spec.ts and update both places (the block at ~177-179 and the one at ~254-256) to use the ProviderLabels keys.tests/e2e/features/model-limits/pages/model-limits.page.ts (1)
73-83: Consider usinggetByTestId()for form inputs.The budget and rate limit inputs use ID selectors (
#modelBudgetMaxLimit,#modelTokenMaxLimit,#modelRequestMaxLimit). Per project conventions,getByTestId()is preferred as the primary selector strategy for maintainability.Example refactor
if (config.budget?.maxLimit !== undefined) { - const budgetInput = this.page.locator('#modelBudgetMaxLimit') + const budgetInput = this.page.getByTestId('model-budget-max-limit') await budgetInput.fill(String(config.budget.maxLimit)) }This would require adding corresponding
data-testidattributes in the UI components.As per coding guidelines: "E2E test page objects must use getByTestId() as the primary selector strategy."
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/model-limits/pages/model-limits.page.ts` around lines 73 - 83, The three locators use ID selectors instead of the project's preferred getByTestId selector; update the calls that reference '#modelBudgetMaxLimit', '#modelTokenMaxLimit', and '#modelRequestMaxLimit' (and the local variable budgetInput) to use this.page.getByTestId('modelBudgetMaxLimit'), this.page.getByTestId('modelTokenMaxLimit'), and this.page.getByTestId('modelRequestMaxLimit') respectively, then call .fill(String(...)) on those test-id-based locators; also add corresponding data-testid="modelBudgetMaxLimit"/"modelTokenMaxLimit"/"modelRequestMaxLimit" attributes to the UI form inputs so the tests can target them.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/e2e/features/governance/pages/governance.page.ts`:
- Around line 97-100: The clicks querying the whole page are flaky because they
can target buttons outside the active dialog; update the selectors to scope to
the active dialog element (this.teamDialog) and prefer getByTestId() as the
primary selector: replace this.page.getByRole(...) and similar page-level
queries with this.teamDialog.getByTestId(...) or this.teamDialog.getByRole(...)
for the Create Team (saveBtn), Cancel/Close buttons and other dialog actions
referenced in this file so the click/assert operations only interact with the
active dialog.
- Around line 103-110: The deleteTeam helper returns immediately after the
success toast which can race with table refresh; update deleteTeam to locate the
team row (e.g., const row = this.page.locator(`tr:has-text("${name}")`) or a
test-id for that row) and after await this.waitForSuccessToast() await
row.waitFor({ state: 'detached', timeout: 5000 }) (or 'hidden' if appropriate)
to ensure the row is removed before returning; apply the same change to the
other delete helper in this file that follows the same pattern so callers
checking exists === false are no longer flaky.
In `@tests/e2e/features/mcp-registry/mcp-registry.spec.ts`:
- Around line 87-90: Add an explicit auth precondition guard to the SSE test so
it doesn't hard-fail in CI: before creating or verifying SSE (the test that
calls mcpRegistryPage.getClientConnectionType), query an auth-status helper
(e.g., an existing mcpRegistryPage.isAuthConfigured / authClient.isAuthenticated
/ testHelpers.getAuthToken) and if auth is missing either skip the test or throw
a clear skip/fail message; ensure the guard runs at the top of the spec (before
SSE creation/assertion) so the rest of the test (including
getClientConnectionType(clientData.name)) only runs when auth is available.
In `@tests/e2e/features/model-limits/pages/model-limits.page.ts`:
- Around line 64-71: The test currently clicks the first dropdown option
(firstOption) and reads its text into selectedModelName, which is
non-deterministic; modify the selection logic in the modelSelectContainer block
to locate the option whose visible text exactly matches config.modelName (use a
locator/query scoped to modelSelectContainer or role='option' with the text),
wait for that specific option to be visible, click it, and set selectedModelName
from that matched option instead of firstOption; ensure you still await the
option selector (timeout) so the test remains stable and, after this change,
update model-limits.data.ts to provide a valid default config.modelName if
needed.
In `@tests/e2e/features/observability/pages/observability.page.ts`:
- Around line 136-148: The toggleConnector method currently uses
expect(toggle).toHaveAttribute(...) and likely imports expect; remove that
direct expect usage and the expect import, and instead call the
BasePage.waitForStateChange helper to wait for the toggle's 'data-state'
attribute to become the new value: compute previousState via
getAttribute('data-state'), derive expectedState ('checked' <-> 'unchecked'),
click the locator returned by getConnectorToggle(connector), then call
BasePage.waitForStateChange(toggle, 'data-state', expectedState, { timeout: 5000
}) (or the helper's exact parameter order) and return true; keep early returns
for visibility/disabled checks intact.
In `@tests/e2e/features/plugins/plugins.spec.ts`:
- Around line 33-41: The bootstrap plugin creation may fail silently because
when pluginsPage.createPlugin(pluginData) returns false the test continues;
update the block that checks pluginsPage.getPluginCount and calls
pluginsPage.createPlugin so that if createPlugin returns false you skip the test
immediately (use the same pattern used elsewhere in this spec, e.g. call
test.skip or the existing skip helper), and only push pluginData.name onto
createdPlugins when createPlugin returned truthy; reference
pluginsPage.getPluginCount, pluginsPage.createPlugin, and the createdPlugins
array to locate the code to change.
In `@tests/e2e/features/providers/providers.spec.ts`:
- Around line 114-128: The test assumes Nebius has zero keys which makes it
flaky; after selecting the provider (providersPage.selectProvider('nebius'))
call providersPage.getKeyCount() and branch: if keyCount === 0 then assert
providersPage.keysTableEmptyState is visible, otherwise skip the empty-state
assertion (or create/use an isolated test provider instead). Update the test
around
providerExists/addKnownProviderFromDropdown/getProviderItem/selectProvider/getKeyCount
to either create an isolated provider for the empty-state scenario or guard the
assertion with a conditional based on getKeyCount.
---
Outside diff comments:
In `@tests/e2e/features/providers/pages/providers.page.ts`:
- Around line 193-204: In deleteProvider, the final confirmation click uses a
global locator this.page.getByRole('button', { name: 'Delete' }) which may hit
the wrong button; scope the click to the confirmation dialog by locating the
dialog/alertdialog first (e.g., use this.page.getByRole('alertdialog' or
'dialog') or a dialog test id) and then call getByRole('button', { name:
'Delete' }) on that dialog locator so the click targets the dialog's Delete
button rather than any other Delete button on the page.
In `@tests/e2e/features/virtual-keys/virtual-keys.spec.ts`:
- Around line 253-261: The test currently only pushes the updated name into
managementVKs before calling virtualKeysPage.editVirtualKey, which leaves the
original key untracked if the edit fails; after creating the VK with
virtualKeysPage.createVirtualKey(vkData) immediately push originalName into
managementVKs to ensure the created resource is tracked, then after a successful
edit replace that entry with updatedName (e.g., find the index of originalName
in managementVKs and set it to updatedName) so cleanup in afterEach will remove
the correct key whether the rename succeeds or not.
In `@ui/app/workspace/config/views/mcpView.tsx`:
- Around line 163-170: Add a data-testid to the Input for the Tool Sync Interval
so automated tests can target it; locate the Input with id
"mcp-tool-sync-interval" (the one bound to value
localValues.mcp_tool_sync_interval and onChange handleToolSyncIntervalChange)
and add a data-testid attribute (e.g. data-testid="mcp-tool-sync-interval") to
the element consistent with other MCP settings controls.
---
Duplicate comments:
In `@tests/e2e/features/model-limits/pages/model-limits.page.ts`:
- Around line 47-51: Update the JSDoc block above the function that creates a
model limit to reflect current deterministic behavior: replace the phrase "picks
the first model in the search dropdown" with a description that the sheet
selects the model specified by config.modelName (i.e., deterministic selection
using config.modelName), and adjust the summary to mention provider selection,
filling budget/rate limits, saving, and returning the selected model name for
use in exists/edit/delete; ensure the comment references config.modelName
explicitly.
In `@tests/e2e/features/observability/pages/observability.page.ts`:
- Around line 52-55: The getConnectorToggle method currently falls back to
this.page.locator('button[role="switch"]').first() when
ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS[connector] is missing, which causes
nondeterministic test behavior; update getConnectorToggle (in ObservabilityPage)
to only use this.page.getByTestId(testId) and remove the .first() fallback—if
testId is undefined, throw a clear error (or assert) indicating the connector
mapping is missing so tests fail fast and remain connector-scoped.
In `@tests/e2e/features/providers/pages/providers.page.ts`:
- Around line 409-418: The getConfigSaveBtn function currently uses role/label
mapping; change it to use test-id-first selectors by replacing the role-based
lookup in getConfigSaveBtn(configType) with this.page.getByTestId(...) using
stable test IDs for each config type (e.g., test ids like 'save-network-config',
'save-proxy-config', 'save-performance-config', 'save-governance-config',
'save-debugging-config'); also update the raw request/response switch locators
referenced in the same page object (the other locators mentioned around the
request/response switch) to use this.page.getByTestId() as primary selectors and
fall back to role/label only if the test-id is missing.
---
Nitpick comments:
In `@tests/e2e/features/config/config.spec.ts`:
- Around line 418-421: The test "should display rate limiting section" currently
asserts with expect(isVisible).toBeDefined(), which doesn't verify visibility;
update the assertion to assert the boolean truthiness of isVisible (e.g.,
expect(isVisible).toBe(true) or expect(isVisible).toBeTruthy()) so the call to
configSettingsPage.isRateLimitingSectionVisible() is actually validated; keep
the test name and the isRateLimitingSectionVisible() helper unchanged.
- Around line 318-360: Both tests ('should change log retention days' and
'should save and persist log retention days') currently no-op when
logRetentionDaysInput.isVisible() is false; update each test to call test.skip()
when the input is not visible (e.g., if (!isVisible) test.skip('log retention
input not available in this environment')), then proceed with the existing logic
that uses configSettingsPage.logRetentionDaysInput,
configSettingsPage.hasPendingChanges(), configSettingsPage.saveSettings(), and
configSettingsPage.goto('logging') so the test either explicitly skips or runs
assertions as intended.
- Around line 483-500: The new test.describe block named 'Pricing Config
Settings' duplicates existing 'Pricing Config' coverage and only checks the
heading while adding state capture/restore via originalState,
configSettingsPage.getCurrentSettings, and configSettingsPage.restoreSettings;
remove this redundant test.describe block entirely or replace it with additional
distinct assertions (e.g., interact with specific pricing toggles or
save/restore behavior) so it provides unique coverage beyond the existing
Pricing Config tests.
- Around line 442-480: For both tests "should change worker pool size" and
"should save and persist worker pool size" replace the current conditional that
wraps the whole test body with an explicit runtime skip when the input is not
visible: call const isVisible = await
configSettingsPage.workerPoolSizeInput.isVisible().catch(() => false) then
invoke test.skip(!isVisible, 'worker pool input not visible') (and return) so
the test is marked skipped instead of silently passing; keep the remainder of
the existing test logic (reading originalValue, computing newValue,
clearing/filling, and in the persist test calling
configSettingsPage.saveSettings() and reload) unchanged.
In `@tests/e2e/features/model-limits/pages/model-limits.page.ts`:
- Around line 73-83: The three locators use ID selectors instead of the
project's preferred getByTestId selector; update the calls that reference
'#modelBudgetMaxLimit', '#modelTokenMaxLimit', and '#modelRequestMaxLimit' (and
the local variable budgetInput) to use
this.page.getByTestId('modelBudgetMaxLimit'),
this.page.getByTestId('modelTokenMaxLimit'), and
this.page.getByTestId('modelRequestMaxLimit') respectively, then call
.fill(String(...)) on those test-id-based locators; also add corresponding
data-testid="modelBudgetMaxLimit"/"modelTokenMaxLimit"/"modelRequestMaxLimit"
attributes to the UI form inputs so the tests can target them.
In `@tests/e2e/features/observability/observability.spec.ts`:
- Around line 177-179: The test is asserting against hard-coded provider display
names; replace literal strings with the authoritative ProviderLabels constants:
import ProviderLabels from 'ui/lib/constants/logs' (or the correct named export)
and change expect(selected).toContain('Prometheus') to
expect(selected).toContain(ProviderLabels.PROMETHEUS) (and similarly replace
'BigQuery' at the other occurrence with ProviderLabels.BIGQUERY); locate the
assertions around observabilityPage.getSelectedConnector() in
observability.spec.ts and update both places (the block at ~177-179 and the one
at ~254-256) to use the ProviderLabels keys.
In `@ui/app/workspace/routing-rules/views/routingRulesView.tsx`:
- Around line 65-67: Update the data-testid on the Button in
routingRulesView.tsx from "create-routing-rule-btn" to follow the
entity-element-qualifier convention, e.g. "routing-rule-create-btn"; locate the
Button with the onClick handler handleCreateNew and change its data-testid
attribute, then search and update any E2E/page-object references in tests/e2e
that use the old test id.
ℹ️ Review info
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (57)
.github/workflows/e2e-tests.ymltests/e2e/core/fixtures/base.fixture.tstests/e2e/core/utils/selectors.tstests/e2e/features/config/config.spec.tstests/e2e/features/config/pages/config-settings.page.tstests/e2e/features/dashboard/dashboard.spec.tstests/e2e/features/governance/governance.data.tstests/e2e/features/governance/governance.spec.tstests/e2e/features/governance/pages/governance.page.tstests/e2e/features/mcp-auth-config/mcp-auth-config.spec.tstests/e2e/features/mcp-auth-config/pages/mcp-auth-config.page.tstests/e2e/features/mcp-registry/mcp-registry.spec.tstests/e2e/features/mcp-registry/pages/mcp-registry.page.tstests/e2e/features/mcp-settings/mcp-settings.spec.tstests/e2e/features/mcp-settings/pages/mcp-settings.page.tstests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.tstests/e2e/features/mcp-tool-groups/pages/mcp-tool-groups.page.tstests/e2e/features/model-limits/model-limits.data.tstests/e2e/features/model-limits/model-limits.spec.tstests/e2e/features/model-limits/pages/model-limits.page.tstests/e2e/features/observability/observability.spec.tstests/e2e/features/observability/pages/observability.page.tstests/e2e/features/placeholders/placeholders.spec.tstests/e2e/features/plugins/pages/plugins.page.tstests/e2e/features/plugins/plugins.spec.tstests/e2e/features/providers/pages/providers.page.tstests/e2e/features/providers/providers.spec.tstests/e2e/features/routing-rules/pages/routing-rules.page.tstests/e2e/features/routing-rules/routing-rules.spec.tstests/e2e/features/virtual-keys/pages/virtual-keys.page.tstests/e2e/features/virtual-keys/virtual-keys.spec.tstests/e2e/playwright.config.tsui/app/workspace/config/views/clientSettingsView.tsxui/app/workspace/config/views/mcpView.tsxui/app/workspace/config/views/pricingConfigView.tsxui/app/workspace/config/views/securityView.tsxui/app/workspace/governance/views/customerDialog.tsxui/app/workspace/governance/views/customerTable.tsxui/app/workspace/governance/views/teamDialog.tsxui/app/workspace/governance/views/teamsTable.tsxui/app/workspace/mcp-registry/views/mcpClientsTable.tsxui/app/workspace/mcp-registry/views/mcpServersEmptyState.tsxui/app/workspace/model-limits/views/modelLimitSheet.tsxui/app/workspace/model-limits/views/modelLimitsEmptyState.tsxui/app/workspace/model-limits/views/modelLimitsTable.tsxui/app/workspace/plugins/fragments/pluginFormFragments.tsxui/app/workspace/plugins/views/pluginsEmptyState.tsxui/app/workspace/providers/fragments/debuggingFormFragment.tsxui/app/workspace/providers/views/addProviderDropdown.tsxui/app/workspace/providers/views/modelProviderKeysTableView.tsxui/app/workspace/providers/views/providersEmptyState.tsxui/app/workspace/routing-rules/views/routingRulesEmptyState.tsxui/app/workspace/routing-rules/views/routingRulesTable.tsxui/app/workspace/routing-rules/views/routingRulesView.tsxui/app/workspace/virtual-keys/views/virtualKeysEmptyState.tsxui/app/workspace/virtual-keys/views/virtualKeysTable.tsxui/components/ui/numberAndSelect.tsx
🚧 Files skipped from review as they are similar to previous changes (28)
- ui/app/workspace/routing-rules/views/routingRulesTable.tsx
- ui/app/workspace/config/views/clientSettingsView.tsx
- tests/e2e/features/model-limits/model-limits.spec.ts
- ui/app/workspace/config/views/pricingConfigView.tsx
- ui/app/workspace/governance/views/customerTable.tsx
- tests/e2e/features/virtual-keys/pages/virtual-keys.page.ts
- ui/app/workspace/model-limits/views/modelLimitsEmptyState.tsx
- ui/app/workspace/virtual-keys/views/virtualKeysEmptyState.tsx
- ui/app/workspace/routing-rules/views/routingRulesEmptyState.tsx
- ui/app/workspace/governance/views/customerDialog.tsx
- tests/e2e/features/mcp-settings/mcp-settings.spec.ts
- ui/app/workspace/governance/views/teamsTable.tsx
- ui/app/workspace/plugins/views/pluginsEmptyState.tsx
- tests/e2e/features/mcp-settings/pages/mcp-settings.page.ts
- ui/app/workspace/providers/views/providersEmptyState.tsx
- ui/app/workspace/mcp-registry/views/mcpServersEmptyState.tsx
- tests/e2e/features/mcp-tool-groups/pages/mcp-tool-groups.page.ts
- tests/e2e/core/fixtures/base.fixture.ts
- tests/e2e/features/plugins/pages/plugins.page.ts
- tests/e2e/features/dashboard/dashboard.spec.ts
- ui/app/workspace/mcp-registry/views/mcpClientsTable.tsx
- ui/components/ui/numberAndSelect.tsx
- tests/e2e/features/config/pages/config-settings.page.ts
- ui/app/workspace/config/views/securityView.tsx
- tests/e2e/features/routing-rules/pages/routing-rules.page.ts
- tests/e2e/features/mcp-auth-config/mcp-auth-config.spec.ts
- tests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.ts
- ui/app/workspace/plugins/fragments/pluginFormFragments.tsx
Radheshg04
force-pushed
the
02-27-feat_extend_e2e_ui_tests
branch
from
53a87f2 to
bef4ba4
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
tests/e2e/features/routing-rules/pages/routing-rules.page.ts (1)
1-1:⚠️ Potential issue | 🟡 MinorImport
expectfrom the fixture, not directly from@playwright/test.Per coding guidelines, E2E test files should import
expectfrom../../core/fixtures/base.fixture, not from@playwright/test. While importing types likeLocatorandPagefrom Playwright is acceptable for page objects,expectshould come from the fixture to maintain consistency with files likegovernance.page.tsandmodel-limits.page.ts.Suggested fix
-import { Locator, Page, expect } from '@playwright/test' +import { Locator, Page } from '@playwright/test' +import { expect } from '../../../core/fixtures/base.fixture' import { BasePage } from '../../../core/pages/base.page'🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/routing-rules/pages/routing-rules.page.ts` at line 1, Replace the direct import of expect from '@playwright/test' with the fixture-provided expect: keep importing Locator and Page from '@playwright/test' but import expect from '../../core/fixtures/base.fixture' (as used in governance.page.ts/model-limits.page.ts) so the top-level import statement in routing-rules.page.ts references the fixture expect; update the import line that currently names Locator, Page, expect to instead import Locator and Page from '@playwright/test' and import expect separately from the base.fixture module.
♻️ Duplicate comments (3)
tests/e2e/features/plugins/plugins.spec.ts (1)
33-41:⚠️ Potential issue | 🟠 MajorSkip early when bootstrap plugin creation fails.
At Line 37, when
createPlugin()returnsfalse, the test still continues and can fail for backend/plugin-loading reasons instead of UI behavior. Add an earlytest.skip(...)path here, same as other plugin tests in this file.Proposed fix
if (count === 0) { const pluginData = createPluginData({ name: `e2e-display-table-${Date.now()}` }) const created = await pluginsPage.createPlugin(pluginData) - if (created) { - createdPlugins.push(pluginData.name) - } + if (!created) { + test.skip(true, 'Backend rejected plugin creation (failed to load .so)') + return + } + createdPlugins.push(pluginData.name) }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/plugins/plugins.spec.ts` around lines 33 - 41, When attempting to ensure a plugin exists, handle the failure path from pluginsPage.createPlugin: if pluginsPage.createPlugin(pluginData) returns false, call test.skip with a short message and return so the test stops early (same pattern used by other plugin tests); update the block that uses pluginsPage.getPluginCount, createPluginData, pluginsPage.createPlugin and mutates createdPlugins to skip the test on creation failure instead of continuing.ui/app/workspace/model-limits/views/modelLimitsTable.tsx (1)
118-118:⚠️ Potential issue | 🟡 MinorUse a 3-part
data-testidfor the table container.Line 118 uses
model-limits-table, which still misses the qualifier segment required by the convention (e.g.,model-limits-table-main).Suggested fix
- <div className="rounded-sm border" data-testid="model-limits-table"> + <div className="rounded-sm border" data-testid="model-limits-table-main">As per coding guidelines,
ui/app/**/*.{tsx,ts}requiresdata-testid="<entity>-<element>-<qualifier>".🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/model-limits/views/modelLimitsTable.tsx` at line 118, The container div in modelLimitsTable.tsx uses a two-part data-testid "model-limits-table"; update it to follow the three-part convention "<entity>-<element>-<qualifier>" (for example "model-limits-table-main") by changing the data-testid on the div with data-testid="model-limits-table" inside the ModelLimitsTable component to include a qualifier segment.tests/e2e/features/providers/pages/providers.page.ts (1)
409-418:⚠️ Potential issue | 🟠 MajorUse
getByTestId()for debugging save/switch selectors in the page object.
getConfigSaveBtn()and the raw request/response switch getters still rely on role/label + DOM traversal, which is brittle and against the page-object selector convention.Suggested page-object update
getConfigSaveBtn(configType: 'network' | 'proxy' | 'performance' | 'governance' | 'debugging'): Locator { - const buttonNames: Record<string, string> = { - network: 'Save Network Configuration', - proxy: 'Save Proxy Configuration', - performance: 'Save Performance Configuration', - governance: 'Save Governance Configuration', - debugging: 'Save Debugging Configuration', - } - return this.page.getByRole('button', { name: buttonNames[configType] }) + const saveTestIds: Record<string, string> = { + network: 'provider-config-network-save-btn', + proxy: 'provider-config-proxy-save-btn', + performance: 'provider-config-performance-save-btn', + governance: 'provider-config-governance-save-btn', + debugging: 'provider-config-debugging-save-btn', + } + return this.page.getByTestId(saveTestIds[configType]) } getRawRequestSwitch(): Locator { - return this.page.getByLabel('Send Back Raw Request').locator('..').locator('button[role="switch"]') + return this.page.getByTestId('send-back-raw-request-switch') } getRawResponseSwitch(): Locator { - return this.page.getByLabel('Send Back Raw Response').locator('..').locator('button[role="switch"]') + return this.page.getByTestId('send-back-raw-response-switch') }As per coding guidelines,
tests/e2e/**/*.tspage objects must usegetByTestId()as the primary selector strategy.Also applies to: 442-450
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/providers/pages/providers.page.ts` around lines 409 - 418, Replace brittle role/label selectors in getConfigSaveBtn and the raw request/response switch getters with test-id based selectors: change the buttonNames mapping lookup in getConfigSaveBtn to return this.page.getByTestId(<testId>) (e.g., map 'network' -> 'config-save-network', 'proxy' -> 'config-save-proxy', etc.) and update the raw request/response switch getters (the getters around lines 442-450) to use this.page.getByTestId('<appropriate-test-id>') instead of getByRole/getByLabelText + traversal so the page object consistently uses getByTestId() as the primary selector strategy.
🧹 Nitpick comments (8)
tests/e2e/features/observability/observability.spec.ts (2)
5-16:originalStateis captured but never used in cleanup.The
beforeEachcapturesoriginalStateand the comment inafterEachstates "disable all connectors that weren't enabled before", but the implementation unconditionally callsdisableAllConnectors()without referencingoriginalState. Either:
- Remove
originalStateand update the comment to reflect the actual behavior, or- Implement selective restoration based on the original state
Option 1: Remove unused state (simpler)
test.describe('Observability', () => { - let originalState: ObservabilityState - test.beforeEach(async ({ observabilityPage }) => { await observabilityPage.goto() - // Capture original state for restoration - originalState = await observabilityPage.getCurrentState() }) test.afterEach(async ({ observabilityPage }) => { - // Restore original state - disable all connectors that weren't enabled before + // Clean up: disable all connectors after each test await observabilityPage.disableAllConnectors() })🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/observability/observability.spec.ts` around lines 5 - 16, The test captures originalState in beforeEach via observabilityPage.getCurrentState but never uses it in afterEach, so either remove originalState and update the comment to reflect that afterEach unconditionally calls observabilityPage.disableAllConnectors, or implement selective restoration: store originalState in beforeEach and in afterEach iterate over originalState vs current state (using observabilityPage.getCurrentState and observabilityPage.disableConnector / enableConnector or similar methods) to only disable connectors that were not enabled originally; update the comment to describe the chosen behavior and remove dead variables if opting for option 1.
83-95: Redundant assertion after skip guard.The test checks
isVisible, skips if!isVisible, then assertstoBeVisible(). The final assertion is guaranteed to pass after the guard.Suggested simplification
test('should display OTel delete button when connector is configured', async ({ observabilityPage }) => { await observabilityPage.selectConnector('otel') const deleteBtn = observabilityPage.getConnectorDeleteBtn('otel') const isVisible = await deleteBtn.isVisible().catch(() => false) if (!isVisible) { test.skip(true, 'OTel delete button not visible (connector may not be configured)') return } - await expect(deleteBtn).toBeVisible() + // Already confirmed visible above; test passes })🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/observability/observability.spec.ts` around lines 83 - 95, The test 'should display OTel delete button when connector is configured' contains a redundant assertion: after computing isVisible and skipping when !isVisible, the final await expect(deleteBtn).toBeVisible() is guaranteed and should be removed; update the test by either removing the final expect(deleteBtn).toBeVisible() or replace the skip-guard with a direct assertion on isVisible (e.g., await expect(isVisible).toBe(true)) so the logic uses either the skip path or a single explicit assertion; target the observabilityPage.getConnectorDeleteBtn('otel') usage and the isVisible variable when making the change.tests/e2e/features/routing-rules/routing-rules.spec.ts (1)
299-299: Consider replacing hardcoded timeouts with explicit wait conditions.The
waitForTimeout(500)calls wait for rule builder animations/renders. While acceptable when no better DOM marker exists, these fixed delays can contribute to test flakiness if animations complete faster or slower than expected.If the rule builder has a specific element that appears when ready (e.g., a new rule row or updated CEL preview), consider waiting for that element instead.
Example: Wait for CEL preview to update instead of fixed timeout
// Add a rule condition await routingRulesPage.clickAddRule() - // Wait for rule row to appear and CEL to update - await routingRulesPage.page.waitForTimeout(500) + // Wait for CEL to update (no longer shows "No rules defined") + await expect.poll( + () => routingRulesPage.getCelExpression(), + { timeout: 5000 } + ).not.toContain('No rules defined')Also applies to: 326-326
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/routing-rules/routing-rules.spec.ts` at line 299, The test uses a hardcoded delay via routingRulesPage.page.waitForTimeout(500) which causes flakiness; replace this fixed timeout with an explicit wait for a DOM marker that indicates the rule builder finished rendering (for example wait for the new rule row, an updated CEL preview element, or a specific selector/text change). Update the test to call routingRulesPage.page.waitForSelector(...) or routingRulesPage.page.waitForFunction(...) targeting the new rule row selector or the CEL preview content, or add a helper on routingRulesPage like waitForRuleRow or waitForCelPreview that waits for the element/text change before proceeding.tests/e2e/features/virtual-keys/virtual-keys.spec.ts (1)
121-122: Prefer page-object/test-id locators over inline#idselectors in specs.These new assertions work, but using inline
page.locator('#budgetMaxLimit')andpage.locator('#tokenMaxLimit')in the spec makes selector strategy harder to keep consistent. Consider exposing these as page-object locators (backed bygetByTestId) and using those here.Suggested refactor
-const budgetInput = virtualKeysPage.page.locator('#budgetMaxLimit') +const budgetInput = virtualKeysPage.budgetMaxLimitInput -const tokenLimitInput = virtualKeysPage.page.locator('#tokenMaxLimit') +const tokenLimitInput = virtualKeysPage.tokenMaxLimitInput// tests/e2e/features/virtual-keys/pages/virtual-keys.page.ts // add stable locators there (prefer getByTestId-backed locators) readonly budgetMaxLimitInput = this.page.getByTestId('budget-max-limit-input') readonly tokenMaxLimitInput = this.page.getByTestId('token-max-limit-input')As per coding guidelines,
tests/e2e/**/*.ts: “E2E test page objects must extend BasePage, use getByTestId() as the primary selector strategy, and import test/expect from ../../core/fixtures/base.fixture, never from@playwright/test.”Also applies to: 166-167
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/virtual-keys/virtual-keys.spec.ts` around lines 121 - 122, Replace inline '#budgetMaxLimit' and '#tokenMaxLimit' locators with page-object locators: add readonly budgetMaxLimitInput = this.page.getByTestId('budget-max-limit-input') and readonly tokenMaxLimitInput = this.page.getByTestId('token-max-limit-input') to the VirtualKeys page object (ensure the page class extends BasePage and uses the test fixture imports required by E2E guidelines), then update the spec to use virtualKeysPage.budgetMaxLimitInput and virtualKeysPage.tokenMaxLimitInput in the expect assertions (e.g., expect(...).toHaveValue(String(SAMPLE_BUDGETS.small.maxLimit))).tests/e2e/features/config/pages/config-settings.page.ts (1)
285-298: ReusesetInputValuein new setters to reduce duplication.The new setter methods repeat
clear()+fill()logic that already exists insetInputValue.♻️ Suggested refactor
async setRequiredHeaders(value: string): Promise<void> { - await this.requiredHeadersTextarea.clear() - await this.requiredHeadersTextarea.fill(value) + await this.setInputValue(this.requiredHeadersTextarea, value) } async setWorkspaceLoggingHeaders(value: string): Promise<void> { - await this.workspaceLoggingHeadersTextarea.clear() - await this.workspaceLoggingHeadersTextarea.fill(value) + await this.setInputValue(this.workspaceLoggingHeadersTextarea, value) } async setAsyncJobResultTtl(value: string): Promise<void> { - await this.asyncJobResultTtlInput.clear() - await this.asyncJobResultTtlInput.fill(value) + await this.setInputValue(this.asyncJobResultTtlInput, value) } async setPricingDatasheetUrl(url: string): Promise<void> { - await this.pricingDatasheetUrlInput.clear() - await this.pricingDatasheetUrlInput.fill(url) + await this.setInputValue(this.pricingDatasheetUrlInput, url) }Also applies to: 323-326
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/config/pages/config-settings.page.ts` around lines 285 - 298, The three new setters (setRequiredHeaders, setWorkspaceLoggingHeaders, setAsyncJobResultTtl) duplicate the clear()+fill() pattern; refactor each to call the existing helper setInputValue instead (pass the appropriate element handle and the value) and remove the explicit clear()/fill() calls; do the same for the similar setters referenced around the other occurrence (the methods at the other noted lines) so all input setters reuse setInputValue for consistency.tests/e2e/features/model-limits/model-limits.spec.ts (1)
26-29: Align test intent with assertion.The test name says “create button or empty state,” but the assertion only checks the create button. Either include the empty-state branch or rename the test for precision.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/model-limits/model-limits.spec.ts` around lines 26 - 29, The test "should display create button or empty state" in model-limits.spec.ts is asserting only the create button; either update the assertion to allow the empty-state branch or rename the test to match its current behavior. Fix option A: change the test (test 'should display create button or empty state') to assert that either modelLimitsPage.createBtn.isVisible() OR modelLimitsPage.emptyState.isVisible() is true (check both selectors like createBtn and emptyState and expect a boolean OR). Fix option B: if you intend to only check the create button, rename the test to "should display create button" and keep the existing assertion referencing modelLimitsPage.createBtn.tests/e2e/features/providers/providers.spec.ts (2)
329-338: Avoid hard-coded 9-second wait; use a deterministic condition instead.The
waitForTimeout(9000)is unreliable and slows down tests. The toggle state should propagate within a much shorter period. Consider polling for the expected state with a reasonable timeout.♻️ Suggested refactor
// Toggle to disabled await providersPage.toggleKeyEnabled(keyData.name) - await providersPage.page.waitForTimeout(9000) - isEnabled = await providersPage.getKeyEnabledState(keyData.name) - expect(isEnabled).toBe(false) + // Wait for the state to update with a reasonable timeout + await expect(async () => { + const state = await providersPage.getKeyEnabledState(keyData.name) + expect(state).toBe(false) + }).toPass({ timeout: 10000 })🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/providers/providers.spec.ts` around lines 329 - 338, Replace the hard-coded waitForTimeout(9000) with a deterministic wait that polls for the expected toggle result: after calling providersPage.toggleKeyEnabled(keyData.name), repeatedly call providersPage.getKeyEnabledState(keyData.name) (or use providersPage.page.waitForFunction) until it returns false or a short timeout/interval elapses, then assert the state; remove the fixed 9s sleep and use this polling pattern so the test completes as soon as the state propagates.
684-730: Governance tests modify form values but never restore them.Unlike other test blocks (Performance Tuning, Network Config, etc.) that capture and restore original values,
should set budget limitandshould set rate limitsfill inputs without saving or restoring. If these tests ever call save, subsequent runs will inherit stale state.Either:
- Capture original values in
beforeEachand restore inafterEach(consistent with the rest of the file), or- Explicitly avoid saving and add a comment clarifying these are read-only/no-persist tests.
♻️ Suggested refactor (option 1: add restoration)
test.describe('Governance (Budget & Rate Limits)', () => { + let originalBudget: string + let originalTokenLimit: string + let originalRequestLimit: string + test.beforeEach(async ({ providersPage }) => { await providersPage.goto() await providersPage.selectProvider('openai') + const isVisible = await providersPage.isGovernanceTabVisible() + if (isVisible) { + await providersPage.selectConfigTab('governance') + originalBudget = await providersPage.page.locator('#providerBudgetMaxLimit').inputValue() + originalTokenLimit = await providersPage.page.locator('#providerTokenMaxLimit').inputValue() + originalRequestLimit = await providersPage.page.locator('#providerRequestMaxLimit').inputValue() + } + }) + + test.afterEach(async ({ providersPage }) => { + const isVisible = await providersPage.isGovernanceTabVisible() + if (isVisible) { + await providersPage.selectConfigTab('governance') + // Restore original values if they were captured + if (originalBudget !== undefined) { + const budgetInput = providersPage.page.locator('#providerBudgetMaxLimit') + await budgetInput.click() + await budgetInput.fill(originalBudget) + } + // ... similar for other fields + } })🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/providers/providers.spec.ts` around lines 684 - 730, These governance tests change inputs without restoring them; update the spec to capture original values for `#providerBudgetMaxLimit`, `#providerTokenMaxLimit` and `#providerRequestMaxLimit` in a beforeEach (using providersPage.isGovernanceTabVisible() and providersPage.selectConfigTab('governance') to access the fields) and restore those values in an afterEach (set inputs back and click providersPage.getConfigSaveBtn('governance') if the UI requires saving), or explicitly mark the two tests ('should set budget limit' and 'should set rate limits') as read-only and ensure they never trigger a save; use the existing providersPage locator IDs and methods to locate and set the values so state is returned to its original form after each test.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/e2e/features/model-limits/model-limits.spec.ts`:
- Around line 60-75: The test fails to register the created model limit for
cleanup and may use non-unique names; update the test so createModelLimitData
uses Date.now() to produce a unique model name, then push the returned modelName
from modelLimitsPage.createModelLimit into the test-wide createdLimits array
immediately after creation, and ensure the test-suite has an afterEach that
iterates createdLimits to delete any remaining limits via
modelLimitsPage.deleteModelLimit; reference createModelLimitData,
modelLimitsPage.createModelLimit, createdLimits and
modelLimitsPage.deleteModelLimit when making these changes.
---
Outside diff comments:
In `@tests/e2e/features/routing-rules/pages/routing-rules.page.ts`:
- Line 1: Replace the direct import of expect from '@playwright/test' with the
fixture-provided expect: keep importing Locator and Page from '@playwright/test'
but import expect from '../../core/fixtures/base.fixture' (as used in
governance.page.ts/model-limits.page.ts) so the top-level import statement in
routing-rules.page.ts references the fixture expect; update the import line that
currently names Locator, Page, expect to instead import Locator and Page from
'@playwright/test' and import expect separately from the base.fixture module.
---
Duplicate comments:
In `@tests/e2e/features/plugins/plugins.spec.ts`:
- Around line 33-41: When attempting to ensure a plugin exists, handle the
failure path from pluginsPage.createPlugin: if
pluginsPage.createPlugin(pluginData) returns false, call test.skip with a short
message and return so the test stops early (same pattern used by other plugin
tests); update the block that uses pluginsPage.getPluginCount, createPluginData,
pluginsPage.createPlugin and mutates createdPlugins to skip the test on creation
failure instead of continuing.
In `@tests/e2e/features/providers/pages/providers.page.ts`:
- Around line 409-418: Replace brittle role/label selectors in getConfigSaveBtn
and the raw request/response switch getters with test-id based selectors: change
the buttonNames mapping lookup in getConfigSaveBtn to return
this.page.getByTestId(<testId>) (e.g., map 'network' -> 'config-save-network',
'proxy' -> 'config-save-proxy', etc.) and update the raw request/response switch
getters (the getters around lines 442-450) to use
this.page.getByTestId('<appropriate-test-id>') instead of
getByRole/getByLabelText + traversal so the page object consistently uses
getByTestId() as the primary selector strategy.
In `@ui/app/workspace/model-limits/views/modelLimitsTable.tsx`:
- Line 118: The container div in modelLimitsTable.tsx uses a two-part
data-testid "model-limits-table"; update it to follow the three-part convention
"<entity>-<element>-<qualifier>" (for example "model-limits-table-main") by
changing the data-testid on the div with data-testid="model-limits-table" inside
the ModelLimitsTable component to include a qualifier segment.
---
Nitpick comments:
In `@tests/e2e/features/config/pages/config-settings.page.ts`:
- Around line 285-298: The three new setters (setRequiredHeaders,
setWorkspaceLoggingHeaders, setAsyncJobResultTtl) duplicate the clear()+fill()
pattern; refactor each to call the existing helper setInputValue instead (pass
the appropriate element handle and the value) and remove the explicit
clear()/fill() calls; do the same for the similar setters referenced around the
other occurrence (the methods at the other noted lines) so all input setters
reuse setInputValue for consistency.
In `@tests/e2e/features/model-limits/model-limits.spec.ts`:
- Around line 26-29: The test "should display create button or empty state" in
model-limits.spec.ts is asserting only the create button; either update the
assertion to allow the empty-state branch or rename the test to match its
current behavior. Fix option A: change the test (test 'should display create
button or empty state') to assert that either
modelLimitsPage.createBtn.isVisible() OR modelLimitsPage.emptyState.isVisible()
is true (check both selectors like createBtn and emptyState and expect a boolean
OR). Fix option B: if you intend to only check the create button, rename the
test to "should display create button" and keep the existing assertion
referencing modelLimitsPage.createBtn.
In `@tests/e2e/features/observability/observability.spec.ts`:
- Around line 5-16: The test captures originalState in beforeEach via
observabilityPage.getCurrentState but never uses it in afterEach, so either
remove originalState and update the comment to reflect that afterEach
unconditionally calls observabilityPage.disableAllConnectors, or implement
selective restoration: store originalState in beforeEach and in afterEach
iterate over originalState vs current state (using
observabilityPage.getCurrentState and observabilityPage.disableConnector /
enableConnector or similar methods) to only disable connectors that were not
enabled originally; update the comment to describe the chosen behavior and
remove dead variables if opting for option 1.
- Around line 83-95: The test 'should display OTel delete button when connector
is configured' contains a redundant assertion: after computing isVisible and
skipping when !isVisible, the final await expect(deleteBtn).toBeVisible() is
guaranteed and should be removed; update the test by either removing the final
expect(deleteBtn).toBeVisible() or replace the skip-guard with a direct
assertion on isVisible (e.g., await expect(isVisible).toBe(true)) so the logic
uses either the skip path or a single explicit assertion; target the
observabilityPage.getConnectorDeleteBtn('otel') usage and the isVisible variable
when making the change.
In `@tests/e2e/features/providers/providers.spec.ts`:
- Around line 329-338: Replace the hard-coded waitForTimeout(9000) with a
deterministic wait that polls for the expected toggle result: after calling
providersPage.toggleKeyEnabled(keyData.name), repeatedly call
providersPage.getKeyEnabledState(keyData.name) (or use
providersPage.page.waitForFunction) until it returns false or a short
timeout/interval elapses, then assert the state; remove the fixed 9s sleep and
use this polling pattern so the test completes as soon as the state propagates.
- Around line 684-730: These governance tests change inputs without restoring
them; update the spec to capture original values for `#providerBudgetMaxLimit`,
`#providerTokenMaxLimit` and `#providerRequestMaxLimit` in a beforeEach (using
providersPage.isGovernanceTabVisible() and
providersPage.selectConfigTab('governance') to access the fields) and restore
those values in an afterEach (set inputs back and click
providersPage.getConfigSaveBtn('governance') if the UI requires saving), or
explicitly mark the two tests ('should set budget limit' and 'should set rate
limits') as read-only and ensure they never trigger a save; use the existing
providersPage locator IDs and methods to locate and set the values so state is
returned to its original form after each test.
In `@tests/e2e/features/routing-rules/routing-rules.spec.ts`:
- Line 299: The test uses a hardcoded delay via
routingRulesPage.page.waitForTimeout(500) which causes flakiness; replace this
fixed timeout with an explicit wait for a DOM marker that indicates the rule
builder finished rendering (for example wait for the new rule row, an updated
CEL preview element, or a specific selector/text change). Update the test to
call routingRulesPage.page.waitForSelector(...) or
routingRulesPage.page.waitForFunction(...) targeting the new rule row selector
or the CEL preview content, or add a helper on routingRulesPage like
waitForRuleRow or waitForCelPreview that waits for the element/text change
before proceeding.
In `@tests/e2e/features/virtual-keys/virtual-keys.spec.ts`:
- Around line 121-122: Replace inline '#budgetMaxLimit' and '#tokenMaxLimit'
locators with page-object locators: add readonly budgetMaxLimitInput =
this.page.getByTestId('budget-max-limit-input') and readonly tokenMaxLimitInput
= this.page.getByTestId('token-max-limit-input') to the VirtualKeys page object
(ensure the page class extends BasePage and uses the test fixture imports
required by E2E guidelines), then update the spec to use
virtualKeysPage.budgetMaxLimitInput and virtualKeysPage.tokenMaxLimitInput in
the expect assertions (e.g.,
expect(...).toHaveValue(String(SAMPLE_BUDGETS.small.maxLimit))).
ℹ️ Review info
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (57)
.github/workflows/e2e-tests.ymltests/e2e/core/fixtures/base.fixture.tstests/e2e/core/utils/selectors.tstests/e2e/features/config/config.spec.tstests/e2e/features/config/pages/config-settings.page.tstests/e2e/features/dashboard/dashboard.spec.tstests/e2e/features/governance/governance.data.tstests/e2e/features/governance/governance.spec.tstests/e2e/features/governance/pages/governance.page.tstests/e2e/features/mcp-auth-config/mcp-auth-config.spec.tstests/e2e/features/mcp-auth-config/pages/mcp-auth-config.page.tstests/e2e/features/mcp-registry/mcp-registry.spec.tstests/e2e/features/mcp-registry/pages/mcp-registry.page.tstests/e2e/features/mcp-settings/mcp-settings.spec.tstests/e2e/features/mcp-settings/pages/mcp-settings.page.tstests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.tstests/e2e/features/mcp-tool-groups/pages/mcp-tool-groups.page.tstests/e2e/features/model-limits/model-limits.data.tstests/e2e/features/model-limits/model-limits.spec.tstests/e2e/features/model-limits/pages/model-limits.page.tstests/e2e/features/observability/observability.spec.tstests/e2e/features/observability/pages/observability.page.tstests/e2e/features/placeholders/placeholders.spec.tstests/e2e/features/plugins/pages/plugins.page.tstests/e2e/features/plugins/plugins.spec.tstests/e2e/features/providers/pages/providers.page.tstests/e2e/features/providers/providers.spec.tstests/e2e/features/routing-rules/pages/routing-rules.page.tstests/e2e/features/routing-rules/routing-rules.spec.tstests/e2e/features/virtual-keys/pages/virtual-keys.page.tstests/e2e/features/virtual-keys/virtual-keys.spec.tstests/e2e/playwright.config.tsui/app/workspace/config/views/clientSettingsView.tsxui/app/workspace/config/views/mcpView.tsxui/app/workspace/config/views/pricingConfigView.tsxui/app/workspace/config/views/securityView.tsxui/app/workspace/governance/views/customerDialog.tsxui/app/workspace/governance/views/customerTable.tsxui/app/workspace/governance/views/teamDialog.tsxui/app/workspace/governance/views/teamsTable.tsxui/app/workspace/mcp-registry/views/mcpClientsTable.tsxui/app/workspace/mcp-registry/views/mcpServersEmptyState.tsxui/app/workspace/model-limits/views/modelLimitSheet.tsxui/app/workspace/model-limits/views/modelLimitsEmptyState.tsxui/app/workspace/model-limits/views/modelLimitsTable.tsxui/app/workspace/plugins/fragments/pluginFormFragments.tsxui/app/workspace/plugins/views/pluginsEmptyState.tsxui/app/workspace/providers/fragments/debuggingFormFragment.tsxui/app/workspace/providers/views/addProviderDropdown.tsxui/app/workspace/providers/views/modelProviderKeysTableView.tsxui/app/workspace/providers/views/providersEmptyState.tsxui/app/workspace/routing-rules/views/routingRulesEmptyState.tsxui/app/workspace/routing-rules/views/routingRulesTable.tsxui/app/workspace/routing-rules/views/routingRulesView.tsxui/app/workspace/virtual-keys/views/virtualKeysEmptyState.tsxui/app/workspace/virtual-keys/views/virtualKeysTable.tsxui/components/ui/numberAndSelect.tsx
🚧 Files skipped from review as they are similar to previous changes (28)
- ui/app/workspace/providers/views/modelProviderKeysTableView.tsx
- tests/e2e/features/model-limits/model-limits.data.ts
- tests/e2e/features/mcp-settings/mcp-settings.spec.ts
- ui/app/workspace/governance/views/teamsTable.tsx
- ui/app/workspace/config/views/clientSettingsView.tsx
- ui/app/workspace/virtual-keys/views/virtualKeysEmptyState.tsx
- ui/app/workspace/config/views/pricingConfigView.tsx
- tests/e2e/features/mcp-settings/pages/mcp-settings.page.ts
- ui/app/workspace/governance/views/teamDialog.tsx
- tests/e2e/features/virtual-keys/pages/virtual-keys.page.ts
- tests/e2e/features/model-limits/pages/model-limits.page.ts
- tests/e2e/features/mcp-registry/mcp-registry.spec.ts
- ui/app/workspace/virtual-keys/views/virtualKeysTable.tsx
- ui/app/workspace/mcp-registry/views/mcpServersEmptyState.tsx
- tests/e2e/features/governance/pages/governance.page.ts
- ui/app/workspace/routing-rules/views/routingRulesTable.tsx
- tests/e2e/core/utils/selectors.ts
- ui/app/workspace/governance/views/customerTable.tsx
- tests/e2e/features/mcp-tool-groups/pages/mcp-tool-groups.page.ts
- tests/e2e/features/mcp-auth-config/mcp-auth-config.spec.ts
- ui/app/workspace/plugins/fragments/pluginFormFragments.tsx
- ui/components/ui/numberAndSelect.tsx
- tests/e2e/features/dashboard/dashboard.spec.ts
- ui/app/workspace/routing-rules/views/routingRulesView.tsx
- ui/app/workspace/model-limits/views/modelLimitSheet.tsx
- ui/app/workspace/providers/views/providersEmptyState.tsx
- ui/app/workspace/governance/views/customerDialog.tsx
- ui/app/workspace/providers/fragments/debuggingFormFragment.tsx
Radheshg04
force-pushed
the
02-27-feat_extend_e2e_ui_tests
branch
from
bef4ba4 to
7ca417a
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
ui/app/workspace/governance/views/customerDialog.tsx (1)
242-273:⚠️ Potential issue | 🟡 MinorAdd test ids to the remaining interactive limit controls in this form section.
tokenMaxLimitandrequestMaxLimitcontrols are interactive but still lack a test id, which makes E2E selectors inconsistent in this workspace page.Suggested patch
<NumberAndSelect id="tokenMaxLimit" label="Maximum Tokens" value={formData.tokenMaxLimit} selectValue={formData.tokenResetDuration} onChangeNumber={(value) => updateField("tokenMaxLimit", value)} onChangeSelect={(value) => updateField("tokenResetDuration", value)} options={resetDurationOptions} + dataTestId="customer-token-limit-input" /> @@ <NumberAndSelect id="requestMaxLimit" label="Maximum Requests" value={formData.requestMaxLimit} selectValue={formData.requestResetDuration} onChangeNumber={(value) => updateField("requestMaxLimit", value)} onChangeSelect={(value) => updateField("requestResetDuration", value)} options={resetDurationOptions} + dataTestId="customer-request-limit-input" />As per coding guidelines,
ui/app/workspace/**/*.tsxmust includedata-testidattributes on all interactive elements.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/governance/views/customerDialog.tsx` around lines 242 - 273, The tokenMaxLimit and requestMaxLimit NumberAndSelect components are missing data-testid attributes; update the two JSX elements (the NumberAndSelect with id="tokenMaxLimit" and the one with id="requestMaxLimit") to include a dataTestId prop (e.g., dataTestId="token-max-limit-input" and dataTestId="request-max-limit-input") so all interactive limit controls in this form section have consistent test ids for E2E selectors.
♻️ Duplicate comments (7)
tests/e2e/features/observability/pages/observability.page.ts (2)
52-55:⚠️ Potential issue | 🟠 MajorFail fast instead of using unscoped
.first()for connector toggles.Line 54’s generic
button[role="switch"]fallback is nondeterministic and can toggle/read the wrong connector, which also taintsgetCurrentState()anddisableAllConnectors()flows.Suggested fix
getConnectorToggle(connector: ObservabilityConnector): Locator { const testId = ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS[connector] - return testId ? this.page.getByTestId(testId) : this.page.locator('button[role="switch"]').first() + if (!testId) { + throw new Error(`Missing toggle test id mapping for connector: ${connector}`) + } + return this.page.getByTestId(testId) }#!/bin/bash # Verify unresolved generic fallback and missing connector mappings in this page object rg -n "CONNECTOR_TOGGLE_TESTIDS|getConnectorToggle\\(|button\\[role=\"switch\"\\]\\.first\\(" tests/e2e/features/observability/pages/observability.page.ts -C 2As per coding guidelines "
tests/e2e/**/*.ts: E2E test page objects must extend BasePage, use getByTestId() as the primary selector strategy, and import test/expect from ../../core/fixtures/base.fixture, never from `@playwright/test``."🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/observability/pages/observability.page.ts` around lines 52 - 55, getConnectorToggle currently falls back to an unscoped this.page.locator('button[role="switch"]').first() which is nondeterministic; update ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS to include testIDs for every ObservabilityConnector enum value and change getConnectorToggle(connector) to only use this.page.getByTestId(testId) and throw a clear error if a mapping is missing (do not use .first() fallback). Also audit callers like getCurrentState() and disableAllConnectors() to rely on the strict getByTestId-based selector so they remain deterministic.
1-1:⚠️ Potential issue | 🟠 MajorUse
BasePage.waitForStateChangeand removeexpectfrom@playwright/testin this page object.Line 1 + Line 147 still use direct
expectfrom@playwright/test; this violates the E2E page-object rule and duplicates already-flagged guidance.Suggested fix
-import { Page, Locator, expect } from '@playwright/test' +import { Page, Locator } from '@playwright/test' ... - await expect(toggle).toHaveAttribute('data-state', expectedState, { timeout: 5000 }) + await this.waitForStateChange(toggle, 'data-state', expectedState, 5000)#!/bin/bash # Verify direct expect import/usage in this page object rg -n "import .*expect.*@playwright/test|expect\\(toggle\\)\\.toHaveAttribute\\('data-state'" tests/e2e/features/observability/pages/observability.page.ts -C 2As per coding guidelines "
tests/e2e/**/*.ts: E2E test page objects must extend BasePage, use getByTestId() as the primary selector strategy, and import test/expect from ../../core/fixtures/base.fixture, never from `@playwright/test``."Also applies to: 144-148
ui/app/workspace/model-limits/views/modelLimitsTable.tsx (1)
118-118:⚠️ Potential issue | 🟡 MinorUse a 3-part
data-testidfor the table container.Line 118 uses
model-limits-table(2-part). Please switch to a 3-part value and keep the page-object selector in sync.🔧 Suggested fix
- <div className="rounded-sm border" data-testid="model-limits-table"> + <div className="rounded-sm border" data-testid="model-limits-table-main">- this.table = page.getByTestId('model-limits-table') + this.table = page.getByTestId('model-limits-table-main')As per coding guidelines,
ui/app/**/*.{tsx,ts}requiresdata-testid="<entity>-<element>-<qualifier>", and renamed ids must be aligned with references undertests/e2e/.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/model-limits/views/modelLimitsTable.tsx` at line 118, The container div's data-testid "model-limits-table" violates the 3-part convention; update it to a 3-part ID like "model-limits-table-container" (or "model-limits-table-root") in modelLimitsTable.tsx (the div with data-testid inside the ModelLimitsTable view) and then update the corresponding page-object/test selectors under tests/e2e to use the new three-part identifier so all references remain in sync.tests/e2e/features/model-limits/pages/model-limits.page.ts (1)
47-50:⚠️ Potential issue | 🟠 MajorSelect
config.modelNamedeterministically instead of.first().Current flow ignores the requested model and picks the first option, which makes create/edit/delete target resolution flaky and can break cleanup.
🔧 Suggested fix
- /** - * Create a model limit via the sheet: selects provider, picks the first model - * in the search dropdown, fills budget and rate limit, then saves. Returns - * the selected model name for use in exists/edit/delete. - */ + /** + * Create a model limit via the sheet: selects provider, selects `config.modelName` + * deterministically, fills budget/rate limits, then saves. + */ async createModelLimit(config: ModelLimitConfig): Promise<string> { @@ - // Model multiselect - click input to open dropdown, select first option + // Model multiselect - search and select requested model deterministically const modelSelectContainer = this.sheet.getByTestId('model-limit-model-select') - await modelSelectContainer.locator('input').click() + const modelInput = modelSelectContainer.locator('input') + await modelInput.fill(config.modelName) await this.page.waitForSelector('[role="option"]', { timeout: 10000 }) - const firstOption = this.page.getByRole('option').first() - await expect(firstOption).toBeVisible({ timeout: 10000 }) - const selectedModelName = (await firstOption.textContent())?.trim() ?? '' - await firstOption.click() + const targetOption = this.page.getByRole('option', { name: config.modelName, exact: true }) + await expect(targetOption).toBeVisible({ timeout: 10000 }) + await targetOption.click() + const selectedModelName = config.modelNameAlso applies to: 64-71
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/model-limits/pages/model-limits.page.ts` around lines 47 - 50, The model selection currently uses .first() which ignores the requested config.modelName and makes create/edit/delete flaky; change the selection logic in the sheet flow to locate the dropdown option whose text equals config.modelName and click that option (instead of using .first()), and apply the same deterministic selection change to the other occurrence noted (the block covering the create/edit flows around the second occurrence). Reference config.modelName and replace any .first() usage in the model-picker code path so the selected model matches config.modelName exactly.tests/e2e/features/governance/pages/governance.page.ts (3)
113-116:⚠️ Potential issue | 🟠 MajorScope dialog actions to dialog locators to avoid cross-target clicks.
Using
this.page.getByRole(...)for save/cancel can attach to the wrong button when multiple actionable controls are present. Scope these tothis.teamDialog/this.customerDialog.🔧 Suggested fix pattern
- const saveBtn = this.page.getByRole('button', { name: /Create Team/i }) + const saveBtn = this.teamDialog.getByRole('button', { name: /Create Team/i }) await saveBtn.click() @@ - await this.page.getByRole('button', { name: /Cancel/i }).click() + await this.teamDialog.getByRole('button', { name: /Cancel/i }).click() @@ - const saveBtn = this.page.getByRole('button', { name: /Create Customer/i }) + const saveBtn = this.customerDialog.getByRole('button', { name: /Create Customer/i })As per coding guidelines,
tests/e2e/**/*.tspage objects must usegetByTestId()as the primary selector strategy, with stable scoping to the active UI container.Also applies to: 128-131, 156-159, 200-203, 218-221
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/governance/pages/governance.page.ts` around lines 113 - 116, The save/cancel button lookups use this.page.getByRole(...) and can target the wrong control; change them to scope to the dialog locators and use getByTestId as the primary selector (e.g., replace this.page.getByRole(...) with this.teamDialog.getByTestId(...) for team dialog actions, and similarly use this.customerDialog.getByTestId(...) for customer dialog actions), then perform the click and await this.waitForSuccessToast() and the expect(this.teamDialog).not.toBeVisible(...) as before; apply the same scoping/selector change for the other occurrences referenced (lines near 128-131, 156-159, 200-203, 218-221) so all dialog actions are tied to their dialog locator.
6-30:⚠️ Potential issue | 🟠 MajorDon’t expose config fields that the page object never applies.
TeamConfig/CustomerConfigcurrently acceptbudget.resetDurationandrateLimitfields, but create/edit flows ignore them. That silently drops caller intent and can produce misleading test setups.🔧 Suggested fix (if deferring UI support)
export interface TeamConfig { name: string @@ - budget?: { maxLimit: number; resetDuration?: string } - rateLimit?: { - tokenMaxLimit?: number - tokenResetDuration?: string - requestMaxLimit?: number - requestResetDuration?: string - } + budget?: { maxLimit: number } } export interface CustomerConfig { name: string - budget?: { maxLimit: number; resetDuration?: string } - rateLimit?: { - tokenMaxLimit?: number - tokenResetDuration?: string - requestMaxLimit?: number - requestResetDuration?: string - } + budget?: { maxLimit: number } }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/governance/pages/governance.page.ts` around lines 6 - 30, The TeamConfig and CustomerConfig types expose budget.resetDuration and the entire rateLimit structure which the governance page object never applies; remove budget.resetDuration and the rateLimit property from both TeamConfig and CustomerConfig (and anywhere they are referenced) so the interfaces only include fields the page object actually uses (e.g., name, customerId/customerName, and budget.maxLimit); update any tests or factories that set the removed fields to stop passing them or to map them to supported fields.
119-126:⚠️ Potential issue | 🟠 MajorWait for row disappearance before returning from delete helpers.
Both delete helpers return immediately after toast. Callers that assert non-existence right away can race table refresh and flake.
🔧 Suggested fix
async deleteTeam(name: string): Promise<void> { @@ await confirmDialog.getByRole('button', { name: /Delete/i }).click() await this.waitForSuccessToast() + await expect.poll(() => this.teamExists(name), { timeout: 10000 }).toBe(false) } @@ async deleteCustomer(name: string): Promise<void> { @@ await confirmBtn.click() await this.waitForSuccessToast() + await expect.poll(() => this.customerExists(name), { timeout: 10000 }).toBe(false) }Also applies to: 162-171
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/governance/pages/governance.page.ts` around lines 119 - 126, The deleteTeam helper returns as soon as the success toast appears, which can race with the table update; after clicking confirm (currently using confirmDialog.getByRole('button', { name: /Delete/i }).click()) and awaiting this.waitForSuccessToast(), also wait for the deleted team's row to be removed before returning — e.g., capture the delete button locator (deleteBtn) or its row ancestor and call a wait-for-detached/hidden on that locator (locator.waitFor({ state: 'detached' }) or equivalent) so callers asserting non-existence won't race; apply the same change to the other delete helper referenced in the diff (the duplicate helper at lines 162-171).
🧹 Nitpick comments (4)
tests/e2e/features/providers/providers.spec.ts (1)
329-338: Excessive hard-coded timeout may cause flaky/slow tests.The 9000ms
waitForTimeouton line 335 is quite long and suggests timing uncertainty. Consider using a more deterministic wait strategy, such as waiting for the switch'sdata-stateattribute to change or for a network request to complete.🔧 Suggested refactor
// Toggle to disabled await providersPage.toggleKeyEnabled(keyData.name) - await providersPage.page.waitForTimeout(9000) + // Wait for the switch state to update after the API call + const switchEl = providersPage.getKeyRow(keyData.name).getByTestId('key-enabled-switch') + await expect(switchEl).toHaveAttribute('data-state', 'unchecked', { timeout: 10000 }) isEnabled = await providersPage.getKeyEnabledState(keyData.name) expect(isEnabled).toBe(false)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/providers/providers.spec.ts` around lines 329 - 338, The long fixed 9000ms sleep in the test should be replaced with a deterministic wait: after calling providersPage.toggleKeyEnabled(keyData.name) use a targeted wait that watches for the toggle state change (e.g., waitForFunction/waitForSelector against the toggle element's data-state attribute or aria-checked, or waitForResponse for the request the toggle triggers) instead of providersPage.page.waitForTimeout; update the test to poll via providersPage.getKeyEnabledState(keyData.name) or wait for the specific DOM attribute to reflect the disabled state and proceed as soon as that condition is true.tests/e2e/features/config/config.spec.ts (1)
161-168: Consider extracting a shared “availability guard” helper.The same
isVisible().catch(...)+test.skippattern is repeated across multiple tests. A small helper would reduce duplication and keep skip behavior consistent.Also applies to: 267-274, 384-416
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/config/config.spec.ts` around lines 161 - 168, Extract a reusable availability guard to replace the repeated isVisible().catch(() => false) + test.skip pattern: add a helper (e.g., skipIfNotVisible(pageObjectElement, testContext, reason)) that checks await pageObjectElement.isVisible().catch(() => false) and calls testContext.skip(true, reason) when false; update tests that use configSettingsPage.asyncJobResultTtlInput (and the other occurrences) to call this helper at the start of the test instead of inlining the visibility check so skip behavior is centralized and consistent.tests/e2e/features/virtual-keys/virtual-keys.spec.ts (2)
121-122: Move raw form selectors behind the page object (prefergetByTestId).These direct
page.locator('#...')calls in spec code bypass the page object abstraction and make selector maintenance harder.♻️ Suggested refactor
- const budgetInput = virtualKeysPage.page.locator('#budgetMaxLimit') + const budgetInput = virtualKeysPage.budgetMaxLimitInput await expect(budgetInput).toHaveValue(String(SAMPLE_BUDGETS.small.maxLimit)) @@ - const tokenLimitInput = virtualKeysPage.page.locator('#tokenMaxLimit') + const tokenLimitInput = virtualKeysPage.tokenMaxLimitInput await expect(tokenLimitInput).toHaveValue(String(SAMPLE_RATE_LIMITS.tokenOnly.tokenMaxLimit))// In tests/e2e/features/virtual-keys/pages/virtual-keys.page.ts budgetMaxLimitInput = this.page.getByTestId('budget-max-limit-input') tokenMaxLimitInput = this.page.getByTestId('token-max-limit-input')As per coding guidelines,
tests/e2e/**/*.ts: “E2E test page objects must extend BasePage, use getByTestId() as the primary selector strategy.”Also applies to: 166-167
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/virtual-keys/virtual-keys.spec.ts` around lines 121 - 122, Replace direct page.locator calls in the spec with page-object properties: add budgetMaxLimitInput and tokenMaxLimitInput to the VirtualKeysPage (virtual-keys.page.ts) using this.page.getByTestId('budget-max-limit-input') and this.page.getByTestId('token-max-limit-input'), then update the spec to reference virtualKeysPage.budgetMaxLimitInput (and tokenMaxLimitInput for the other occurrence) when asserting values like SAMPLE_BUDGETS.small.maxLimit so all raw selectors are encapsulated in the page object.
369-375: Use retrying assertions for key visibility transitions.Single-shot reads immediately after toggles can be timing-sensitive. Use
expect.poll()to poll the visibility state until it reaches the expected value, reducing flake risk.⏱️ Suggested change
- isRevealed = await virtualKeysPage.isKeyRevealed(vkName) - expect(isRevealed).toBe(true) + await expect.poll(() => virtualKeysPage.isKeyRevealed(vkName)).toBe(true)Also applies to: 379-380
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/virtual-keys/virtual-keys.spec.ts` around lines 369 - 375, Replace single-shot checks after visibility toggles with retrying assertions: after calling virtualKeysPage.toggleKeyVisibility(vkName) use expect.poll(() => virtualKeysPage.isKeyRevealed(vkName)) to wait until it resolves to the expected boolean, and do the same for the hide transition. Update the two places that currently call virtualKeysPage.isKeyRevealed(vkName) followed by expect(...).toBe(...) so they use expect.poll to repeatedly read virtualKeysPage.isKeyRevealed until it returns true (for show) or false (for hide).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/e2e/features/config/config.spec.ts`:
- Around line 74-79: The teardown may attempt to restore originalPricingUrl when
setup failed and originalPricingUrl is undefined; update the cleanup (the
test.afterEach or teardown block that calls pricingDatasheetUrlInput.fill/or
restores the value) to first check that originalPricingUrl is defined (or not
undefined/null) before using it, and skip the restore if it wasn't set by
test.beforeEach; refer to the test.beforeEach that assigns originalPricingUrl
and the corresponding afterEach/teardown that restores it to locate and guard
the restore logic.
In `@tests/e2e/features/governance/governance.spec.ts`:
- Around line 4-6: The module-level mutable arrays createdTeams and
createdCustomers cause race conditions under fullyParallel execution; replace
them with Playwright fixtures (via test.extend) that provide per-test or
per-worker arrays (e.g., a fixture named createdTeams and createdCustomers) and
perform cleanup in the fixture's teardown (or in test.afterEach using the
fixture-provided arrays). Update tests in governance.spec.ts to accept the new
fixtures instead of referencing module globals and move any cleanup logic to the
fixture teardown so each worker/test has its own isolated trackers.
In `@tests/e2e/features/plugins/pages/plugins.page.ts`:
- Around line 57-59: The test uses non-testid selectors (this.createBtn and
this.pluginList) for readiness and counts; change these to use getByTestId as
the primary selector: replace usages of
this.createBtn.or(this.page.getByTestId('plugins-empty-state')) and any
references to this.pluginList with stable calls like
this.page.getByTestId('<appropriate-testid>') (using .first(), .count(), or
.waitFor(...) on that getByTestId result) so readiness and counting rely on
data-testid; update all occurrences (including the block around the
create/waitFor at lines ~57-59 and the pluginList usage around 116-125) to use
page.getByTestId('<name>') instead of the non-testid locators.
In `@tests/e2e/features/virtual-keys/virtual-keys.spec.ts`:
- Around line 401-407: Remove the call to
virtualKeysPage.page.waitForLoadState('networkidle') and replace the fragile
isVisible-then-assert pattern with a Promise.race between the two element
waiters: start both virtualKeysPage.table.waitFor({ state: 'visible', timeout })
and virtualKeysPage.emptyState.waitFor({ state: 'visible', timeout }) in
parallel, await the Promise.race result to determine which resolved, then assert
visibility of that resolved element (and fail if neither resolves within the
timeout). Apply the same Promise.race-based replacement wherever
virtualKeysPage.page.waitForLoadState('networkidle') and the table/empty-state
branching pattern are used (e.g., the block around lines 417–423).
In `@ui/app/workspace/governance/views/customerDialog.tsx`:
- Line 213: The DialogContent element uses a data-testid value that doesn't
follow the project's three-part selector convention; update the data-testid on
the DialogContent component (symbol: DialogContent) from "customer-dialog" to a
three-part id such as "customer-dialog-content" (entity-element-qualifier) so it
matches the ui/app/**/*.{tsx,ts} test-id pattern and existing selectors; ensure
any tests or references that use the old "customer-dialog" id are also updated
to the new "customer-dialog-content".
---
Outside diff comments:
In `@ui/app/workspace/governance/views/customerDialog.tsx`:
- Around line 242-273: The tokenMaxLimit and requestMaxLimit NumberAndSelect
components are missing data-testid attributes; update the two JSX elements (the
NumberAndSelect with id="tokenMaxLimit" and the one with id="requestMaxLimit")
to include a dataTestId prop (e.g., dataTestId="token-max-limit-input" and
dataTestId="request-max-limit-input") so all interactive limit controls in this
form section have consistent test ids for E2E selectors.
---
Duplicate comments:
In `@tests/e2e/features/governance/pages/governance.page.ts`:
- Around line 113-116: The save/cancel button lookups use
this.page.getByRole(...) and can target the wrong control; change them to scope
to the dialog locators and use getByTestId as the primary selector (e.g.,
replace this.page.getByRole(...) with this.teamDialog.getByTestId(...) for team
dialog actions, and similarly use this.customerDialog.getByTestId(...) for
customer dialog actions), then perform the click and await
this.waitForSuccessToast() and the expect(this.teamDialog).not.toBeVisible(...)
as before; apply the same scoping/selector change for the other occurrences
referenced (lines near 128-131, 156-159, 200-203, 218-221) so all dialog actions
are tied to their dialog locator.
- Around line 6-30: The TeamConfig and CustomerConfig types expose
budget.resetDuration and the entire rateLimit structure which the governance
page object never applies; remove budget.resetDuration and the rateLimit
property from both TeamConfig and CustomerConfig (and anywhere they are
referenced) so the interfaces only include fields the page object actually uses
(e.g., name, customerId/customerName, and budget.maxLimit); update any tests or
factories that set the removed fields to stop passing them or to map them to
supported fields.
- Around line 119-126: The deleteTeam helper returns as soon as the success
toast appears, which can race with the table update; after clicking confirm
(currently using confirmDialog.getByRole('button', { name: /Delete/i }).click())
and awaiting this.waitForSuccessToast(), also wait for the deleted team's row to
be removed before returning — e.g., capture the delete button locator
(deleteBtn) or its row ancestor and call a wait-for-detached/hidden on that
locator (locator.waitFor({ state: 'detached' }) or equivalent) so callers
asserting non-existence won't race; apply the same change to the other delete
helper referenced in the diff (the duplicate helper at lines 162-171).
In `@tests/e2e/features/model-limits/pages/model-limits.page.ts`:
- Around line 47-50: The model selection currently uses .first() which ignores
the requested config.modelName and makes create/edit/delete flaky; change the
selection logic in the sheet flow to locate the dropdown option whose text
equals config.modelName and click that option (instead of using .first()), and
apply the same deterministic selection change to the other occurrence noted (the
block covering the create/edit flows around the second occurrence). Reference
config.modelName and replace any .first() usage in the model-picker code path so
the selected model matches config.modelName exactly.
In `@tests/e2e/features/observability/pages/observability.page.ts`:
- Around line 52-55: getConnectorToggle currently falls back to an unscoped
this.page.locator('button[role="switch"]').first() which is nondeterministic;
update ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS to include testIDs for every
ObservabilityConnector enum value and change getConnectorToggle(connector) to
only use this.page.getByTestId(testId) and throw a clear error if a mapping is
missing (do not use .first() fallback). Also audit callers like
getCurrentState() and disableAllConnectors() to rely on the strict
getByTestId-based selector so they remain deterministic.
In `@ui/app/workspace/model-limits/views/modelLimitsTable.tsx`:
- Line 118: The container div's data-testid "model-limits-table" violates the
3-part convention; update it to a 3-part ID like "model-limits-table-container"
(or "model-limits-table-root") in modelLimitsTable.tsx (the div with data-testid
inside the ModelLimitsTable view) and then update the corresponding
page-object/test selectors under tests/e2e to use the new three-part identifier
so all references remain in sync.
---
Nitpick comments:
In `@tests/e2e/features/config/config.spec.ts`:
- Around line 161-168: Extract a reusable availability guard to replace the
repeated isVisible().catch(() => false) + test.skip pattern: add a helper (e.g.,
skipIfNotVisible(pageObjectElement, testContext, reason)) that checks await
pageObjectElement.isVisible().catch(() => false) and calls
testContext.skip(true, reason) when false; update tests that use
configSettingsPage.asyncJobResultTtlInput (and the other occurrences) to call
this helper at the start of the test instead of inlining the visibility check so
skip behavior is centralized and consistent.
In `@tests/e2e/features/providers/providers.spec.ts`:
- Around line 329-338: The long fixed 9000ms sleep in the test should be
replaced with a deterministic wait: after calling
providersPage.toggleKeyEnabled(keyData.name) use a targeted wait that watches
for the toggle state change (e.g., waitForFunction/waitForSelector against the
toggle element's data-state attribute or aria-checked, or waitForResponse for
the request the toggle triggers) instead of providersPage.page.waitForTimeout;
update the test to poll via providersPage.getKeyEnabledState(keyData.name) or
wait for the specific DOM attribute to reflect the disabled state and proceed as
soon as that condition is true.
In `@tests/e2e/features/virtual-keys/virtual-keys.spec.ts`:
- Around line 121-122: Replace direct page.locator calls in the spec with
page-object properties: add budgetMaxLimitInput and tokenMaxLimitInput to the
VirtualKeysPage (virtual-keys.page.ts) using
this.page.getByTestId('budget-max-limit-input') and
this.page.getByTestId('token-max-limit-input'), then update the spec to
reference virtualKeysPage.budgetMaxLimitInput (and tokenMaxLimitInput for the
other occurrence) when asserting values like SAMPLE_BUDGETS.small.maxLimit so
all raw selectors are encapsulated in the page object.
- Around line 369-375: Replace single-shot checks after visibility toggles with
retrying assertions: after calling virtualKeysPage.toggleKeyVisibility(vkName)
use expect.poll(() => virtualKeysPage.isKeyRevealed(vkName)) to wait until it
resolves to the expected boolean, and do the same for the hide transition.
Update the two places that currently call virtualKeysPage.isKeyRevealed(vkName)
followed by expect(...).toBe(...) so they use expect.poll to repeatedly read
virtualKeysPage.isKeyRevealed until it returns true (for show) or false (for
hide).
ℹ️ Review info
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (58)
.github/workflows/e2e-tests.yml.github/workflows/scripts/test-e2e-ui.shtests/e2e/core/fixtures/base.fixture.tstests/e2e/core/utils/selectors.tstests/e2e/features/config/config.spec.tstests/e2e/features/config/pages/config-settings.page.tstests/e2e/features/dashboard/dashboard.spec.tstests/e2e/features/governance/governance.data.tstests/e2e/features/governance/governance.spec.tstests/e2e/features/governance/pages/governance.page.tstests/e2e/features/mcp-auth-config/mcp-auth-config.spec.tstests/e2e/features/mcp-auth-config/pages/mcp-auth-config.page.tstests/e2e/features/mcp-registry/mcp-registry.spec.tstests/e2e/features/mcp-registry/pages/mcp-registry.page.tstests/e2e/features/mcp-settings/mcp-settings.spec.tstests/e2e/features/mcp-settings/pages/mcp-settings.page.tstests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.tstests/e2e/features/mcp-tool-groups/pages/mcp-tool-groups.page.tstests/e2e/features/model-limits/model-limits.data.tstests/e2e/features/model-limits/model-limits.spec.tstests/e2e/features/model-limits/pages/model-limits.page.tstests/e2e/features/observability/observability.spec.tstests/e2e/features/observability/pages/observability.page.tstests/e2e/features/placeholders/placeholders.spec.tstests/e2e/features/plugins/pages/plugins.page.tstests/e2e/features/plugins/plugins.spec.tstests/e2e/features/providers/pages/providers.page.tstests/e2e/features/providers/providers.spec.tstests/e2e/features/routing-rules/pages/routing-rules.page.tstests/e2e/features/routing-rules/routing-rules.spec.tstests/e2e/features/virtual-keys/pages/virtual-keys.page.tstests/e2e/features/virtual-keys/virtual-keys.spec.tstests/e2e/playwright.config.tsui/app/workspace/config/views/clientSettingsView.tsxui/app/workspace/config/views/mcpView.tsxui/app/workspace/config/views/pricingConfigView.tsxui/app/workspace/config/views/securityView.tsxui/app/workspace/governance/views/customerDialog.tsxui/app/workspace/governance/views/customerTable.tsxui/app/workspace/governance/views/teamDialog.tsxui/app/workspace/governance/views/teamsTable.tsxui/app/workspace/mcp-registry/views/mcpClientsTable.tsxui/app/workspace/mcp-registry/views/mcpServersEmptyState.tsxui/app/workspace/model-limits/views/modelLimitSheet.tsxui/app/workspace/model-limits/views/modelLimitsEmptyState.tsxui/app/workspace/model-limits/views/modelLimitsTable.tsxui/app/workspace/plugins/fragments/pluginFormFragments.tsxui/app/workspace/plugins/views/pluginsEmptyState.tsxui/app/workspace/providers/fragments/debuggingFormFragment.tsxui/app/workspace/providers/views/addProviderDropdown.tsxui/app/workspace/providers/views/modelProviderKeysTableView.tsxui/app/workspace/providers/views/providersEmptyState.tsxui/app/workspace/routing-rules/views/routingRulesEmptyState.tsxui/app/workspace/routing-rules/views/routingRulesTable.tsxui/app/workspace/routing-rules/views/routingRulesView.tsxui/app/workspace/virtual-keys/views/virtualKeysEmptyState.tsxui/app/workspace/virtual-keys/views/virtualKeysTable.tsxui/components/ui/numberAndSelect.tsx
✅ Files skipped from review due to trivial changes (2)
- ui/app/workspace/providers/views/addProviderDropdown.tsx
- .github/workflows/e2e-tests.yml
🚧 Files skipped from review as they are similar to previous changes (36)
- ui/app/workspace/config/views/clientSettingsView.tsx
- ui/app/workspace/governance/views/customerTable.tsx
- tests/e2e/features/plugins/plugins.spec.ts
- ui/app/workspace/mcp-registry/views/mcpClientsTable.tsx
- ui/app/workspace/providers/fragments/debuggingFormFragment.tsx
- tests/e2e/features/mcp-settings/mcp-settings.spec.ts
- tests/e2e/features/dashboard/dashboard.spec.ts
- ui/app/workspace/plugins/views/pluginsEmptyState.tsx
- ui/app/workspace/config/views/pricingConfigView.tsx
- ui/app/workspace/governance/views/teamsTable.tsx
- ui/app/workspace/routing-rules/views/routingRulesEmptyState.tsx
- ui/app/workspace/model-limits/views/modelLimitsEmptyState.tsx
- ui/app/workspace/plugins/fragments/pluginFormFragments.tsx
- ui/app/workspace/virtual-keys/views/virtualKeysTable.tsx
- tests/e2e/features/model-limits/model-limits.data.ts
- tests/e2e/features/placeholders/placeholders.spec.ts
- ui/components/ui/numberAndSelect.tsx
- ui/app/workspace/routing-rules/views/routingRulesView.tsx
- tests/e2e/features/mcp-settings/pages/mcp-settings.page.ts
- ui/app/workspace/virtual-keys/views/virtualKeysEmptyState.tsx
- tests/e2e/features/virtual-keys/pages/virtual-keys.page.ts
- ui/app/workspace/governance/views/teamDialog.tsx
- tests/e2e/features/mcp-tool-groups/pages/mcp-tool-groups.page.ts
- tests/e2e/features/model-limits/model-limits.spec.ts
- ui/app/workspace/providers/views/providersEmptyState.tsx
- tests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.ts
- tests/e2e/features/mcp-registry/pages/mcp-registry.page.ts
- ui/app/workspace/config/views/mcpView.tsx
- tests/e2e/features/mcp-auth-config/pages/mcp-auth-config.page.ts
- tests/e2e/features/routing-rules/pages/routing-rules.page.ts
- tests/e2e/core/utils/selectors.ts
- ui/app/workspace/routing-rules/views/routingRulesTable.tsx
- ui/app/workspace/config/views/securityView.tsx
- tests/e2e/features/mcp-auth-config/mcp-auth-config.spec.ts
- tests/e2e/core/fixtures/base.fixture.ts
- tests/e2e/features/config/pages/config-settings.page.ts
Radheshg04
force-pushed
the
02-27-feat_extend_e2e_ui_tests
branch
from
7ca417a to
6e22595
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (5)
tests/e2e/features/observability/pages/observability.page.ts (1)
52-55:⚠️ Potential issue | 🟠 MajorUnscoped switch fallback is still nondeterministic for non-mapped connectors.
Line 54 falls back to
button[role="switch"]+.first(), which can hit the wrong toggle when multiple switches exist in the panel. This makesisConnectorEnabled/toggleConnectorflaky for connectors without dedicated test IDs.As per coding guidelines "
tests/e2e/**/*.ts: E2E test page objects must extend BasePage, use getByTestId() as the primary selector strategy, and import test/expect from ../../core/fixtures/base.fixture, never from@playwright/test."🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/observability/pages/observability.page.ts` around lines 52 - 55, The fallback in getConnectorToggle is nondeterministic because it uses this.page.locator('button[role="switch"]').first(); update getConnectorToggle (and any callers like isConnectorEnabled / toggleConnector) to scope the switch lookup to the connector's row or panel (e.g., find the row by connector name/label or a nearby test-id) and then call getByTestId/getByRole on that scoped locator instead of a global .first(); also ensure ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS remains the primary selector source and that this page object extends BasePage and test fixtures import test/expect from ../../core/fixtures/base.fixture per E2E guidelines.tests/e2e/features/config/config.spec.ts (1)
125-133:⚠️ Potential issue | 🟠 MajorGuard the URL validation test before typing into the input.
At Line 126,
fill()runs before the RBAC/save guard. In read-only setups, the test can fail before it reachestest.skip(...).🔧 Suggested patch
test('should validate URL format', async ({ configSettingsPage }) => { - await configSettingsPage.pricingDatasheetUrlInput.fill('invalid-url-no-http') + const canEdit = await configSettingsPage.pricingDatasheetUrlInput.isEditable().catch(() => false) + if (!canEdit) { + test.skip(true, 'Pricing datasheet input not editable (RBAC)') + return + } + await configSettingsPage.pricingDatasheetUrlInput.fill('invalid-url-no-http') const canSave = await configSettingsPage.pricingSaveBtn.isDisabled().then((d) => !d) if (!canSave) { test.skip(true, 'Save button disabled (RBAC)') return } await configSettingsPage.pricingSaveBtn.click()🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/config/config.spec.ts` around lines 125 - 133, The test types into pricingDatasheetUrlInput before checking RBAC/save state, which causes failures in read-only setups; change the flow in the test 'should validate URL format' to first check pricingSaveBtn.isDisabled() (using the existing canSave logic) and call test.skip if save is disabled, and only then call pricingDatasheetUrlInput.fill(...) and subsequent pricingSaveBtn.click(); reference the pricingDatasheetUrlInput and pricingSaveBtn locators when moving the guard so the RBAC check runs before any input interactions.tests/e2e/features/mcp-settings/mcp-settings.spec.ts (1)
12-15:⚠️ Potential issue | 🟠 MajorUse
MCPSettingsPagetest-id-backed locators instead of label selectors.Lines 13-14 bypass the page object’s stable locators and couple the test to UI text.
🔧 Suggested patch
test('should display MCP settings form fields', async ({ mcpSettingsPage }) => { - await expect(mcpSettingsPage.page.getByLabel('Max Agent Depth')).toBeVisible() - await expect(mcpSettingsPage.page.getByLabel('Tool Execution Timeout (seconds)')).toBeVisible() + await expect(mcpSettingsPage.maxAgentDepthInput).toBeVisible() + await expect(mcpSettingsPage.toolTimeoutInput).toBeVisible() })As per coding guidelines, E2E page objects should use
getByTestId()as the primary selector strategy.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/mcp-settings/mcp-settings.spec.ts` around lines 12 - 15, Replace the label-based selectors with the MCPSettingsPage's stable test-id locators: instead of mcpSettingsPage.page.getByLabel('Max Agent Depth') and mcpSettingsPage.page.getByLabel('Tool Execution Timeout (seconds)'), use the page object's test-id backed locators (e.g. mcpSettingsPage.getByTestId('max-agent-depth') or the provided properties like mcpSettingsPage.maxAgentDepthLocator and mcpSettingsPage.toolExecutionTimeoutLocator) and assert visibility on those locators.tests/e2e/features/providers/pages/providers.page.ts (1)
409-417:⚠️ Potential issue | 🟠 MajorSwitch debugging save/toggle selectors to
getByTestId()(still text/DOM-coupled).These methods still rely on accessible text and structural traversal, which is brittle and outside the page-object selector rule.
🔧 Suggested update
getConfigSaveBtn(configType: 'network' | 'proxy' | 'performance' | 'governance' | 'debugging'): Locator { + if (configType === 'debugging') { + return this.page.getByTestId('provider-config-debugging-save-btn') + } const buttonNames: Record<string, string> = { network: 'Save Network Configuration', proxy: 'Save Proxy Configuration', performance: 'Save Performance Configuration', governance: 'Save Governance Configuration', debugging: 'Save Debugging Configuration', } return this.page.getByRole('button', { name: buttonNames[configType] }) } getRawRequestSwitch(): Locator { - return this.page.getByLabel('Send Back Raw Request').locator('..').locator('button[role="switch"]') + return this.page.getByTestId('send-back-raw-request-switch') } getRawResponseSwitch(): Locator { - return this.page.getByLabel('Send Back Raw Response').locator('..').locator('button[role="switch"]') + return this.page.getByTestId('send-back-raw-response-switch') }#!/bin/bash # Verify current selectors in page object and whether UI has matching test IDs. rg -n "getConfigSaveBtn|getRawRequestSwitch|getRawResponseSwitch|getByRole\\('button'.*Save|getByLabel\\('Send Back Raw" tests/e2e/features/providers/pages/providers.page.ts -C2 rg -n "data-testid=.*(provider-config-debugging-save-btn|send-back-raw-request-switch|send-back-raw-response-switch)" ui/app/workspace/providers/fragments/debuggingFormFragment.tsxAs per coding guidelines "
tests/e2e/**/*.ts: E2E test page objects must extend BasePage, use getByTestId() as the primary selector strategy, and import test/expect from ../../core/fixtures/base.fixture, never from@playwright/test."Also applies to: 442-450
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/providers/pages/providers.page.ts` around lines 409 - 417, Update the brittle text-based selectors in getConfigSaveBtn to use getByTestId() with the debugging save button test id (e.g. provider-config-debugging-save-btn) and likewise change the related methods getRawRequestSwitch and getRawResponseSwitch to use getByTestId() with send-back-raw-request-switch and send-back-raw-response-switch respectively; ensure the page object still extends BasePage and that test fixtures/imports follow the project rule (use core/fixtures/base.fixture) so selectors are test-id based and imports conform to the E2E page-object guidelines.tests/e2e/features/providers/providers.spec.ts (1)
464-495:⚠️ Potential issue | 🟠 MajorAlways restore debugging settings in a
finallyblock.If this test fails before the restore section, mutated persisted settings can leak into later tests.
🔧 Suggested update
test('should toggle and save raw request/response', async ({ providersPage }) => { await providersPage.selectConfigTab('debugging') const rawRequestSwitch = providersPage.getRawRequestSwitch() const rawResponseSwitch = providersPage.getRawResponseSwitch() // Capture original states const originalRawRequest = await rawRequestSwitch.getAttribute('data-state') === 'checked' const originalRawResponse = await rawResponseSwitch.getAttribute('data-state') === 'checked' - // Toggle both switches - await rawRequestSwitch.click() - await rawResponseSwitch.click() - - // Save and verify success - const saveBtn = providersPage.getConfigSaveBtn('debugging') - await expect(saveBtn).toBeEnabled() - await providersPage.saveDebuggingConfig() - - // Restore original states - const currentRawRequest = await rawRequestSwitch.getAttribute('data-state') === 'checked' - const currentRawResponse = await rawResponseSwitch.getAttribute('data-state') === 'checked' - - if (currentRawRequest !== originalRawRequest) { - await rawRequestSwitch.click() - } - if (currentRawResponse !== originalRawResponse) { - await rawResponseSwitch.click() - } - - await providersPage.saveDebuggingConfig() + try { + await rawRequestSwitch.click() + await rawResponseSwitch.click() + + const saveBtn = providersPage.getConfigSaveBtn('debugging') + await expect(saveBtn).toBeEnabled() + await providersPage.saveDebuggingConfig() + } finally { + const currentRawRequest = await rawRequestSwitch.getAttribute('data-state') === 'checked' + const currentRawResponse = await rawResponseSwitch.getAttribute('data-state') === 'checked' + + if (currentRawRequest !== originalRawRequest) await rawRequestSwitch.click() + if (currentRawResponse !== originalRawResponse) await rawResponseSwitch.click() + + const restoreBtn = providersPage.getConfigSaveBtn('debugging') + if (await restoreBtn.isEnabled().catch(() => false)) { + await providersPage.saveDebuggingConfig() + } + } })As per coding guidelines "
**: always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)".🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/providers/providers.spec.ts` around lines 464 - 495, The test's teardown that restores the original rawRequest/rawResponse settings must be moved into a finally block so settings are always restored even if assertions fail; capture originalRawRequest and originalRawResponse as you already do, perform the toggles and save inside a try, and in the finally use rawRequestSwitch and rawResponseSwitch to read current state and click to revert any differences then call providersPage.saveDebuggingConfig() (awaiting each async action) to persist the restoration; update the test function named "should toggle and save raw request/response" to wrap the main test steps in try/finally and ensure all awaits remain.
🧹 Nitpick comments (8)
ui/app/workspace/providers/views/modelProviderKeysTableView.tsx (1)
173-180: Consider making test IDs unique within the loop for consistency.These new test IDs are static but appear inside a
.map()loop, meaning multiple rows will share the samedata-testid. This is inconsistent with the existing pattern in this file where other elements within the loop use dynamic suffixes (e.g.,key-row-${key.name},key-status-success-${key.name}).Static test IDs can cause ambiguity in e2e tests when targeting specific rows.
♻️ Proposed fix to add unique suffixes
- <TableCell data-testid="key-weight-value"> + <TableCell data-testid={`key-weight-value-${key.name}`}> <div className="flex items-center space-x-2"> <span className="font-mono text-sm">{key.weight}</span> </div> </TableCell> <TableCell> <Switch - data-testid="key-enabled-switch" + data-testid={`key-enabled-switch-${key.name}`} checked={isKeyEnabled}🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/providers/views/modelProviderKeysTableView.tsx` around lines 173 - 180, The two test IDs inside the row render (data-testid="key-weight-value" and data-testid="key-enabled-switch") are static but rendered inside a .map() loop; update them to include a unique suffix (for example using the row identifier already used elsewhere like key.name) so they match the pattern used for key-row-{key.name} and key-status-success-{key.name}; locate the TableCell rendering the weight and the Switch component and change their data-testid values to something like `key-weight-value-${key.name}` and `key-enabled-switch-${key.name}` (or use another unique property on the key) to ensure each row has unique test IDs.tests/e2e/features/plugins/plugins.spec.ts (1)
251-263: Add toast cleanup after error-branch assertions.Both error-handling branches assert correctly now, but they leave error toasts visible. Dismissing them at the end of each block helps isolate subsequent tests.
🧹 Suggested cleanup patch
if (hasInlineError) { await expect(pluginsPage.sheet).toBeVisible() await expect(inlineError).toBeVisible() await pluginsPage.cancelPlugin() } else { await expect(errorToast).toBeVisible() } + await pluginsPage.dismissToasts()if (hasInlineError) { await expect(pluginsPage.sheet).toBeVisible() await expect(inlineError).toBeVisible() await pluginsPage.cancelPlugin() } else { await expect(errorToast).toBeVisible() } + await pluginsPage.dismissToasts()Also applies to: 352-364
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/plugins/plugins.spec.ts` around lines 251 - 263, The test leaves error toasts visible after both branches of the duplicate-creation assertion; after the inline-error branch (where you call pluginsPage.cancelPlugin()) and after the error-toast branch you should dismiss the toast to avoid leaking UI state; locate the toast using the existing errorToast locator (pluginsPage.page.locator('[data-sonner-toast][data-type="error"]').first()) and either click its close/dismiss button or call click() on the toast then wait for it to be hidden (waitFor state 'hidden'), and apply the same cleanup to the analogous block around lines 352-364.tests/e2e/features/plugins/pages/plugins.page.ts (1)
117-125: Avoid transient zero-counts ingetPluginCount().
getPluginCount()can still return0during async loading when empty state isn’t visible yet. Consider waiting for either empty state or first list item before counting to reduce flakes.♻️ Suggested reliability tweak
async getPluginCount(): Promise<number> { - // Check if it's empty state (no plugin list, only empty state is shown) const emptyState = this.page.getByTestId('plugins-empty-state') - const isEmptyVisible = await emptyState.isVisible().catch(() => false) + const pluginItems = this.page.getByTestId('plugin-list-item') + await emptyState + .or(pluginItems.first()) + .waitFor({ state: 'visible', timeout: 10000 }) + .catch(() => {}) + const isEmptyVisible = await emptyState.isVisible().catch(() => false) if (isEmptyVisible) { return 0 } - - const buttons = this.page.getByTestId('plugin-list-item') - const count = await buttons.count() - - return count + return await pluginItems.count() }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/plugins/pages/plugins.page.ts` around lines 117 - 125, getPluginCount() can return 0 transiently while the plugin list is still loading; change the logic to wait for either the empty-state or at least one list item before counting. Specifically, use the existing locators (emptyState = getByTestId('plugins-empty-state') and buttons = getByTestId('plugin-list-item')) and await one of: emptyState.waitFor({ state: 'visible' }) OR buttons.first().waitFor({ state: 'attached' }) (or use page.waitForSelector for those test ids), then if emptyState is visible return 0 else return buttons.count(); update the getPluginCount() implementation to perform this wait before computing the count.tests/e2e/features/governance/pages/governance.page.ts (1)
109-110: Scope budget input lookup to the active dialog for stability.These budget field locators are page-scoped; prefer
this.teamDialog.getByTestId(...)/this.customerDialog.getByTestId(...)to avoid cross-dialog collisions in mixed DOM states.Proposed refactor
- const budgetInput = this.page.getByTestId('budget-max-limit-input') + const budgetInput = this.teamDialog.getByTestId('budget-max-limit-input') await budgetInput.fill(String(config.budget.maxLimit))- const budgetInput = this.page.getByTestId('budget-max-limit-input') + const budgetInput = this.customerDialog.getByTestId('budget-max-limit-input') await budgetInput.fill(String(config.budget.maxLimit))- const budgetInput = this.page.getByTestId('budget-max-limit-input') + const budgetInput = this.teamDialog.getByTestId('budget-max-limit-input') await budgetInput.clear() await budgetInput.fill(String(updates.budget.maxLimit))As per coding guidelines "
tests/e2e/**/*.ts: E2E test page objects must extend BasePage, use getByTestId() as the primary selector strategy..."Also applies to: 154-155, 199-201
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/governance/pages/governance.page.ts` around lines 109 - 110, The budget input locator is page-scoped (const budgetInput = this.page.getByTestId('budget-max-limit-input')) and can collide with other dialogs; replace it with the appropriate dialog-scoped locator (e.g., this.teamDialog.getByTestId('budget-max-limit-input') or this.customerDialog.getByTestId('budget-max-limit-input') depending on which dialog is active) so the lookup is scoped to the active dialog; apply the same change to the other similar budget field lookups in this file (the other occurrences of this.page.getByTestId for budget inputs).tests/e2e/features/observability/pages/observability.page.ts (1)
102-108: Return selected connector key instead of display text for stable assertions.
getSelectedConnector()currently returnstextContent(), which is copy-sensitive. Since selection already usesdata-testid, returning the connector key (otel,prometheus, etc.) would make specs less brittle.Suggested refactor
- async getSelectedConnector(): Promise<string | null> { + async getSelectedConnector(): Promise<ObservabilityConnector | null> { // Observability view uses plain buttons with aria-current="page" for the selected tab const selected = this.page.locator('[data-testid^="observability-provider-btn-"][aria-current="page"]') const isVisible = await selected.isVisible().catch(() => false) if (!isVisible) return null - return await selected.textContent() + const testId = await selected.getAttribute('data-testid') + if (!testId) return null + return testId.replace('observability-provider-btn-', '') as ObservabilityConnector }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/observability/pages/observability.page.ts` around lines 102 - 108, getSelectedConnector currently returns the button's textContent which is brittle; change it to return the connector key parsed from the data-testid attribute. In getSelectedConnector(): locate the selected button using the same selector ('[data-testid^="observability-provider-btn-"][aria-current="page"]'), check visibility as before, then call getAttribute('data-testid') on the locator, extract and return the suffix after the 'observability-provider-btn-' prefix (e.g. 'otel', 'prometheus'), returning null if not visible or attribute is missing.ui/app/workspace/config/views/mcpView.tsx (1)
111-227: Add a test ID for the Tool Sync Interval input to complete MCP selector coverage.The MCP form now has stable selectors for most controls, but the interactive input at Line 163 (
mcp-tool-sync-interval) still lacks adata-testid.🔧 Suggested patch
<Input id="mcp-tool-sync-interval" + data-testid="mcp-tool-sync-interval-input" type="number" className="w-24" value={localValues.mcp_tool_sync_interval}As per coding guidelines,
ui/app/workspace/**/*.tsxpages must includedata-testidattributes on all interactive elements.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/config/views/mcpView.tsx` around lines 111 - 227, The Tool Sync Interval numeric Input with id "mcp-tool-sync-interval" is missing a data-testid which prevents stable selector coverage; update the <Input> for mcp-tool-sync-interval (the element using value={localValues.mcp_tool_sync_interval} and onChange={handleToolSyncIntervalChange}) to include a data-testid attribute (e.g. data-testid="mcp-tool-sync-interval-input") so test selectors can reliably target this control.tests/e2e/features/providers/pages/providers.page.ts (1)
168-172: UseProviderLabelsinstead of a local hard-coded provider label map.
getBaseProviderLabel()can drift from UI source-of-truth labels and breakfillSelectwhen display names evolve.Based on learnings "In E2E tests, use the authoritative ProviderLabels constant exported from ui/lib/constants/logs.ts as the source of truth for provider display names."
Also applies to: 363-372
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/providers/pages/providers.page.ts` around lines 168 - 172, Replace the hard-coded/local label mapping used by getBaseProviderLabel when calling fillSelect with the authoritative ProviderLabels constant from ui/lib/constants/logs.ts so tests use the UI source-of-truth; update the calls in providers.page.ts (including the instances around getBaseProviderLabel at the shown locations and the similar block at lines 363-372) to look up the display name via ProviderLabels[config.baseProviderType] (or the appropriate key) and pass that into fillSelect instead of calling getBaseProviderLabel.tests/e2e/features/providers/providers.spec.ts (1)
334-337: Replace the fixed 9s sleep with state-based polling.The static wait increases runtime and can still be flaky under variable backend latency. Use
expect.poll()for robust state verification instead.🔧 Suggested update
- await providersPage.toggleKeyEnabled(keyData.name) - await providersPage.page.waitForTimeout(9000) - isEnabled = await providersPage.getKeyEnabledState(keyData.name) - expect(isEnabled).toBe(false) + await providersPage.toggleKeyEnabled(keyData.name) + await expect + .poll(() => providersPage.getKeyEnabledState(keyData.name), { + timeout: 10000, + intervals: [250, 500, 1000], + }) + .toBe(false)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/providers/providers.spec.ts` around lines 334 - 337, The test uses a fixed 9s sleep after calling providersPage.toggleKeyEnabled(keyData.name); replace that with state-based polling using expect.poll to wait until providersPage.getKeyEnabledState(keyData.name) becomes false; remove providersPage.page.waitForTimeout(9000) and instead call expect.poll(() => providersPage.getKeyEnabledState(keyData.name)).toBe(false) (optionally supplying timeout/interval options) so the assertion retries until the backend state changes.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/e2e/features/mcp-registry/mcp-registry.spec.ts`:
- Around line 73-74: The assertion reads the connection cell immediately and can
flake because the row may appear before its cell text stabilizes; change the
test to poll until mcpRegistryPage.getClientConnectionType(clientData.name)
returns 'HTTP' (or until a reasonable timeout) rather than asserting immediately
— use a retry/wait loop that first ensures
mcpRegistryPage.clientExists(clientData.name) and then repeatedly calls
getClientConnectionType until the value equals 'HTTP' to stabilize the
assertion.
In `@ui/app/workspace/routing-rules/views/routingRulesTable.tsx`:
- Line 128: Update the span with data-testid "routing-rule-description" to
follow the 3-part convention by changing it to "routing-rule-description-text"
in the routingRulesTable component (the span rendering rule.description with
className "text-xs text-muted-foreground truncate max-w-xs"); also search and
update any E2E references under tests/e2e that use the old test id so tests keep
working.
---
Duplicate comments:
In `@tests/e2e/features/config/config.spec.ts`:
- Around line 125-133: The test types into pricingDatasheetUrlInput before
checking RBAC/save state, which causes failures in read-only setups; change the
flow in the test 'should validate URL format' to first check
pricingSaveBtn.isDisabled() (using the existing canSave logic) and call
test.skip if save is disabled, and only then call
pricingDatasheetUrlInput.fill(...) and subsequent pricingSaveBtn.click();
reference the pricingDatasheetUrlInput and pricingSaveBtn locators when moving
the guard so the RBAC check runs before any input interactions.
In `@tests/e2e/features/mcp-settings/mcp-settings.spec.ts`:
- Around line 12-15: Replace the label-based selectors with the
MCPSettingsPage's stable test-id locators: instead of
mcpSettingsPage.page.getByLabel('Max Agent Depth') and
mcpSettingsPage.page.getByLabel('Tool Execution Timeout (seconds)'), use the
page object's test-id backed locators (e.g.
mcpSettingsPage.getByTestId('max-agent-depth') or the provided properties like
mcpSettingsPage.maxAgentDepthLocator and
mcpSettingsPage.toolExecutionTimeoutLocator) and assert visibility on those
locators.
In `@tests/e2e/features/observability/pages/observability.page.ts`:
- Around line 52-55: The fallback in getConnectorToggle is nondeterministic
because it uses this.page.locator('button[role="switch"]').first(); update
getConnectorToggle (and any callers like isConnectorEnabled / toggleConnector)
to scope the switch lookup to the connector's row or panel (e.g., find the row
by connector name/label or a nearby test-id) and then call getByTestId/getByRole
on that scoped locator instead of a global .first(); also ensure
ObservabilityPage.CONNECTOR_TOGGLE_TESTIDS remains the primary selector source
and that this page object extends BasePage and test fixtures import test/expect
from ../../core/fixtures/base.fixture per E2E guidelines.
In `@tests/e2e/features/providers/pages/providers.page.ts`:
- Around line 409-417: Update the brittle text-based selectors in
getConfigSaveBtn to use getByTestId() with the debugging save button test id
(e.g. provider-config-debugging-save-btn) and likewise change the related
methods getRawRequestSwitch and getRawResponseSwitch to use getByTestId() with
send-back-raw-request-switch and send-back-raw-response-switch respectively;
ensure the page object still extends BasePage and that test fixtures/imports
follow the project rule (use core/fixtures/base.fixture) so selectors are
test-id based and imports conform to the E2E page-object guidelines.
In `@tests/e2e/features/providers/providers.spec.ts`:
- Around line 464-495: The test's teardown that restores the original
rawRequest/rawResponse settings must be moved into a finally block so settings
are always restored even if assertions fail; capture originalRawRequest and
originalRawResponse as you already do, perform the toggles and save inside a
try, and in the finally use rawRequestSwitch and rawResponseSwitch to read
current state and click to revert any differences then call
providersPage.saveDebuggingConfig() (awaiting each async action) to persist the
restoration; update the test function named "should toggle and save raw
request/response" to wrap the main test steps in try/finally and ensure all
awaits remain.
---
Nitpick comments:
In `@tests/e2e/features/governance/pages/governance.page.ts`:
- Around line 109-110: The budget input locator is page-scoped (const
budgetInput = this.page.getByTestId('budget-max-limit-input')) and can collide
with other dialogs; replace it with the appropriate dialog-scoped locator (e.g.,
this.teamDialog.getByTestId('budget-max-limit-input') or
this.customerDialog.getByTestId('budget-max-limit-input') depending on which
dialog is active) so the lookup is scoped to the active dialog; apply the same
change to the other similar budget field lookups in this file (the other
occurrences of this.page.getByTestId for budget inputs).
In `@tests/e2e/features/observability/pages/observability.page.ts`:
- Around line 102-108: getSelectedConnector currently returns the button's
textContent which is brittle; change it to return the connector key parsed from
the data-testid attribute. In getSelectedConnector(): locate the selected button
using the same selector
('[data-testid^="observability-provider-btn-"][aria-current="page"]'), check
visibility as before, then call getAttribute('data-testid') on the locator,
extract and return the suffix after the 'observability-provider-btn-' prefix
(e.g. 'otel', 'prometheus'), returning null if not visible or attribute is
missing.
In `@tests/e2e/features/plugins/pages/plugins.page.ts`:
- Around line 117-125: getPluginCount() can return 0 transiently while the
plugin list is still loading; change the logic to wait for either the
empty-state or at least one list item before counting. Specifically, use the
existing locators (emptyState = getByTestId('plugins-empty-state') and buttons =
getByTestId('plugin-list-item')) and await one of: emptyState.waitFor({ state:
'visible' }) OR buttons.first().waitFor({ state: 'attached' }) (or use
page.waitForSelector for those test ids), then if emptyState is visible return 0
else return buttons.count(); update the getPluginCount() implementation to
perform this wait before computing the count.
In `@tests/e2e/features/plugins/plugins.spec.ts`:
- Around line 251-263: The test leaves error toasts visible after both branches
of the duplicate-creation assertion; after the inline-error branch (where you
call pluginsPage.cancelPlugin()) and after the error-toast branch you should
dismiss the toast to avoid leaking UI state; locate the toast using the existing
errorToast locator
(pluginsPage.page.locator('[data-sonner-toast][data-type="error"]').first()) and
either click its close/dismiss button or call click() on the toast then wait for
it to be hidden (waitFor state 'hidden'), and apply the same cleanup to the
analogous block around lines 352-364.
In `@tests/e2e/features/providers/pages/providers.page.ts`:
- Around line 168-172: Replace the hard-coded/local label mapping used by
getBaseProviderLabel when calling fillSelect with the authoritative
ProviderLabels constant from ui/lib/constants/logs.ts so tests use the UI
source-of-truth; update the calls in providers.page.ts (including the instances
around getBaseProviderLabel at the shown locations and the similar block at
lines 363-372) to look up the display name via
ProviderLabels[config.baseProviderType] (or the appropriate key) and pass that
into fillSelect instead of calling getBaseProviderLabel.
In `@tests/e2e/features/providers/providers.spec.ts`:
- Around line 334-337: The test uses a fixed 9s sleep after calling
providersPage.toggleKeyEnabled(keyData.name); replace that with state-based
polling using expect.poll to wait until
providersPage.getKeyEnabledState(keyData.name) becomes false; remove
providersPage.page.waitForTimeout(9000) and instead call expect.poll(() =>
providersPage.getKeyEnabledState(keyData.name)).toBe(false) (optionally
supplying timeout/interval options) so the assertion retries until the backend
state changes.
In `@ui/app/workspace/config/views/mcpView.tsx`:
- Around line 111-227: The Tool Sync Interval numeric Input with id
"mcp-tool-sync-interval" is missing a data-testid which prevents stable selector
coverage; update the <Input> for mcp-tool-sync-interval (the element using
value={localValues.mcp_tool_sync_interval} and
onChange={handleToolSyncIntervalChange}) to include a data-testid attribute
(e.g. data-testid="mcp-tool-sync-interval-input") so test selectors can reliably
target this control.
In `@ui/app/workspace/providers/views/modelProviderKeysTableView.tsx`:
- Around line 173-180: The two test IDs inside the row render
(data-testid="key-weight-value" and data-testid="key-enabled-switch") are static
but rendered inside a .map() loop; update them to include a unique suffix (for
example using the row identifier already used elsewhere like key.name) so they
match the pattern used for key-row-{key.name} and key-status-success-{key.name};
locate the TableCell rendering the weight and the Switch component and change
their data-testid values to something like `key-weight-value-${key.name}` and
`key-enabled-switch-${key.name}` (or use another unique property on the key) to
ensure each row has unique test IDs.
ℹ️ Review info
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (59)
.github/workflows/e2e-tests.yml.github/workflows/scripts/test-e2e-ui.shtests/e2e/core/fixtures/base.fixture.tstests/e2e/core/utils/selectors.tstests/e2e/features/config/config.spec.tstests/e2e/features/config/pages/config-settings.page.tstests/e2e/features/dashboard/dashboard.spec.tstests/e2e/features/governance/governance.data.tstests/e2e/features/governance/governance.spec.tstests/e2e/features/governance/pages/governance.page.tstests/e2e/features/mcp-auth-config/mcp-auth-config.spec.tstests/e2e/features/mcp-auth-config/pages/mcp-auth-config.page.tstests/e2e/features/mcp-registry/mcp-registry.spec.tstests/e2e/features/mcp-registry/pages/mcp-registry.page.tstests/e2e/features/mcp-settings/mcp-settings.spec.tstests/e2e/features/mcp-settings/pages/mcp-settings.page.tstests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.tstests/e2e/features/mcp-tool-groups/pages/mcp-tool-groups.page.tstests/e2e/features/model-limits/model-limits.data.tstests/e2e/features/model-limits/model-limits.spec.tstests/e2e/features/model-limits/pages/model-limits.page.tstests/e2e/features/observability/observability.spec.tstests/e2e/features/observability/pages/observability.page.tstests/e2e/features/placeholders/placeholders.spec.tstests/e2e/features/plugins/pages/plugins.page.tstests/e2e/features/plugins/plugins.spec.tstests/e2e/features/providers/pages/providers.page.tstests/e2e/features/providers/providers.spec.tstests/e2e/features/routing-rules/pages/routing-rules.page.tstests/e2e/features/routing-rules/routing-rules.spec.tstests/e2e/features/virtual-keys/pages/virtual-keys.page.tstests/e2e/features/virtual-keys/virtual-keys.spec.tstests/e2e/playwright.config.tsui/app/workspace/config/views/clientSettingsView.tsxui/app/workspace/config/views/mcpView.tsxui/app/workspace/config/views/pricingConfigView.tsxui/app/workspace/config/views/securityView.tsxui/app/workspace/governance/views/customerDialog.tsxui/app/workspace/governance/views/customerTable.tsxui/app/workspace/governance/views/teamDialog.tsxui/app/workspace/governance/views/teamsTable.tsxui/app/workspace/mcp-registry/views/mcpClientsTable.tsxui/app/workspace/mcp-registry/views/mcpServersEmptyState.tsxui/app/workspace/model-limits/views/modelLimitSheet.tsxui/app/workspace/model-limits/views/modelLimitsEmptyState.tsxui/app/workspace/model-limits/views/modelLimitsTable.tsxui/app/workspace/plugins/fragments/pluginFormFragments.tsxui/app/workspace/plugins/page.tsxui/app/workspace/plugins/views/pluginsEmptyState.tsxui/app/workspace/providers/fragments/debuggingFormFragment.tsxui/app/workspace/providers/views/addProviderDropdown.tsxui/app/workspace/providers/views/modelProviderKeysTableView.tsxui/app/workspace/providers/views/providersEmptyState.tsxui/app/workspace/routing-rules/views/routingRulesEmptyState.tsxui/app/workspace/routing-rules/views/routingRulesTable.tsxui/app/workspace/routing-rules/views/routingRulesView.tsxui/app/workspace/virtual-keys/views/virtualKeysEmptyState.tsxui/app/workspace/virtual-keys/views/virtualKeysTable.tsxui/components/ui/numberAndSelect.tsx
🚧 Files skipped from review as they are similar to previous changes (32)
- .github/workflows/scripts/test-e2e-ui.sh
- ui/app/workspace/model-limits/views/modelLimitSheet.tsx
- tests/e2e/features/governance/governance.spec.ts
- ui/app/workspace/mcp-registry/views/mcpClientsTable.tsx
- .github/workflows/e2e-tests.yml
- tests/e2e/features/governance/governance.data.ts
- ui/app/workspace/routing-rules/views/routingRulesView.tsx
- ui/app/workspace/providers/fragments/debuggingFormFragment.tsx
- ui/app/workspace/config/views/clientSettingsView.tsx
- ui/app/workspace/governance/views/customerDialog.tsx
- tests/e2e/features/model-limits/model-limits.data.ts
- ui/app/workspace/governance/views/customerTable.tsx
- ui/app/workspace/mcp-registry/views/mcpServersEmptyState.tsx
- ui/app/workspace/providers/views/providersEmptyState.tsx
- tests/e2e/playwright.config.ts
- tests/e2e/features/mcp-auth-config/mcp-auth-config.spec.ts
- tests/e2e/features/mcp-tool-groups/pages/mcp-tool-groups.page.ts
- tests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.ts
- tests/e2e/core/fixtures/base.fixture.ts
- tests/e2e/features/dashboard/dashboard.spec.ts
- ui/app/workspace/governance/views/teamDialog.tsx
- tests/e2e/features/model-limits/model-limits.spec.ts
- ui/app/workspace/config/views/pricingConfigView.tsx
- ui/app/workspace/routing-rules/views/routingRulesEmptyState.tsx
- tests/e2e/features/routing-rules/pages/routing-rules.page.ts
- ui/app/workspace/providers/views/addProviderDropdown.tsx
- ui/app/workspace/model-limits/views/modelLimitsEmptyState.tsx
- tests/e2e/features/mcp-auth-config/pages/mcp-auth-config.page.ts
- tests/e2e/features/placeholders/placeholders.spec.ts
- ui/app/workspace/virtual-keys/views/virtualKeysEmptyState.tsx
- ui/app/workspace/plugins/fragments/pluginFormFragments.tsx
- ui/app/workspace/governance/views/teamsTable.tsx
Radheshg04
force-pushed
the
02-27-feat_extend_e2e_ui_tests
branch
from
6e22595 to
0bddca0
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (1)
tests/e2e/features/virtual-keys/virtual-keys.spec.ts (1)
401-407:⚠️ Potential issue | 🟠 MajorTable/empty-state branching is still race-prone.
At Line 401 and Line 417, waiting for heading-or-empty then snapshotting
table.isVisible()can mis-branch before the final table/empty state settles. Wait on table vs empty directly, then branch from the resolved state.🔧 Stabilization pattern
- await virtualKeysPage.page.getByRole('heading', { name: /Virtual Keys/i }).or(virtualKeysPage.emptyState).first().waitFor({ state: 'visible', timeout: 10000 }) - const hadTable = await virtualKeysPage.table.isVisible().catch(() => false) - if (!hadTable) { + const state = await Promise.race([ + virtualKeysPage.table.waitFor({ state: 'visible', timeout: 10000 }).then(() => 'table' as const), + virtualKeysPage.emptyState.waitFor({ state: 'visible', timeout: 10000 }).then(() => 'empty' as const), + ]) + if (state === 'empty') { await expect(virtualKeysPage.emptyState).toBeVisible({ timeout: 10000 }) } else { await expect(virtualKeysPage.table).toBeVisible({ timeout: 10000 }) } @@ - await virtualKeysPage.page.getByRole('heading', { name: /Virtual Keys/i }).or(virtualKeysPage.emptyState).first().waitFor({ state: 'visible', timeout: 10000 }) - const tableVisible = await virtualKeysPage.table.isVisible().catch(() => false) - if (tableVisible) { + const state = await Promise.race([ + virtualKeysPage.table.waitFor({ state: 'visible', timeout: 10000 }).then(() => 'table' as const), + virtualKeysPage.emptyState.waitFor({ state: 'visible', timeout: 10000 }).then(() => 'empty' as const), + ]) + if (state === 'table') { test.skip(true, 'Pre-existing virtual keys found; empty-state assertion requires isolated data.') return }#!/bin/bash # Verify all race-prone branching sites in this spec for targeted updates. rg -n -C2 "getByRole\\('heading', \\{ name: /Virtual Keys/i \\}\\)\\.or\\(virtualKeysPage\\.emptyState\\).*waitFor" tests/e2e/features/virtual-keys/virtual-keys.spec.ts rg -n -C2 "table\\.isVisible\\(\\)\\.catch\\(\\) => false\\)|table\\.isVisible\\(\\)\\.catch\\(\\(\\) => false\\)" tests/e2e/features/virtual-keys/virtual-keys.spec.tsAlso applies to: 417-423
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/virtual-keys/virtual-keys.spec.ts` around lines 401 - 407, The branching is race-prone because you wait on virtualKeysPage.page.getByRole(...).or(virtualKeysPage.emptyState) then snapshot virtualKeysPage.table.isVisible(); instead, wait explicitly for the two final UI states (virtualKeysPage.table and virtualKeysPage.emptyState) to resolve and branch from the resolved result: call waitFor({state: 'visible', timeout: ...}) (or use Promise.race on two waitFor calls) for virtualKeysPage.table and virtualKeysPage.emptyState, then check which wait completed to decide whether to assert table visibility or emptyState visibility, using the explicit locators virtualKeysPage.table and virtualKeysPage.emptyState (and keeping the heading wait only if still needed).
🧹 Nitpick comments (5)
tests/e2e/features/config/pages/config-settings.page.ts (1)
285-298: Consider reusingsetInputValueto remove repeated clear/fill code.
setRequiredHeaders,setWorkspaceLoggingHeaders,setAsyncJobResultTtl, andsetPricingDatasheetUrlrepeat logic already present insetInputValue, so this can be simplified for consistency.♻️ Proposed refactor
async setRequiredHeaders(value: string): Promise<void> { - await this.requiredHeadersTextarea.clear() - await this.requiredHeadersTextarea.fill(value) + await this.setInputValue(this.requiredHeadersTextarea, value) } async setWorkspaceLoggingHeaders(value: string): Promise<void> { - await this.workspaceLoggingHeadersTextarea.clear() - await this.workspaceLoggingHeadersTextarea.fill(value) + await this.setInputValue(this.workspaceLoggingHeadersTextarea, value) } async setAsyncJobResultTtl(value: string): Promise<void> { - await this.asyncJobResultTtlInput.clear() - await this.asyncJobResultTtlInput.fill(value) + await this.setInputValue(this.asyncJobResultTtlInput, value) } async setPricingDatasheetUrl(url: string): Promise<void> { - await this.pricingDatasheetUrlInput.clear() - await this.pricingDatasheetUrlInput.fill(url) + await this.setInputValue(this.pricingDatasheetUrlInput, url) }Also applies to: 323-326
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/config/pages/config-settings.page.ts` around lines 285 - 298, Replace the duplicate clear()/fill() sequences in setRequiredHeaders, setWorkspaceLoggingHeaders, setAsyncJobResultTtl (and setPricingDatasheetUrl) by delegating to the existing helper setInputValue: call this.setInputValue(this.requiredHeadersTextarea, value), this.setInputValue(this.workspaceLoggingHeadersTextarea, value), this.setInputValue(this.asyncJobResultTtlInput, value) and similarly for the pricing datasheet input, so each method becomes a single call to setInputValue and removes the explicit clear/fill duplication.tests/e2e/features/providers/providers.spec.ts (1)
329-338: Avoid hardcoded 9-second timeout; prefer event-driven waits.The
waitForTimeout(9000)is a long arbitrary delay that will slow down tests and may still be flaky if the actual operation takes longer. Consider waiting for a network request, state change, or a visual indicator instead.♻️ Suggested refactor
// Toggle to disabled await providersPage.toggleKeyEnabled(keyData.name) - await providersPage.page.waitForTimeout(9000) + // Wait for the toggle state to update after the API call + await expect(async () => { + const state = await providersPage.getKeyEnabledState(keyData.name) + expect(state).toBe(false) + }).toPass({ timeout: 10000 }) - isEnabled = await providersPage.getKeyEnabledState(keyData.name) - expect(isEnabled).toBe(false)Alternatively, if
toggleKeyEnabledalready waits for the success toast, the state should be updated immediately after:await providersPage.toggleKeyEnabled(keyData.name) - await providersPage.page.waitForTimeout(9000) isEnabled = await providersPage.getKeyEnabledState(keyData.name) expect(isEnabled).toBe(false)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/providers/providers.spec.ts` around lines 329 - 338, The test uses a hardcoded 9000ms sleep (providersPage.page.waitForTimeout) which is flaky and slow; replace it with an event-driven wait: after calling providersPage.toggleKeyEnabled(keyData.name) wait for a deterministic signal such as the network response for the toggle request (use page.waitForResponse with the toggle route), a success toast/notification selector to appear/disappear, or a DOM state change (use page.waitForSelector or page.waitForFunction that the enabled-state element reflects the disabled state), then call providersPage.getKeyEnabledState(keyData.name) and assert false; update providersPage.toggleKeyEnabled if it can itself await the success indicator so the test need not wait explicitly.tests/e2e/features/observability/pages/observability.page.ts (1)
240-251: Consider: Simplify connector-to-state mapping.The if/else chain is functional but could be replaced with a mapping object for maintainability. However, the camelCase inconsistency (
newrelic→newRelicEnabled) makes a simple template approach less clean, so the current explicit mapping is acceptable.♻️ Optional: Use a mapping object
+ private static readonly CONNECTOR_STATE_KEYS: Record<ObservabilityConnector, keyof ObservabilityState> = { + otel: 'otelEnabled', + prometheus: 'prometheusEnabled', + maxim: 'maximEnabled', + datadog: 'datadogEnabled', + bigquery: 'bigqueryEnabled', + newrelic: 'newRelicEnabled', + } async getCurrentState(): Promise<ObservabilityState> { // ... for (const connector of connectors) { if (await this.isConnectorAvailable(connector)) { await this.selectConnector(connector) const enabled = await this.isConnectorEnabled(connector) - if (connector === 'otel') state.otelEnabled = enabled - else if (connector === 'prometheus') state.prometheusEnabled = enabled - else if (connector === 'maxim') state.maximEnabled = enabled - else if (connector === 'datadog') state.datadogEnabled = enabled - else if (connector === 'bigquery') state.bigqueryEnabled = enabled - else if (connector === 'newrelic') state.newRelicEnabled = enabled + state[ObservabilityPage.CONNECTOR_STATE_KEYS[connector]] = enabled } }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/observability/pages/observability.page.ts` around lines 240 - 251, The long if/else chain mapping connector names to state flags in the loop using connectors, isConnectorAvailable, selectConnector, and isConnectorEnabled should be simplified: replace the explicit conditional chain with a single mapping object (e.g., const mapping = { otel: 'otelEnabled', prometheus: 'prometheusEnabled', maxim: 'maximEnabled', datadog: 'datadogEnabled', bigquery: 'bigqueryEnabled', newrelic: 'newRelicEnabled' }) and then inside the loop, after obtaining enabled call state[mapping[connector]] = enabled if mapping[connector] exists; this preserves the camelCase newRelic key while removing the repetitive if/else logic and uses the existing functions (isConnectorAvailable, selectConnector, isConnectorEnabled) and the state object.tests/e2e/features/virtual-keys/virtual-keys.spec.ts (1)
117-123: Encapsulate the new budget/rate-limit field selectors in the page object.Line 121 and Line 166 add raw
page.locator('#...')selectors in the spec. Please expose these viavirtualKeysPage(preferablygetByTestId) to keep selector churn localized.♻️ Suggested refactor
- const budgetInput = virtualKeysPage.page.locator('#budgetMaxLimit') - await expect(budgetInput).toHaveValue(String(SAMPLE_BUDGETS.small.maxLimit)) + await expect(virtualKeysPage.budgetMaxLimitInput).toHaveValue( + String(SAMPLE_BUDGETS.small.maxLimit), + ) @@ - const tokenLimitInput = virtualKeysPage.page.locator('#tokenMaxLimit') - await expect(tokenLimitInput).toHaveValue(String(SAMPLE_RATE_LIMITS.tokenOnly.tokenMaxLimit)) + await expect(virtualKeysPage.tokenMaxLimitInput).toHaveValue( + String(SAMPLE_RATE_LIMITS.tokenOnly.tokenMaxLimit), + )// tests/e2e/features/virtual-keys/pages/virtual-keys.page.ts (outside this hunk) readonly budgetMaxLimitInput = this.page.getByTestId('budget-max-limit-input') readonly tokenMaxLimitInput = this.page.getByTestId('token-max-limit-input')As per coding guidelines,
tests/e2e/**/*.ts: “E2E test page objects must extend BasePage, use getByTestId() as the primary selector strategy...”.Also applies to: 163-168
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/e2e/features/virtual-keys/virtual-keys.spec.ts` around lines 117 - 123, The spec is using raw page.locator selectors for budget/rate-limit fields (page.locator('#budgetMaxLimit') and similar) instead of the page object; update the VirtualKeys page object (virtualKeysPage) to expose test-id based accessors (e.g., budgetMaxLimitInput and tokenMaxLimitInput using this.page.getByTestId('budget-max-limit-input') and 'token-max-limit-input'), then replace the raw locators in the spec (lines referencing budgetMaxLimit and tokenMaxLimit) with virtualKeysPage.budgetMaxLimitInput and virtualKeysPage.tokenMaxLimitInput and use those in assertions and interactions (waitForSheetAnimation, toHaveValue, closeSheet remains the same).ui/app/workspace/governance/views/customerDialog.tsx (1)
254-273: Add test IDs for the remaining rate-limit inputs for selector parity.
tokenMaxLimitandrequestMaxLimitare still interactive fields without explicit test IDs in this dialog. Adding them keeps governance form selectors consistent and easier to maintain.♻️ Suggested follow-up
<NumberAndSelect id="tokenMaxLimit" label="Maximum Tokens" value={formData.tokenMaxLimit} selectValue={formData.tokenResetDuration} onChangeNumber={(value) => updateField("tokenMaxLimit", value)} onChangeSelect={(value) => updateField("tokenResetDuration", value)} options={resetDurationOptions} + dataTestId="token-max-limit-input" /> {/* Rate Limit Configuration - Request Limits */} <NumberAndSelect id="requestMaxLimit" label="Maximum Requests" value={formData.requestMaxLimit} selectValue={formData.requestResetDuration} onChangeNumber={(value) => updateField("requestMaxLimit", value)} onChangeSelect={(value) => updateField("requestResetDuration", value)} options={resetDurationOptions} + dataTestId="request-max-limit-input" />As per coding guidelines,
ui/app/workspace/**/*.tsxmust includedata-testidattributes on all interactive elements.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/governance/views/customerDialog.tsx` around lines 254 - 273, Add explicit test IDs to the two NumberAndSelect usages so the rate-limit inputs have parity with other governance selectors: on the NumberAndSelect with id "tokenMaxLimit" add a data-testid (e.g. data-testid="tokenMaxLimit") and a select test id prop (e.g. selectTestId="tokenMaxLimit-select"); on the NumberAndSelect with id "requestMaxLimit" add data-testid="requestMaxLimit" and selectTestId="requestMaxLimit-select" (or the equivalent prop names your NumberAndSelect component expects for number vs select elements) so both the number input and selector are addressable by tests.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/e2e/features/governance/governance.spec.ts`:
- Around line 148-150: The test mutates the cleanup tracker createdCustomers
before the rename actually completes, risking orphaned originals if
governancePage.editCustomer(customerData.name, { name: newName }) fails; fix by
performing the edit first and only updating createdCustomers after the await
succeeds (e.g., call await governancePage.editCustomer(customerData.name, {
name: newName }) then set createdCustomers[createdCustomers.length - 1] =
newName), or use the edit result to set the tracker inside a try/await success
block to ensure the tracker reflects the persisted name.
In `@tests/e2e/features/governance/pages/governance.page.ts`:
- Around line 96-100: The locator for customerOption currently uses substring
matching via filter({ hasText: config.customerName }) which can match
overlapping names; update it to require an exact match by replacing the
substring filter with an exact-text match (e.g., use a RegExp anchored start/end
or Playwright's exact text matching) so only the customer named exactly
config.customerName is selected; apply this change where customerOption is
created (this.page.locator(...).filter(...)) and ensure any regex is properly
escaped for special characters in config.customerName.
---
Duplicate comments:
In `@tests/e2e/features/virtual-keys/virtual-keys.spec.ts`:
- Around line 401-407: The branching is race-prone because you wait on
virtualKeysPage.page.getByRole(...).or(virtualKeysPage.emptyState) then snapshot
virtualKeysPage.table.isVisible(); instead, wait explicitly for the two final UI
states (virtualKeysPage.table and virtualKeysPage.emptyState) to resolve and
branch from the resolved result: call waitFor({state: 'visible', timeout: ...})
(or use Promise.race on two waitFor calls) for virtualKeysPage.table and
virtualKeysPage.emptyState, then check which wait completed to decide whether to
assert table visibility or emptyState visibility, using the explicit locators
virtualKeysPage.table and virtualKeysPage.emptyState (and keeping the heading
wait only if still needed).
---
Nitpick comments:
In `@tests/e2e/features/config/pages/config-settings.page.ts`:
- Around line 285-298: Replace the duplicate clear()/fill() sequences in
setRequiredHeaders, setWorkspaceLoggingHeaders, setAsyncJobResultTtl (and
setPricingDatasheetUrl) by delegating to the existing helper setInputValue: call
this.setInputValue(this.requiredHeadersTextarea, value),
this.setInputValue(this.workspaceLoggingHeadersTextarea, value),
this.setInputValue(this.asyncJobResultTtlInput, value) and similarly for the
pricing datasheet input, so each method becomes a single call to setInputValue
and removes the explicit clear/fill duplication.
In `@tests/e2e/features/observability/pages/observability.page.ts`:
- Around line 240-251: The long if/else chain mapping connector names to state
flags in the loop using connectors, isConnectorAvailable, selectConnector, and
isConnectorEnabled should be simplified: replace the explicit conditional chain
with a single mapping object (e.g., const mapping = { otel: 'otelEnabled',
prometheus: 'prometheusEnabled', maxim: 'maximEnabled', datadog:
'datadogEnabled', bigquery: 'bigqueryEnabled', newrelic: 'newRelicEnabled' })
and then inside the loop, after obtaining enabled call state[mapping[connector]]
= enabled if mapping[connector] exists; this preserves the camelCase newRelic
key while removing the repetitive if/else logic and uses the existing functions
(isConnectorAvailable, selectConnector, isConnectorEnabled) and the state
object.
In `@tests/e2e/features/providers/providers.spec.ts`:
- Around line 329-338: The test uses a hardcoded 9000ms sleep
(providersPage.page.waitForTimeout) which is flaky and slow; replace it with an
event-driven wait: after calling providersPage.toggleKeyEnabled(keyData.name)
wait for a deterministic signal such as the network response for the toggle
request (use page.waitForResponse with the toggle route), a success
toast/notification selector to appear/disappear, or a DOM state change (use
page.waitForSelector or page.waitForFunction that the enabled-state element
reflects the disabled state), then call
providersPage.getKeyEnabledState(keyData.name) and assert false; update
providersPage.toggleKeyEnabled if it can itself await the success indicator so
the test need not wait explicitly.
In `@tests/e2e/features/virtual-keys/virtual-keys.spec.ts`:
- Around line 117-123: The spec is using raw page.locator selectors for
budget/rate-limit fields (page.locator('#budgetMaxLimit') and similar) instead
of the page object; update the VirtualKeys page object (virtualKeysPage) to
expose test-id based accessors (e.g., budgetMaxLimitInput and tokenMaxLimitInput
using this.page.getByTestId('budget-max-limit-input') and
'token-max-limit-input'), then replace the raw locators in the spec (lines
referencing budgetMaxLimit and tokenMaxLimit) with
virtualKeysPage.budgetMaxLimitInput and virtualKeysPage.tokenMaxLimitInput and
use those in assertions and interactions (waitForSheetAnimation, toHaveValue,
closeSheet remains the same).
In `@ui/app/workspace/governance/views/customerDialog.tsx`:
- Around line 254-273: Add explicit test IDs to the two NumberAndSelect usages
so the rate-limit inputs have parity with other governance selectors: on the
NumberAndSelect with id "tokenMaxLimit" add a data-testid (e.g.
data-testid="tokenMaxLimit") and a select test id prop (e.g.
selectTestId="tokenMaxLimit-select"); on the NumberAndSelect with id
"requestMaxLimit" add data-testid="requestMaxLimit" and
selectTestId="requestMaxLimit-select" (or the equivalent prop names your
NumberAndSelect component expects for number vs select elements) so both the
number input and selector are addressable by tests.
ℹ️ Review info
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (59)
.github/workflows/e2e-tests.yml.github/workflows/scripts/test-e2e-ui.shtests/e2e/core/fixtures/base.fixture.tstests/e2e/core/utils/selectors.tstests/e2e/features/config/config.spec.tstests/e2e/features/config/pages/config-settings.page.tstests/e2e/features/dashboard/dashboard.spec.tstests/e2e/features/governance/governance.data.tstests/e2e/features/governance/governance.spec.tstests/e2e/features/governance/pages/governance.page.tstests/e2e/features/mcp-auth-config/mcp-auth-config.spec.tstests/e2e/features/mcp-auth-config/pages/mcp-auth-config.page.tstests/e2e/features/mcp-registry/mcp-registry.spec.tstests/e2e/features/mcp-registry/pages/mcp-registry.page.tstests/e2e/features/mcp-settings/mcp-settings.spec.tstests/e2e/features/mcp-settings/pages/mcp-settings.page.tstests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.tstests/e2e/features/mcp-tool-groups/pages/mcp-tool-groups.page.tstests/e2e/features/model-limits/model-limits.data.tstests/e2e/features/model-limits/model-limits.spec.tstests/e2e/features/model-limits/pages/model-limits.page.tstests/e2e/features/observability/observability.spec.tstests/e2e/features/observability/pages/observability.page.tstests/e2e/features/placeholders/placeholders.spec.tstests/e2e/features/plugins/pages/plugins.page.tstests/e2e/features/plugins/plugins.spec.tstests/e2e/features/providers/pages/providers.page.tstests/e2e/features/providers/providers.spec.tstests/e2e/features/routing-rules/pages/routing-rules.page.tstests/e2e/features/routing-rules/routing-rules.spec.tstests/e2e/features/virtual-keys/pages/virtual-keys.page.tstests/e2e/features/virtual-keys/virtual-keys.spec.tstests/e2e/playwright.config.tsui/app/workspace/config/views/clientSettingsView.tsxui/app/workspace/config/views/mcpView.tsxui/app/workspace/config/views/pricingConfigView.tsxui/app/workspace/config/views/securityView.tsxui/app/workspace/governance/views/customerDialog.tsxui/app/workspace/governance/views/customerTable.tsxui/app/workspace/governance/views/teamDialog.tsxui/app/workspace/governance/views/teamsTable.tsxui/app/workspace/mcp-registry/views/mcpClientsTable.tsxui/app/workspace/mcp-registry/views/mcpServersEmptyState.tsxui/app/workspace/model-limits/views/modelLimitSheet.tsxui/app/workspace/model-limits/views/modelLimitsEmptyState.tsxui/app/workspace/model-limits/views/modelLimitsTable.tsxui/app/workspace/plugins/fragments/pluginFormFragments.tsxui/app/workspace/plugins/page.tsxui/app/workspace/plugins/views/pluginsEmptyState.tsxui/app/workspace/providers/fragments/debuggingFormFragment.tsxui/app/workspace/providers/views/addProviderDropdown.tsxui/app/workspace/providers/views/modelProviderKeysTableView.tsxui/app/workspace/providers/views/providersEmptyState.tsxui/app/workspace/routing-rules/views/routingRulesEmptyState.tsxui/app/workspace/routing-rules/views/routingRulesTable.tsxui/app/workspace/routing-rules/views/routingRulesView.tsxui/app/workspace/virtual-keys/views/virtualKeysEmptyState.tsxui/app/workspace/virtual-keys/views/virtualKeysTable.tsxui/components/ui/numberAndSelect.tsx
🚧 Files skipped from review as they are similar to previous changes (35)
- tests/e2e/features/virtual-keys/pages/virtual-keys.page.ts
- tests/e2e/core/utils/selectors.ts
- ui/app/workspace/routing-rules/views/routingRulesView.tsx
- ui/app/workspace/virtual-keys/views/virtualKeysEmptyState.tsx
- tests/e2e/features/mcp-registry/mcp-registry.spec.ts
- tests/e2e/playwright.config.ts
- ui/app/workspace/model-limits/views/modelLimitSheet.tsx
- ui/app/workspace/model-limits/views/modelLimitsEmptyState.tsx
- tests/e2e/features/mcp-tool-groups/mcp-tool-groups.spec.ts
- ui/app/workspace/governance/views/customerTable.tsx
- tests/e2e/features/config/config.spec.ts
- ui/app/workspace/virtual-keys/views/virtualKeysTable.tsx
- ui/app/workspace/config/views/securityView.tsx
- ui/app/workspace/providers/fragments/debuggingFormFragment.tsx
- tests/e2e/features/model-limits/pages/model-limits.page.ts
- ui/app/workspace/config/views/mcpView.tsx
- .github/workflows/scripts/test-e2e-ui.sh
- ui/app/workspace/governance/views/teamDialog.tsx
- ui/app/workspace/config/views/pricingConfigView.tsx
- tests/e2e/features/observability/observability.spec.ts
- tests/e2e/features/mcp-tool-groups/pages/mcp-tool-groups.page.ts
- ui/app/workspace/providers/views/providersEmptyState.tsx
- ui/app/workspace/model-limits/views/modelLimitsTable.tsx
- tests/e2e/features/mcp-settings/mcp-settings.spec.ts
- tests/e2e/features/placeholders/placeholders.spec.ts
- ui/app/workspace/providers/views/modelProviderKeysTableView.tsx
- tests/e2e/features/model-limits/model-limits.spec.ts
- ui/app/workspace/plugins/page.tsx
- ui/app/workspace/mcp-registry/views/mcpClientsTable.tsx
- ui/app/workspace/config/views/clientSettingsView.tsx
- tests/e2e/features/plugins/pages/plugins.page.ts
- ui/app/workspace/governance/views/teamsTable.tsx
- tests/e2e/features/mcp-registry/pages/mcp-registry.page.ts
- ui/app/workspace/routing-rules/views/routingRulesTable.tsx
- tests/e2e/features/mcp-auth-config/pages/mcp-auth-config.page.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Expands end-to-end test coverage across multiple features including governance, model limits, MCP settings, observability connectors, and configuration options. Adds comprehensive test suites for team/customer management, pricing overrides, and documentation link validation.
Changes
Type of change
Affected areas
How to test
Run the expanded E2E test suite:
The tests cover both OSS and enterprise features, with appropriate skipping for unavailable functionality.
Screenshots/Recordings
N/A - Test infrastructure changes only
Breaking changes
Related issues
Improves test coverage and reliability for the web UI components.
Security considerations
Tests include validation of RBAC-protected features and proper handling of sensitive data like API keys in test scenarios.
Checklist
docs/contributing/README.mdand followed the guidelines