Playwright testing Enso 101 #13865

otulach · 2025-08-27T15:51:56Z

Pull Request Description

This PR adds additional Playwright tests.
The main objective is to cover the functionality of the entire Enso Analytics 101 walkthrough:
https://community.ensoanalytics.com/c/enso-analytics-101/sections/637012/lessons/2410664

Important Notes

Exercise 1

Provides full coverage of the first exercise
Divided into separate objectives ( as done in the tutorial )

enso-bot · 2025-08-27T21:07:11Z

Ondrej Tulach reports a new STANDUP for today (2025-08-27):

Progress: Starting with the first exercise and building fundamental infrastructure. It should be finished by 2025-08-29.

enso-bot · 2025-08-29T20:41:12Z

Ondrej Tulach reports a new STANDUP for today (2025-08-29):

Progress: Fixing CI issues. Overall restructuring of Playwright test. It should be finished by 2025-08-29.

enso-bot · 2025-09-01T15:43:17Z

Ondrej Tulach reports a new STANDUP for today (2025-09-01):

Progress: Finished first exercise. Diversified into objectives and used functions from previous PR. It should be finished by 2025-09-12.

vitvakatu

It’s nice to have more tests! I would like to see more unification in testing strategy with existing tests in project-view/widgets.spec.ts, but it can become complicated. If you face difficulties, don’t be afraid to raise questions — a not ideal test is better than no tests at all.

app/ide-desktop/client/tests/gettingStarted.spec.ts

vitvakatu · 2025-09-01T20:43:49Z

app/ide-desktop/client/tests/electronTest.ts

+export async function createNewComponent(page: Page) {
+  const moreButton = page.getByTestId('more-button').getByRole('button', { name: 'More' }).last()
+  await expect(moreButton).toBeVisible()
+  await moreButton.click()
+
+  await page.keyboard.press('Enter')
+}


I’m a bit confused here. Why do we need to click on the more-button, but then just use Enter to create a node? Also the last() selector seems a bit fragile. I suggest using the same workflow we do in other tests:

Find the node you want to use as ‘parent’ using locate.graphNodeByBinding or some other method

Click on its grab-handle: .locator('.grab-handle').click() to select it

Press Enter to create a new node.

I don’t like choosing nodes by their binding, because it is not obviously visible from graph view at the moment, and it can easily change in the future, but I also don’t like the last() approach. Maybe .last() is enough though!

Yeah the .last() selector is a bit sketchy.
I was using a bit different method for some other components:

Found the component trough either .locator() or .getBy or some sort

Right click the defining text of that component.

Press Enter

Do you think I could unify it to this method?
(Thanks for the review, I will get back to it on friday, once I get back home)

I’m still not sure we need to click on the ‘More’ button. Should clicking on a node icon be enough? I even found the graphNodeIcon locator in locate.ts.

app/ide-desktop/client/tests/gettingStarted.spec.ts

vitvakatu · 2025-09-01T20:54:02Z

app/ide-desktop/client/tests/gettingStarted.spec.ts

+    // Set the actual filtered number value
+    const numberBox = page.locator('input.WidgetNumber')
+    await expect(numberBox).toBeVisible()
+    await numberBox.fill('3')


I think we had bugs in the past when using fill instead of typing manually. Here is the excerpt from widgets.spec.ts where we set value of a text input. I suggest using the same approach when editing text fields in these tests as well:

const topLevelArgs = node.locator('.WidgetTopLevelArgument') const pathArg = topLevelArgs.filter({ has: page.getByText('path') }) const pathDropdown = new DropDownLocator(pathArg) const pathArgInput = pathArg.getByTestId('widget-text-content') // Editing text input shows and filters drop down await pathArgInput.click() // Using `type` instead of `inputText` here to catch keydown bugs like #13505. await page.keyboard.type('File 1')

I tried using the keyboard.type method, but kept getting unreasonable issues. So I kept the fill, but unified all the "typing in" into a single function, targeting particular boxes around the widget-text-content as you suggest.

somebody1234

CR ❔

app/ide-desktop/client/tests/gettingStarted.spec.ts

…ByText.

enso-bot · 2025-09-07T11:49:48Z

Ondrej Tulach reports a new STANDUP for today (2025-09-07):

Progress: Added new functions and made the test more uderstandable by utilising getByText method. It should be finished by 2025-09-07.

vitvakatu

I think it is in a good enough state to be merged.

vitvakatu · 2025-09-08T09:46:22Z

app/ide-desktop/client/tests/electronTest.ts

+export async function createNewComponent(page: Page) {
+  const moreButton = page.getByTestId('more-button').getByRole('button', { name: 'More' }).last()
+  await expect(moreButton).toBeVisible()
+  await moreButton.click()
+
+  await page.keyboard.press('Enter')
+}


I’m still not sure we need to click on the ‘More’ button. Should clicking on a node icon be enough? I even found the graphNodeIcon locator in locate.ts.

app/ide-desktop/client/tests/gettingStarted.spec.ts

vitvakatu · 2025-09-08T09:51:02Z

app/ide-desktop/client/tests/electronTest.ts

+/**
+ * Creating new component from the name of its parent component
+ */
+export async function createComponentText(page: Page, parentComponent: string) {


I would call it openComponentBrowser—we are not creating a component; we are just preparing to do so!

Absolutely! Everything done.

createNewComponent is still using the 'More' button method, but I am not using this function anymore. Replaced it completely by the newly named openComponentBrowser, which targets the parent components directly. However, there is still the need to keep the original one, due to previous tests.

app/ide-desktop/client/tests/gettingStarted.spec.ts

somebody1234 · 2025-09-11T09:30:00Z

app/ide-desktop/client/tests/electronTest.ts

+  await expect(
+    page
+      .getByRole('group', { name: TEXT.licenseAgreementCheckbox })
+      .getByText(TEXT.licenseAgreementCheckbox),
+  ).toBeVisible({ timeout: 60000 })


i mentioned this on the first PR but - are you sure we need to wait for toBeVisible?

playwright waits for visibility for clicks:
https://playwright.dev/docs/input#mouse-click

so this should not be necessary on paper. in general playwright prefers doing things as if a human/user were using the app (e.g. roles), rather than via code/programmatic access (e.g. CSS selectors)

in this PR it seems there is quite a bit of of boilerplate caused by this, that may be unnecessary due to how playwright works (and if it is necessary, consider maybe writing a helper function do to the visibility check and the click)

Do you suggest removing waiting toBeVisible in general?
I found few cases where Playwright, even though it should behave like a human, performs an action too fast, which then causes the whole test to crash. So the easiest solution was to add a small wait.

nah, toBeVisible would be preferable i think (?) - i think it would be good to verify which specific cases - the most ideal result would be we get a somewhat reliable repro and figure out what's actually causing it

as for a wait: that's unreliable since it will cause flakiness if the runner is too slow

somebody1234 · 2025-09-11T09:32:45Z

app/ide-desktop/client/tests/electronTest.ts

+}
+
+/**
+ * Creating new component from the name of its parent component


incorrect docs

somebody1234 · 2025-09-11T09:33:57Z

app/ide-desktop/client/tests/electronTest.ts

+}
+
+/**
+ * Creating new component from the name of its parent component


nit: in general, docs for functions should be described like:
"the goal of this function is to ___"

so rather than "creating" or "creates", we should prefer "create"

somebody1234 · 2025-09-11T09:38:06Z

app/ide-desktop/client/tests/electronTest.ts

+/**
+ * Creating new component from the name of its parent component
+ */
+export async function fillText(page: Page, containerName: string, value: string) {


should this be fillWidgetText based on its body?

somebody1234

CR ✅, nits above

Exercise 1, up until objective 2

a3be74c

otulach requested review from AdRiley, Frizi, PabloBuchu, farmaazon, indiv0, kazcw, somebody1234 and vitvakatu as code owners August 27, 2025 15:51

This comment was marked as off-topic.

Sign in to view

otulach changed the title ~~Dozen new Playwright tests~~ Playwright testing Enso 101 Aug 27, 2025

Targeting log in issue in CI. More to Excercise 1.

5454a2d

otulach added the CI: No changelog needed Do not require a changelog entry for this PR. label Aug 29, 2025

otulach added 8 commits August 30, 2025 16:26

Few more to exercise 1.

02712e1

All 4 objectives. Functionality added

5ec9f4d

Exercise 1 passing fully, divided into individual sections.

a29f484

Merging with develop to get #13787.

5460196

Deduplicating and changing up objective 4.

77be7e8

Merge branch 'develop' into playwright-getting-started

63bef7e

Avoiding warning calls.

1a7718d

Formating with prettier.

6c45fce

vitvakatu reviewed Sep 1, 2025

View reviewed changes

somebody1234 reviewed Sep 2, 2025

View reviewed changes

app/ide-desktop/client/tests/gettingStarted.spec.ts Outdated Show resolved Hide resolved

app/ide-desktop/client/tests/gettingStarted.spec.ts Outdated Show resolved Hide resolved

app/ide-desktop/client/tests/gettingStarted.spec.ts Outdated Show resolved Hide resolved

somebody1234 reviewed Sep 2, 2025

View reviewed changes

app/ide-desktop/client/tests/gettingStarted.spec.ts Outdated Show resolved Hide resolved

Unified targeting and filling textboxes. ApplyPadding replaced by get…

bf9303b

…ByText.

otulach added the CI: Ready to merge This PR is eligible for automatic merge label Sep 7, 2025

otulach requested a review from somebody1234 September 7, 2025 11:45

Removing commented out code.

c6c7ad5

vitvakatu approved these changes Sep 8, 2025

View reviewed changes

otulach added 2 commits September 8, 2025 12:17

Adding a small wait before clicking 'NotEqual' button.

a87146b

Removing unneccessary vizualisations. Renaming.

8bcdb20

somebody1234 reviewed Sep 11, 2025

View reviewed changes

somebody1234 approved these changes Sep 11, 2025

View reviewed changes

otulach added 7 commits September 17, 2025 21:20

Proper documentation to helper funtions.

942dd19

Renaming fillText function.

351a38c

Prettier.

cacfa7d

Wait for Samples being downloaded. Handle uncaught promises.

f62629b

Prettier.

2ff9601

Removing visibility check causing issues in CI.

9eb6b38

ToBeVisible removed from electron.

d90f480

Playwright testing Enso 101 #13865

Are you sure you want to change the base?

Playwright testing Enso 101 #13865

Uh oh!

Conversation

otulach commented Aug 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Description

Important Notes

Exercise 1

Uh oh!

This comment was marked as off-topic.

enso-bot bot commented Aug 27, 2025

Uh oh!

enso-bot bot commented Aug 29, 2025

Uh oh!

enso-bot bot commented Sep 1, 2025

Uh oh!

vitvakatu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

otulach Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

somebody1234 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

enso-bot bot commented Sep 7, 2025

Uh oh!

vitvakatu left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

somebody1234 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

otulach commented Aug 27, 2025 •

edited

Loading

otulach Sep 2, 2025 •

edited

Loading