test(wtr): run in multiple browsers and with API_VERSION=66 #5533

wjhsf · 2025-10-07T18:54:06Z

Details

This PR adds a CI run of the tests with API_VERSION=66 and updates the config to run in chrome, firefox, and safari. It also (hopefully) connects to saucelabs when run in CI. If the LEGACY_BROWSERS var is set in CI, then it runs slightly old chrome and safari (not that legacy, imo 🤷).

Does this pull request introduce a breaking change?

😮‍💨 No, it does not introduce a breaking change.
💔 Yes, it does introduce a breaking change.

Does this pull request introduce an observable change?

🤞 No, it does not introduce an observable change.
🔬 Yes, it does include an observable change.

GUS work item

probably still wrongly, tho

I'm not convinced this is the right place to do it

packages/@lwc/integration-not-karma/package.json

FAIL: packages/@lwc/engine-dom/dist/index.js: 24.79KB > 24.79KB RUDE

installing in package break nucleus

wjhsf · 2025-10-09T16:05:20Z

/nucleus ingore --reason "PR isn't ready and I want to watch CI"

salesforce-nucleus · 2025-10-09T16:05:24Z

❌ Error Parsing Command

Unknown arguments: reason, ingore

/nucleus <command>

Commands:
  /nucleus deploy                           Perform deploy
  /nucleus help                             Display usage information
  /nucleus ignore                           Ignore test failures
  /nucleus ping                             Ping Nucleus
  /nucleus release                          Perform release
  /nucleus reset-auto-increment             Reset auto-increment value for
  [incrementValue]                          project
  /nucleus skip-dep-update                  Opt-out from the upstream dependency
                                            update
  /nucleus skip-deployment                  Mark deployment as skipped for the
                                            workflow associated with this PR
  /nucleus start                            Start the Nucleus workflow
                                            associated with this pull-request.
  /nucleus stop                             Stops the workflow associated with
                                            this PR.
  /nucleus test                             Build project and execute tests.
                                            This command may be executed to
                                            re-run the testing stage.
  /nucleus update-downstream-deps           Update downstream dependents
  /nucleus bump-version [version]           Bump version

Options:
  --help  Show help                                                    [boolean]

Examples:
  /nucleus release 1.2.0  Release version 1.2.0

Unknown arguments: reason, ingore

wjhsf · 2025-10-09T16:05:42Z

/nucleus ignore --reason "PR isn't ready and I want to watch CI"

* test(wtr): change from filename-based config to magic directive * test(wtr): actually use JSON * test(wtr): let spec file set config for whole dir * test(wtr): fix CTE tests

hopefully it doesn't break

not working as intended; may fix later

it got deleted oops

rax-it · 2025-10-16T14:29:54Z

packages/@lwc/integration-not-karma/configs/shared/browsers.js

+                browserName: 'safari',
+                browserVersion: 'latest',
+            }),
+            ...(options.LEGACY_BROWSERS


This is somewhat misleading, how much does things change in (2 versions prior to latest?) to call them legacy?

It's a silly name, but it's what we did with the Karma tests.

rax-it · 2025-10-16T14:32:12Z

packages/@lwc/integration-not-karma/test/template-expressions/smoke-test/index.spec.js

+import { API_VERSION } from '../../../helpers/options';

-it(`should support call expressions`, () => {
+const cteEnabled = API_VERSION >= 66;


thinking if this should be hoisted to global config or something so it's easier to update this in the future and we don't have to drill down all the way?

CTE was introduced in API version 66; this check will never need to be updated.

rax-it · 2025-10-16T14:34:11Z

packages/@lwc/integration-not-karma/configs/shared/base-config.js

        // Potential workaround: https://github.com/modernweb-dev/web/issues/2588
-        concurrency: 1,
-        browserLogs: false,
+        concurrency: options.CI ? 6 : 1,


curious about 6, does this follow any known limitation?
Also, any thoughts on decoupling concurrency with CI and making it an ENV variable?

I copied this from the docs. When run locally, tests cannot be run concurrently, due to the timing issue mentioned in the code comment. I don't know whether or not the limitation exists when run with SauceLabs, so I made this change to find out.

to see if it makes it less flaky

This reverts commit 41ce390.

rax-it · 2025-10-17T17:23:05Z

packages/@lwc/integration-not-karma/configs/shared/browsers.js

+                browserName: 'chrome',
+                browserVersion: 'latest',
+            }),
+            // sauceLabsLauncher({


SauceLabs seems to be flaky. So I'm starting with running just one browser, and I'll add more later.

wjhsf added 5 commits October 6, 2025 14:40

test(wtr): use vitest spy instead of jasmine

099ab86

test(wtr): add API version 66 coverage to CI

a249b96

test(wtr): indent consistently

1e50ff1

probably still wrongly, tho

test(wtr): move base config into shared dir

715cac8

test(wtr): run tests in multiple browsers

22cfa8d

wjhsf requested a review from a team as a code owner October 7, 2025 18:54

test(wtr): use build script to install playwright

af60a96

I'm not convinced this is the right place to do it

wjhsf commented Oct 7, 2025

View reviewed changes

packages/@lwc/integration-not-karma/package.json Outdated Show resolved Hide resolved

wjhsf added 12 commits October 7, 2025 15:06

chore: bundlesize

41676b5

FAIL: packages/@lwc/engine-dom/dist/index.js: 24.79KB > 24.79KB RUDE

test(wtr): add type for param

1b44785

test(wtr): use sauce helper in CI

2072694

test(wtr): add install to playwright setup

423c051

test(wtr): enable saucelabs in all runs

7841184

test(wtr): kerjigger playwright again

52915ff

test(wtr): rekerjigger playwright again

f17448b

test(wtr): make fake SSR file more clear

6412646

test(wtr): install playwright deps in WTR CI

e4092b4

installing in package break nucleus

test(wtr): and much rekerjiggering there was

5357cd7

test(wtr): isolate server-side LWC

3fade97

test(wtr): hack for nucleus

9e4ac3e

wjhsf and others added 3 commits October 14, 2025 15:32

test(wtr): re-enable CTE tests @W-19098211 (#5535)

465dd72

* test(wtr): change from filename-based config to magic directive * test(wtr): actually use JSON * test(wtr): let spec file set config for whole dir * test(wtr): fix CTE tests

chore(ci): split hydration tests into engine-server and ssr-v2

5c81592

chore(ci): split integration tests into two groups

e928017

wjhsf force-pushed the wjh/wtr-keep-on-enving branch from 5bf64d8 to e928017 Compare October 15, 2025 19:37

wjhsf and others added 3 commits October 15, 2025 20:05

Merge branch 'master' into wjh/wtr-keep-on-enving

1deb775

chore(ci): build first and cache result so parallel jobs are faster

f6ea8de

test(wtr): check correct CI env var

e83ec4e

wjhsf added 5 commits October 15, 2025 17:07

test(wtr): use concurrency in CI

02c8206

hopefully it doesn't break

test(wtr): always run browsers concurrently

8564bec

chore(ci): give up on caching (for now?)

e1288ce

not working as intended; may fix later

test(wtr): try difference sauce config

b1fef23

chore(ci): don't needs: install-and-build

2aa45e1

it got deleted oops

rax-it reviewed Oct 16, 2025

View reviewed changes

wjhsf and others added 8 commits October 16, 2025 12:58

chore(ci): use distinct tunnel IDs for SauceLabs

9d59c9a

Merge branch 'master' into wjh/wtr-keep-on-enving

3b7657d

test(wtr): revert concurrency in CI

45d38da

test(wtr): disable tests that sauce doesn't like

199412f

test(wtr): only run chrome in saucelabs

4672320

to see if it makes it less flaky

chore(ci): use env var for runs-on for easier updates

41ce390

chore(ci): use matrix values to ensure unique tunnel IDs

3fc2bc2

Revert "chore(ci): use env var for runs-on for easier updates"

570da42

This reverts commit 41ce390.

rax-it approved these changes Oct 17, 2025

View reviewed changes

rax-it reviewed Oct 17, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

test(wtr): run in multiple browsers and with API_VERSION=66 #5533

test(wtr): run in multiple browsers and with API_VERSION=66 #5533

Uh oh!

wjhsf commented Oct 7, 2025

Uh oh!

Uh oh!

wjhsf commented Oct 9, 2025

Uh oh!

salesforce-nucleus bot commented Oct 9, 2025

Uh oh!

wjhsf commented Oct 9, 2025

Uh oh!

rax-it Oct 16, 2025

Uh oh!

wjhsf Oct 16, 2025

Uh oh!

rax-it Oct 16, 2025

Uh oh!

wjhsf Oct 16, 2025

Uh oh!

rax-it Oct 16, 2025

Uh oh!

wjhsf Oct 16, 2025

Uh oh!

rax-it Oct 17, 2025

Uh oh!

wjhsf Oct 17, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

test(wtr): run in multiple browsers and with API_VERSION=66 #5533

Are you sure you want to change the base?

test(wtr): run in multiple browsers and with API_VERSION=66 #5533

Uh oh!

Conversation

wjhsf commented Oct 7, 2025

Details

Does this pull request introduce a breaking change?

Does this pull request introduce an observable change?

GUS work item

Uh oh!

Uh oh!

wjhsf commented Oct 9, 2025

Uh oh!

salesforce-nucleus bot commented Oct 9, 2025

❌ Error Parsing Command

Uh oh!

wjhsf commented Oct 9, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants