Skip to content

Migrate the testing environment to Vitest: rename equalMarkup to toEqualMarkup and remove the automated test runner#1413

Merged
pomek merged 15 commits into
masterfrom
remove-karma-test-runner
Jul 7, 2026
Merged

Migrate the testing environment to Vitest: rename equalMarkup to toEqualMarkup and remove the automated test runner#1413
pomek merged 15 commits into
masterfrom
remove-karma-test-runner

Conversation

@pomek

@pomek pomek commented Jul 6, 2026

Copy link
Copy Markdown
Member

🚀 Summary

Migrates the testing environment to Vitest and removes the automated test runner.

equalMarkuptoEqualMarkup(): the Chai assertion is renamed to a Vitest matcher (matching the Vitest naming style) and the unused attribute Chai assertion is removed. The matcher is registered in a Vitest setup file via expect.extend( { toEqualMarkup } ), importing it from the main entry point of @ckeditor/ckeditor5-dev-tests, which is now browser-safe (it exposes only the matcher). The package ships the vitest type augmentation (lib/vitest/matchers.d.ts), so consumers get the typings out of the box.

Automated test runner removal: the ckeditor5-dev-tests-run-automated binary and the whole Karma testing environment are removed — automated tests are executed directly with Vitest using the packages' own test scripts. This covers bin/testautomated.js and the runAutomatedTests() task, the whole lib/utils/automated-tests/ directory (Karma and webpack configuration, reporters, leak detectors), the automated CLI options in parseArguments() (including --production — the strict checks live in each package's Vitest setup and apply in every run, so local and CI executions follow the same rules), desktop notifications (node-notifier), the IntelliJ Karma integration, 26 dependencies of ckeditor5-dev-tests, and loaders.getCoverageLoader() with its babel-loader dependency in ckeditor5-dev-utils.

The package now provides the manual test server (ckeditor5-dev-tests-run-manual, untouched) and the toEqualMarkup() Vitest matcher.


📌 Related issues


💡 Additional information

  • Replaces Rename the equalMarkup Chai assertion to the toEqualMarkup Vitest matcher #1412 — both changes are combined here, as the matcher migration and the runner removal are tightly coupled (the Karma entry-file generator was the only in-repo registrar of the old assertions).
  • The attribute Chai assertion had zero call sites in ckeditor5 and ckeditor5-commercial, hence it is removed instead of migrated (a deviation from the issue's DoD).
  • Consumer adoption (setup files, ~1800 call-site renames, pnpm-based test runs): ckeditor/ckeditor5-commercial#10862 — after it, neither repository calls the automated binary.
  • Because ckeditor5-dev-tests was the only workspace package with a postinstall script, the root scripts/postinstall.js now uses pnpm run -r --if-present postinstall.
  • Changelog entries (major breaking changes) are included.

@pomek pomek force-pushed the remove-karma-test-runner branch from 9b811db to e94f1c7 Compare July 6, 2026 09:54
@pomek pomek force-pushed the remove-karma-test-runner branch from e94f1c7 to 02bbee8 Compare July 6, 2026 10:01
@pomek pomek force-pushed the remove-karma-test-runner branch from 02bbee8 to 5ecf41a Compare July 6, 2026 10:08
@pomek pomek force-pushed the remove-karma-test-runner branch from 5ecf41a to f736e52 Compare July 6, 2026 10:19
@pomek pomek force-pushed the remove-karma-test-runner branch from f736e52 to e6bb75c Compare July 6, 2026 11:03
Comment thread packages/ckeditor5-dev-tests/lib/tasks/runautomatedtests.js
@pomek pomek changed the title Remove the Karma test runner Remove the automated test runner (Karma environment and the run-automated binary) Jul 6, 2026
Comment thread packages/ckeditor5-dev-tests/lib/utils/parsearguments.js
@pomek pomek changed the title Remove the automated test runner (Karma environment and the run-automated binary) Migrate the testing environment to Vitest: rename equalMarkup to toEqualMarkup and remove the automated test runner Jul 6, 2026
@pomek pomek changed the base branch from ci/4595-to-equal-markup-vitest-matcher to master July 6, 2026 13:00
@przemyslaw-zan przemyslaw-zan self-requested a review July 6, 2026 14:14

@psmyrek psmyrek left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two changelog entries describe more or less the same change:

  • .changelog/20260706081519_ci_4595_to_equal_markup_vitest_matcher.md
  • .changelog/20260706081521_ci_4595_to_equal_markup_vitest_matcher.md

Would it be possible to merge these messages together?

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit f01521b. Configure here.

Comment thread packages/ckeditor5-dev-tests/lib/utils/parsearguments.js
@pomek pomek merged commit a15674c into master Jul 7, 2026
8 checks passed
@pomek pomek deleted the remove-karma-test-runner branch July 7, 2026 08:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants