Skip to content

Build/Test Tools: Trim the PHPUnit matrix to boundary PHP versions (full matrix runs weekly) - #12719

Closed
lancewillett wants to merge 2 commits into
WordPress:trunkfrom
lancewillett:trim-active-branch-phpunit-matrix
Closed

Build/Test Tools: Trim the PHPUnit matrix to boundary PHP versions (full matrix runs weekly)#12719
lancewillett wants to merge 2 commits into
WordPress:trunkfrom
lancewillett:trim-active-branch-phpunit-matrix

Conversation

@lancewillett

@lancewillett lancewillett commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

What

Trims the PHPUnit test matrix to boundary PHP versions on trunk and pull requests: the highest and lowest supported version of each major. 7.4, 8.0, and 8.5 replace the full seven-minor list, in test-with-mysql, test-with-mariadb, and test-innovation-releases.

Full coverage is preserved by the existing weekly scheduled run: on the Sunday cron, every supported PHP version is tested; push and pull request events test the boundary versions only.

Why

This mirrors the trim #64083 applied to older branches, whose title named "(and possibly trunk)". Version-specific failures cluster at the boundaries of each major, so the intermediate minors add job count without adding coverage. The weekly full run keeps the intermediate versions covered without multiplying job count on every change.

Impact

Base PHPUnit combinations drop from ~168 to ~72 on push and pull requests, roughly a 55% cut, before the unchanged include: jobs. The weekly run is unchanged.

How the weekly full matrix works

The php matrix is conditional on the event:

php: ${{ github.event_name == 'schedule'
        && fromJSON('["7.4","8.0","8.1","8.2","8.3","8.4","8.5"]')
        || fromJSON('["7.4","8.0","8.5"]') }}

Scheduled runs fire from the default branch, so the weekly full sweep covers trunk.

Scope

PHP axis only. Database versions, multisite, memcached, and the single-representative include: jobs are untouched. A database-axis trim on the same principle is a possible follow-up, filed separately.

Trac ticket: Core-65736

Use of AI Tools

AI assistance: Yes
Tool(s): Claude Code
Model(s): Opus 4.8
Used for: Identifying the boundary-trim approach from the existing older-branch precedent, the weekly-full conditional, and drafting the change and ticket text. Verified manually.

Reduce the PHPUnit matrix on the active branch to the highest and lowest supported version of each major PHP branch, matching the boundary testing already used on older branches. Version-specific failures cluster at the boundaries, so the intermediate minor versions add job count without adding coverage.

This is especially useful when preparing backports across multiple branches, where the full matrix multiplies queued jobs and floods the shared runner pool during a release.

Props lance.willett@automattic.com.

See #65736.
Copilot AI review requested due to automatic review settings July 27, 2026 15:48
@github-actions

github-actions Bot commented Jul 27, 2026

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Core Committers: Use this line as a base for the props when committing in SVN:

Props lancewillett, johnbillion.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reduces GitHub Actions PHPUnit job counts by trimming the PHP version matrix to boundary versions (lowest/highest supported per major) on the active branch, aiming to preserve meaningful coverage while easing runner load during releases.

Changes:

  • Reduce the php matrix in test-with-mysql from seven minors to 7.4, 8.0, and 8.5.
  • Reduce the php matrix in test-with-mariadb from seven minors to 7.4, 8.0, and 8.5.
  • Reduce the php matrix in test-innovation-releases from seven minors to 7.4, 8.0, and 8.5.
Comments suppressed due to low confidence (1)

.github/workflows/phpunit-tests.yml:157

  • The new comment is slightly misleading in this job: while the base php: axis is trimmed to boundary versions, the include: section below still runs additional jobs on intermediate PHP minors (e.g. 8.3). Consider rewording so it’s clear the boundary trim applies to the base matrix only.
        # Test highest and lowest supported version of each major.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/phpunit-tests.yml Outdated
@lancewillett lancewillett changed the title Build/Test Tools: Trim the unit test matrix to boundary PHP versions Build/test tools: trim the unit test matrix to boundary PHP versions Jul 27, 2026
Push and pull request events test the boundary PHP versions only. The weekly scheduled run tests every supported version, so full coverage is preserved without multiplying job count on every change.

Props lance.willett@automattic.com.

See #65736.

@johnbillion johnbillion left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an element of risk with this change, but:

  • That can be offset with a new daily or weekly workflow that exercises the full matrix
  • The benefit of faster overall feedback as a result of pushes and PRs is worth it, especially shortly prior to and during release when lots of pushes are being made

Copilot AI review requested due to automatic review settings July 27, 2026 16:20
@lancewillett lancewillett changed the title Build/test tools: trim the unit test matrix to boundary PHP versions Build/Test Tools: Trim the PHPUnit matrix to boundary PHP versions (full matrix runs weekly) Jul 27, 2026
@lancewillett

Copy link
Copy Markdown
Contributor Author

@johnbillion Agree on the tradeoff. If we could make all runs faster, maybe we can keep the full matrix.

Another idea is to allow a PR tag full-ci to kick off the full, ad-hoc.

Apologies, I made a mistake in the first commit. Can you re-review 2538d74 ? Adds the weekly back.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

.github/workflows/phpunit-tests.yml:78

  • workflow_dispatch runs will now use the trimmed boundary PHP list because the condition only treats schedule as the full-matrix case. If you want manual runs to retain the ability to exercise the full PHP matrix on demand (without waiting for the weekly cron), include workflow_dispatch in the full-matrix branch and update the comment accordingly.
        # The scheduled run tests every supported PHP version. Other events test the highest and lowest of each major.
        php: ${{ github.event_name == 'schedule' && fromJSON('["7.4","8.0","8.1","8.2","8.3","8.4","8.5"]') || fromJSON('["7.4","8.0","8.5"]') }}

.github/workflows/phpunit-tests.yml:158

  • workflow_dispatch runs will now use the trimmed boundary PHP list because the condition only treats schedule as the full-matrix case. If you want manual runs to retain the ability to exercise the full PHP matrix on demand (without waiting for the weekly cron), include workflow_dispatch in the full-matrix branch and update the comment accordingly.
        # The scheduled run tests every supported PHP version. Other events test the highest and lowest of each major.
        php: ${{ github.event_name == 'schedule' && fromJSON('["7.4","8.0","8.1","8.2","8.3","8.4","8.5"]') || fromJSON('["7.4","8.0","8.5"]') }}

.github/workflows/phpunit-tests.yml:213

  • workflow_dispatch runs will now use the trimmed boundary PHP list because the condition only treats schedule as the full-matrix case. If you want manual runs to retain the ability to exercise the full PHP matrix on demand (without waiting for the weekly cron), include workflow_dispatch in the full-matrix branch and update the comment accordingly.
        # The scheduled run tests every supported PHP version. Other events test the highest and lowest of each major.
        php: ${{ github.event_name == 'schedule' && fromJSON('["7.4","8.0","8.1","8.2","8.3","8.4","8.5"]') || fromJSON('["7.4","8.0","8.5"]') }}

pento pushed a commit that referenced this pull request Jul 27, 2026
Test only the highest and lowest supported PHP version of each major (7.4, 8.0, 8.5) on push and pull request events; the weekly scheduled run still exercises every supported version.

Version-specific failures cluster at the boundaries of each major, so the intermediate minors add job count without adding coverage. Cuts base PHPUnit combinations from roughly 168 to 72 per CI run for both time and cost savings, and a much better developer experience in core GitHub repos.

Developed in: #12719

Props johnbillion, adrianmoldovanwp
Fixes #65736, see #64083


git-svn-id: https://develop.svn.wordpress.org/trunk@62858 602fd350-edb4-49c9-b593-d223f7449a82
@github-actions

Copy link
Copy Markdown

A commit was made that fixes the Trac ticket referenced in the description of this pull request.

SVN changeset: 62858
GitHub commit: 5174167

This PR will be closed, but please confirm the accuracy of this and reopen if there is more work to be done.

@github-actions github-actions Bot closed this Jul 27, 2026
markjaquith pushed a commit to markjaquith/WordPress that referenced this pull request Jul 27, 2026
Test only the highest and lowest supported PHP version of each major (7.4, 8.0, 8.5) on push and pull request events; the weekly scheduled run still exercises every supported version.

Version-specific failures cluster at the boundaries of each major, so the intermediate minors add job count without adding coverage. Cuts base PHPUnit combinations from roughly 168 to 72 per CI run for both time and cost savings, and a much better developer experience in core GitHub repos.

Developed in: WordPress/wordpress-develop#12719

Props johnbillion, adrianmoldovanwp
Fixes #65736, see #64083

Built from https://develop.svn.wordpress.org/trunk@62858


git-svn-id: http://core.svn.wordpress.org/trunk@62138 1a063a9b-81f0-0310-95a4-ce76da25c4cd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants