unit test suite and CI#66
Merged
Merged
Conversation
jan-petr
self-requested a review
July 15, 2026 15:01
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the first CI pipeline, a comprehensive unit test suite, and an example-based integration test runner to the pyaslreport package. Before this, the package had no CI and only a single pre-existing test file covering get_bids_metadata orchestration. Without automated coverage, regressions in validation, parameter extraction, or report generation could go undetected.
What's included
Unit tests covering the internal processing logic:
Field renaming and unit conversion
All six validator classes (Number, String, Boolean, NumberArray, NumberOrNumberArray, Consistency) at their schema-configured boundaries, plus a schema-loading smoke test
M0 contradiction paths, TSV validation, and background-suppression warning branches
File grouping for both BIDS (exact-suffix) and DICOM (substring) modes, including TSV header enforcement
Integration tests covering the tool end-to-end:
An example-based runner: each example is a folder of real BIDS inputs plus a saved expected_output.json. Examples are discovered automatically, so contributors add test cases by dropping in a folder and running one command, no code changes required
Discovery is recursive and subject-agnostic (anchors on perf/ and BIDS filename suffixes), supporting multiple subjects, sessions, and runs in a single example
Golden/characterization comparison of the full 21-key output, including generated report text. pytest -m integration --update-expected regenerates the saved outputs when a behavior change is intended
M0/TSV pairing follows the agreed rules (1:1 by prefix; reuse the shared M0 when there are more ASLs; use the latest M0 when there are more M0s), with a PairingFallbackWarning surfaced whenever a fallback is applied
Harness self-tests that verify discovery, file ordering, pairing, and the golden update cycle, so the machinery is validated on every push even before example data is committed
CI via GitHub Actions:
Runs the unit suite on Python 3.11 and 3.12
Runs the integration suite in a separate job (-m integration), partitioning the suite with the unit job
Black and isort formatting gates (pinned in the test extra)