🧪 Declarative fixture + snapshot extraction tests#83
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #83 +/- ##
==========================================
+ Coverage 90.82% 91.40% +0.57%
==========================================
Files 33 34 +1
Lines 2942 3012 +70
Branches 315 322 +7
==========================================
+ Hits 2672 2753 +81
+ Misses 168 160 -8
+ Partials 102 99 -3 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Add tests/data/extraction/ with a README describing the declarative extraction-test format: per-case YAML fixtures (lang/config/source) plus a normalized JSON snapshot of the extraction output (needs/need_refs/marked_rst/ warnings). Harness + fixtures + snapshots follow in subsequent commits.
Add a parametrized harness that loads tests/data/extraction/*.yaml cases
(lang/config/source), runs SourceAnalyse on a temp source file, normalizes the
output to {needs,need_refs,marked_rst,warnings} (1-indexed lines) and asserts a
single-file JSON snapshot per case (ExtractionSnapshotExtension).
Fixtures: default one-liners across all eight languages (cpp,c,python,csharp,
rust,yaml in oneline.yaml; go,jsonc in extra_languages.yaml).
…ixtures Add declarative fixtures + JSON snapshots for: escape-sequence resolution (escapes.yaml), custom one-line configs incl bracketed + space-separated (custom_config.yaml), need-id references with default/custom markers (need_refs.yaml), four parser warning kinds (warnings.yaml), and @rst/@endrst blocks (marked_rst.yaml). Add an 'extract' fixture toggle (subset of oneline/need_refs/rst, default all) so a focused case isn't polluted by the '@' one-line start also matching '@need-ids:'/'@rst'. 25 cases total; all snapshots captured.
f6e0be6 to
83d6cd1
Compare
ubmarco
left a comment
There was a problem hiding this comment.
Reviewed — test-only, no library changes, and the data-as-tests approach is a good fit for the extraction surface.
Went through:
- the harness (
tests/test_extraction_fixtures.py): parametrization overtests/data/extraction/*.yaml, theSourceAnalyserun on a temp source file, and the_normalizereduction to{needs, need_refs, marked_rst, warnings}with 1-indexed lines and stable sorting; - the single-file JSON snapshots via
ExtractionSnapshotExtensioninconftest.py; - all eight fixtures + their snapshots: the language matrix (cpp / c / python / csharp / rust / yaml / go / jsonc), escape-sequence resolution, the bracketed and space-separated custom one-line configs, need-id refs (default + custom marker), the four parser warning kinds, and the
@rst/@endrstblocks; - the
README.mdformat + how-to-update docs.
The per-case extract toggle is a nice touch — scoping a @need-ids: case to [need_refs] so the @ start isn't also picked up as a one-line need avoids exactly the kind of cross-talk that would otherwise make these fixtures flaky. The note that newline_in_field is only reachable at the parser level is helpful context too.
One optional, non-blocking thought: _build_oneline_style (and the need_id_markers lookup in the test body) silently ignore unknown keys, so a misspelled config key in a fixture — start_seq: for start_sequence:, say — would quietly no-op and the snapshot would capture default behavior instead. Since the fixtures are the source of truth here, it might be worth rejecting unknown config keys so a typo fails loudly.
Nothing blocking from me.
What
Adds a declarative fixture + snapshot test layer for C/C++/… marker
extraction. Each case is a YAML fixture (input:
lang+config+source)plus a captured JSON snapshot (expected: the normalized extraction output), so
the whole language matrix is covered as data rather than as bespoke test
functions.
Details
tests/test_extraction_fixtures.py— parametrizes overtests/data/extraction/*.yaml, runsSourceAnalyseon a temp source file,normalizes the result to
{needs, need_refs, marked_rst, warnings}(1-indexedlines), and asserts a single-file JSON snapshot per case (via a new
ExtractionSnapshotExtensioninconftest.py).tests/data/extraction/): default one-liners across all eightlanguages (cpp, c, python, csharp, rust, yaml, go, jsonc); escape-sequence
resolution; custom one-line configs (bracketed + space-separated); need-id
references (default + custom marker); four parser warning kinds; and
@rst/@endrstblocks.extracttoggle (subset ofoneline/need_refs/rst)keeps a case focused so, e.g., a
@need-ids:marker isn't also parsed as aone-line need.
tests/data/extraction/README.md.Scope
Test-only; no library changes. Covers the marker-extraction surface; existing
parser / pipeline / discovery tests are untouched. (
newline_in_fieldisreachable only at the parser level — noted in
warnings.yaml.)Running