Epic: migrate codeanalyzer-python to canonical schema v2 + four analysis levels
Migrate the analyzer from the v1 payload (PyApplication + a separate program_graphs section) to the CLDK canonical schema v2 — one additive Code Property Graph tree — exposed as four gated analysis levels (-a 1|2|3|4), in both the analysis.json and Neo4j projections. This is a breaking major release (the codeanalyzer-backend "existing analyzer → schema v2" path). It supersedes the L3 approach in #67: dataflow moves inline into the tree, and the interprocedural SDG moves from -a 3 to -a 4.
Target shape
application → symbol_table{module} → types/functions → callables{ body{}, cfg[], cdg[], ddg[], summary[] }
call_graph[], param_in[], param_out[] — cross-function edges, at application scope
can://python/<app>/<file>/<type>/<callable-sig> durable ids (≥ callable); <callable-id>@<line>:<col> / @tag ordinal ids (< callable).
source stored once per module + byte-offset spans (every node's text slices off it); the per-callable code field is dropped.
- Payload envelope:
schema_version: "2.0.0", language, max_level, k_limit.
Levels
| Level |
Grows |
Adds edges |
| 1 |
tree to callable + call nodes in body (callee:null) |
— |
| 2 |
callee: null→id backfill |
call_graph |
| 3 |
rest of body + @entry/@exit |
cfg, cdg, ddg (syntactic, prov:["ssa"]) |
| 4 |
synthetic param vertices |
param_in, param_out, summary, + ddg (semantic, prov:["points-to"]) |
--graphs sdg requires -a 4 (requesting it at -a 3 is a flag error).
Key decisions
- Reuse the existing dataflow kernels (
cfg/dominance/defuse/sdg/summaries/alias); a new identity-mapping layer translates internal (signature, node_id) keys → can:// ids at emit time, which keeps the JSON and Neo4j projections in lockstep.
- Scalpel (
python-scalpel) is the primary L4 points-to oracle behind the frozen may_alias interface, with TypeBasedAliasOracle as an automatic fallback.
- Analyzer-only scope. The python-sdk Pydantic revision is a separate follow-up (via
cldk-sdk-frontend). Taint stays deferred to the SDK; the backward slicer stays internal as the L3/L4 validation gate (no product "queryable slicer" framing, no taint_flows section).
- Neo4j is a re-identify + one
prov property + source/span parity + MAJOR bump (schema.neo4j.json → 2.0.0); the near-identity projection structure is unchanged. Always full-depth.
Invariants / gates
- Additive superset:
analysis.json(-a 1) ⊆ … ⊆ analysis.json(-a 4), modulo the callee: null→id refinement and the L3→L4 DDG ssa→points-to widening.
- Two-projection agreement: JSON node/edge ids == Neo4j merge-key values.
- Per-level gates (validated per stage): L1 canonical-schema conformance; L2 no dangling call-graph endpoints; L3 intraprocedural backward slice == hand-computed set; L4 no dangling SDG endpoints + PARAM arity + a
summary for a known transitive flow.
Stages
Each stage is an independently shippable PR-unit with its own gate. Child issues:
Epic: migrate
codeanalyzer-pythonto canonical schema v2 + four analysis levelsMigrate the analyzer from the v1 payload (
PyApplication+ a separateprogram_graphssection) to the CLDK canonical schema v2 — one additive Code Property Graph tree — exposed as four gated analysis levels (-a 1|2|3|4), in both theanalysis.jsonand Neo4j projections. This is a breaking major release (thecodeanalyzer-backend"existing analyzer → schema v2" path). It supersedes the L3 approach in #67: dataflow moves inline into the tree, and the interprocedural SDG moves from-a 3to-a 4.Target shape
can://python/<app>/<file>/<type>/<callable-sig>durable ids (≥ callable);<callable-id>@<line>:<col>/@tagordinal ids (< callable).sourcestored once per module + byte-offset spans (every node's text slices off it); the per-callablecodefield is dropped.schema_version: "2.0.0",language,max_level,k_limit.Levels
callnodes inbody(callee:null)callee: null→idbackfillcall_graphbody+@entry/@exitcfg,cdg,ddg(syntactic,prov:["ssa"])param_in,param_out,summary, +ddg(semantic,prov:["points-to"])--graphs sdgrequires-a 4(requesting it at-a 3is a flag error).Key decisions
cfg/dominance/defuse/sdg/summaries/alias); a new identity-mapping layer translates internal(signature, node_id)keys →can://ids at emit time, which keeps the JSON and Neo4j projections in lockstep.python-scalpel) is the primary L4 points-to oracle behind the frozenmay_aliasinterface, withTypeBasedAliasOracleas an automatic fallback.cldk-sdk-frontend). Taint stays deferred to the SDK; the backward slicer stays internal as the L3/L4 validation gate (no product "queryable slicer" framing, notaint_flowssection).provproperty + source/span parity + MAJOR bump (schema.neo4j.json→2.0.0); the near-identity projection structure is unchanged. Always full-depth.Invariants / gates
analysis.json(-a 1) ⊆ … ⊆ analysis.json(-a 4), modulo thecallee: null→idrefinement and the L3→L4 DDGssa→points-towidening.summaryfor a known transitive flow.Stages
Each stage is an independently shippable PR-unit with its own gate. Child issues: