diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md index 561b4efbf..1bccffccc 100644 --- a/DEVELOPER_GUIDE.md +++ b/DEVELOPER_GUIDE.md @@ -1810,7 +1810,7 @@ Process exit codes are coarse (`0` success including valid zero-row queries, `1` - **Ambiguous source extensions stay explicit** — `.m` and `.pl` are not assigned to Objective-C and Perl by default. `FileIndexer` checks an authoritative recognized shebang, then a 64 KiB bounded prefix for strong mutually exclusive Objective-C/MATLAB or Perl/Prolog markers, then at most 256 entries per ancestor directory for conservative project markers. Conflicting or weak evidence is indexed as `ambiguous_m` / `ambiguous_pl`; unresolved `.m` files run the bounded MATLAB and Objective-C symbol/reference paths after a shared position-preserving comment mask, while Prolog and `ambiguous_pl` advertise conservative reference/graph support and the ambiguous `.pl` bucket uses union symbol/reference rules without changing content-based classification (#4612, #4738, #4746). - **Dynamic reference-graph readiness follows extractor contracts** — when indexed Crystal, Groovy, Tcl, Prolog, or `ambiguous_pl` rows have a missing or stale symbol-extractor version stamp, status reports `dynamic_reference_graph_contract_stale` and keeps `reference_graph_complete` / `graph_data_current` false until a normal index refresh rewrites those rows (#4746). - **Hotspot marker fingerprints share one bounded tree traversal** — full/update CLI and MCP indexing compute C#, VB, F#, and MSBuild marker fingerprints together instead of walking the directory tree once per language. Each distinct marker glob retains the platform filesystem's matching behavior and is enumerated once per visited directory, while child directories are enumerated once; marker sets, budgets, truncation sentinels, and warning order remain isolated per language. The single-language API delegates to the same engine, preserving ignore rules, nested-repository/submodule boundaries, and MCP authorized-read failures. -- **Lock-file dependency graphs model package relationships** — `packages.lock.json`, `package-lock.json`, and `npm-shrinkwrap.json` keep package declarations as symbols, but emit `dependency` references only for explicit parent-package to child-package entries. The parent package is stored as the reference container, so `callers` can identify which package requires a child and `deps` does not infer lock-file-to-lock-file similarity merely because two files contain the same resolved package set (#4409). +- **Lock-file dependency graphs model package relationships** — `packages.lock.json`, `package-lock.json`, and `npm-shrinkwrap.json` keep package declarations as symbols, but emit `dependency` references only for explicit parent-package to child-package entries. NuGet lock symbols and references preserve the current file, target/RID, parent package, and exact JSON property span; candidate resolution stays file-local, while file-level `deps` suppresses cross-file package-name inference. Normal index updates invalidate the prior dependency-lock extraction and reference-identity contracts, so `callers` identifies the requiring package without connecting unrelated lock files or collapsing repeated declarations to the first matching line (#4409, #4845). - **Dependency-cycle audits separate analysis from display** — CLI `deps --cycles` and MCP `deps` with `cycles=true` analyze a deterministic, path-ordered edge set up to the independent `--graph-budget` / `graphBudget` before computing and stably ranking strongly connected components. `--limit` / `limit` only paginates that ranked SCC set, and opaque cursors are bound to the filters, graph budget, and indexed graph that produced them. Machine-readable responses expose `analysis_complete`, graph edge count/budget, stable ranking mode, authoritative total-cycle status, and continuation metadata; exhausting the graph budget is reported as an explicitly incomplete analysis rather than a complete cycle audit (#4731). - **No ORM** — Raw `Microsoft.Data.Sqlite` with parameterized queries. Keeps dependencies minimal and control explicit. - **Batch commits** — 500 records per transaction for write performance. Reduces fsync overhead. @@ -4999,7 +4999,7 @@ USER_GUIDEの[終了コード](USER_GUIDE.md#終了コード)セクションを - **曖昧な source extension は曖昧なまま明示** — `.m` と `.pl` を既定で Objective-C / Perl に割り当てません。`FileIndexer` は authoritative な認識済み shebang、64 KiB 上限 prefix 内の相互排他的で強い Objective-C/MATLAB または Perl/Prolog marker、各 ancestor directory 最大 256 entry の保守的な project marker の順に確認します。競合または弱い証拠は `ambiguous_m` / `ambiguous_pl` として index し、未確定の `.m` は位置を保つ共通コメントマスクの後で上限付きの MATLAB / Objective-C symbol・reference 経路を実行します。一方、Prolog と `ambiguous_pl` は保守的な reference / graph 対応を広告し、曖昧な `.pl` bucket は content-based classification を変えずに symbol / reference rule の和集合を使います(#4612、#4738、#4746)。 - **動的言語の reference-graph readiness は extractor contract に従う** — index 済みの Crystal、Groovy、Tcl、Prolog、`ambiguous_pl` row で symbol-extractor version stamp が欠落または古い場合、status は `dynamic_reference_graph_contract_stale` を報告し、通常の index refresh が対象 row を更新するまで `reference_graph_complete` / `graph_data_current` を false に保ちます(#4746)。 - **hotspot marker fingerprint は上限付きtree traversalを1回共有** — full/update CLIとMCP indexingは、directory treeを言語ごとに歩かず、C#、VB、F#、MSBuildのmarker fingerprintをまとめて計算します。各directoryでは固有marker globごとにplatform filesystemのmatching挙動を保って1回ずつ列挙し、child directoryも1回だけ列挙する一方、marker集合、budget、truncation sentinel、warning順は言語別に分離します。single-language APIも同じengineへ委譲し、ignore rule、nested repository/submodule境界、MCP authorized read failureを維持します。 -- **lock file の依存グラフは package 間の関係をモデル化** — `packages.lock.json`、`package-lock.json`、`npm-shrinkwrap.json` は package 宣言を symbol として保持しますが、`dependency` reference は明示された親 package → 子 package の項目だけに出力します。親 package を reference container に保存するため、`callers` はどの package が子 package を必要としているかを特定でき、`deps` は同じ resolved package 集合を持つだけの lock file 同士を類似関係として推論しません(#4409)。 +- **lock file の依存グラフは package 間の関係をモデル化** — `packages.lock.json`、`package-lock.json`、`npm-shrinkwrap.json` は package 宣言を symbol として保持しますが、`dependency` reference は明示された親 package → 子 package の項目だけに出力します。NuGet lock の symbol / reference は現在の file、target/RID、親 package、正確な JSON property span を保持し、candidate 解決を file 内に限定します。file 単位の `deps` は package 名による file 間推論を抑止し、通常の index update は以前の dependency-lock 抽出 contract と reference-identity contract を無効化します。そのため、`callers` は無関係な lock file を接続したり、反復宣言を最初の一致行へ畳み込んだりせず、要求元 package を特定できます(#4409、#4845)。 - **依存サイクル監査では解析と表示を分離** — CLI の `deps --cycles` と MCP `deps` の `cycles=true` は、独立した `--graph-budget` / `graphBudget` まで path 順で決定的な edge 集合を解析してから、強連結成分を安定順位付けします。`--limit` / `limit` はその SCC 順位集合をページ分割するだけで、不透明 cursor は生成時の filter、graph budget、indexed graph に結び付けます。machine-readable 応答は `analysis_complete`、graph edge 件数/予算、安定 ranking mode、authoritative な総 cycle 件数かどうか、continuation metadata を公開し、graph budget 枯渇時は完全な cycle 監査を装わず明示的な未完了解析として報告します(#4731)。 - **ORMなし** — `Microsoft.Data.Sqlite`でパラメータ化クエリを直接使用。依存関係を最小限に、制御を明確に。 - **バッチコミット** — 書き込み性能のため1トランザクション500レコード。fsyncオーバーヘッドを削減。 diff --git a/TESTING_GUIDE.md b/TESTING_GUIDE.md index 00eb30d65..c504088c9 100644 --- a/TESTING_GUIDE.md +++ b/TESTING_GUIDE.md @@ -540,7 +540,7 @@ Use `docs/test-doc-maintenance-plan.md` before moving oversized suites or adding State-changing notification authentication coverage stays table-driven across cancellation, roots, shutdown, and exit methods; every denied notification must remain response-free, emit only a bounded diagnostic, and leave cancellation, roots, and lifecycle state unchanged. `resources/read` coverage shares the seeded resource helpers for range, UTF-8 byte-boundary, and cursor-reassembly assertions. Keep deterministic coverage for a concurrent reindex during a read snapshot, stale cursors at direct and transport boundaries, empty and newline-only resources, leading/middle/trailing chunk gaps, missing content and chunk tables, chunk-count and aggregate-scan caps, and escape-heavy responses under both stdio and HTTP response ceilings. Range-seek plan and execution checks share the production SQL strings, verify VM-step scaling and the legacy VM-step budget, and confirm that a budget abort leaves the connection reusable. Consume `maxLines` cursors through exact reconstruction, and cover immutable legacy databases that have only `idx_chunks_file`. Keep real stdio and HTTP wire coverage for aggregate response budgeting across batched `resources/read` calls, including long error responses that must retain every request ID through compact budget errors, plus a real `StdioMcpTransport` wire test for both long single-line and multi-line resources, so regressions cannot hide behind direct handler tests. - `DependencyPackageExtractorTests.cs` - Dependency-lock graph fixtures assert explicit parent-package to child-package references and the absence of synthetic top-level package references. Keep NuGet and npm coverage aligned so shared resolved package sets cannot recreate lock-file-to-lock-file similarity edges while `callers` retains the requiring package as its container. + Dependency-lock graph fixtures assert explicit parent-package to child-package references and the absence of synthetic top-level package references. Keep NuGet and npm coverage aligned so shared resolved package sets cannot recreate lock-file-to-lock-file similarity edges in either `deps` or caller traversal while `callers` retains the requiring package as its container. NuGet query regressions must use two unrelated lock files and repeated packages across targets and multiple RIDs, then assert file-local definitions, owner-scoped callers/callees, and every JSON node's real line. Upgrade regressions must also prove that prior dependency-lock extractor and reference-identity contract stamps invalidate unchanged rows and stale candidates. - `HttpMcpTransportTests.cs` HTTP MCP transport behavior, including authentication responses, warm server reuse, concurrent requests, and request logging. Request-log assertions must validate recorded contents without assuming callback order between independently handled HTTP requests. Its default database fixture follows the same thread-safe lazy-initialization rule as `McpServerTests`, so tests that use an isolated loopback/direct harness do not create an unused schema while concurrent access still initializes the fixture once. @@ -1476,6 +1476,8 @@ dotnet test --filter "FullyQualifiedName~GitHelperTests" timeout 対象 hook の delay は callback budget より十分長く、かつ bounded な leak-observation window より短く保ってください。高負荷 runner で hook が timeout 境界上に完了する競合を避けながら、worker kill の回帰を観測可能にします。 - `GitHubIssueReporterTests.cs` GitHubトークン解決ロジック(CDIDX_GITHUB_TOKENのみ。汎用GITHUB_TOKENは無視)、送信前のコード scrubbing、冪等性チェック、rate-limit diagnostics を扱います。 +- `DependencyPackageExtractorTests.cs` + dependency-lock graph の fixture は明示的な親 package → 子 package reference と、合成された top-level package reference が存在しないことを検証します。NuGet と npm の coverage を揃え、同じ resolved package 集合から `deps` と caller traversal のいずれにも lock-file 間の類似 edge が再生成されず、`callers` が要求元 package を container として維持することを確認してください。NuGet の query regression では無関係な2つの lock file と、複数 target / RID に反復する package を使い、file 内に限定された定義、owner scope の callers/callees、各 JSON node の実際の行を検証します。upgrade regression では、以前の dependency-lock extractor / reference-identity contract stamp により unchanged row と stale candidate が無効化されることも検証します。 - `PackagesLockTests.cs` すべての target framework で同期が必要な direct package reference の NuGet lock-file guard。CI の locked restore を通すための net9.0 compatibility reference も対象です。 workflow restore contract の coverage では、release native lane の locked `net8.0` test-project restore と cross-compile lane の locked production-project restore を区別しつつ、全 surface の exact cache key を維持します。 diff --git a/changelog.d/unreleased/4845.fixed.md b/changelog.d/unreleased/4845.fixed.md new file mode 100644 index 000000000..669c0f728 --- /dev/null +++ b/changelog.d/unreleased/4845.fixed.md @@ -0,0 +1,29 @@ +--- +category: fixed +issues: + - 4845 +affected: + - src/CodeIndex/Indexer/DependencyPackageExtractor.cs + - src/CodeIndex/Indexer/DependencyPackageExtractor.JsonLocations.cs + - src/CodeIndex/Indexer/Symbols/SymbolExtractor.Contracts.cs + - src/CodeIndex/Database/DbContext.SchemaMetadata.cs + - src/CodeIndex/Database/DbReader.Dependencies.cs + - src/CodeIndex/Database/DbWriter.References.cs + - src/CodeIndex/Database/DbReader.cs + - src/CodeIndex/Database/DbReader.GraphQueries.cs + - tests/CodeIndex.Tests/DatabaseTests.cs + - tests/CodeIndex.Tests/DbReaderTests.cs + - tests/CodeIndex.Tests/DbReaderInspectCandidateTests.cs + - tests/CodeIndex.Tests/SymbolExtractorTests.cs + - DEVELOPER_GUIDE.md + - tests/CodeIndex.Tests/IndexCommandRunnerUpdateTests.cs + - TESTING_GUIDE.md +--- + +## English + +- **NuGet lock graphs now preserve file, target, owner, and JSON-node identity (#4845)** — `packages.lock.json` symbols retain each declaration's real property span, nested dependency references retain their parent package and target/RID, and dependency-lock candidate resolution stays within the current file. Default caller/callee traversal includes these ownership edges, file-level `deps` suppresses cross-lock name inference, and normal index updates invalidate prior extraction/identity contracts, preventing the #4409 regression that connected unrelated lock files or attached children to the project scope. + +## 日本語 + +- **NuGet lock graph で file、target、owner、JSON node の identity を保持するようにしました (#4845)** — `packages.lock.json` の symbol は各宣言の実際の property span を保持し、入れ子 dependency reference は親 package と target/RID を維持し、dependency-lock の candidate 解決は現在の file 内に限定されます。既定の caller/callee traversal はこの所有関係 edge を含み、file 単位の `deps` は lock file 間の名前推論を抑止し、通常の index update は以前の抽出 / identity contract を無効化するため、無関係な lock file を接続したり、子 package を project scope に関連付けたりした #4409 の regression を防ぎます。 diff --git a/src/CodeIndex/Database/DbContext.SchemaMetadata.cs b/src/CodeIndex/Database/DbContext.SchemaMetadata.cs index 447cf8092..2c776ba92 100644 --- a/src/CodeIndex/Database/DbContext.SchemaMetadata.cs +++ b/src/CodeIndex/Database/DbContext.SchemaMetadata.cs @@ -62,14 +62,13 @@ public static string BuildIncompleteHotspotFamilyMarkerFingerprint(string? finge public const string SqlGraphContractVersionMetaKey = "sql_graph_contract_version"; public const int HdlGraphContractVersion = 1; public const string HdlGraphContractVersionMetaKey = "hdl_graph_contract_version"; - // Version 3 (#4825) invalidates persisted reference candidates written before C# - // type references were constrained to type-like symbols with compatible generic arity. - // Version 2 DBs can otherwise keep authoritative-looking property bindings and false - // dependency edges until an explicit index refresh. - // バージョン 3 (#4825) では、C# の型参照を型相当 symbol と互換 generic arity に限定する - // 前に永続化された reference candidate を無効化する。バージョン 2 の DB をそのまま信頼すると、 - // 明示的な index 更新まで property への誤結合と偽 dependency edge が残るためである。 - public const int ReferenceIdentityContractVersion = 3; + // Version 4 (#4845) invalidates dependency-lock candidates written before resolution + // became file-local. Version 3 (#4825) previously invalidated C# type-reference + // candidates written before type-like kind and compatible generic-arity constraints. + // バージョン 4 (#4845) では、解決を file 内に限定する前に書かれた dependency-lock + // candidate を無効化する。バージョン 3 (#4825) では、それ以前に C# 型参照の + // type-like kind / 互換 generic arity 制約より前に書かれた candidate を無効化した。 + public const int ReferenceIdentityContractVersion = 4; public const string ReferenceIdentityContractVersionMetaKey = "reference_identity_contract_version"; public static string GetDynamicReferenceGraphContractVersionMetaKey(string lang) => $"dynamic_reference_graph_contract_version_{lang}"; diff --git a/src/CodeIndex/Database/DbReader.Dependencies.cs b/src/CodeIndex/Database/DbReader.Dependencies.cs index 33d227007..173efef6e 100644 --- a/src/CodeIndex/Database/DbReader.Dependencies.cs +++ b/src/CodeIndex/Database/DbReader.Dependencies.cs @@ -243,17 +243,22 @@ public List GetFileDependencies( var contextSql = ReferenceContextSql("r"); var hasResolvedReferenceTargets = _referenceIdentityContractCurrent; var resolutionStateSql = hasResolvedReferenceTargets ? "r.resolution_state" : "NULL"; - var identityScopeConditionSql = "src.lang = 'csharp' AND r.reference_kind NOT IN ('attribute', 'annotation')"; + var identityScopeConditionSql = """ + ( + (src.lang = 'csharp' AND r.reference_kind NOT IN ('attribute', 'annotation')) + OR (src.lang = 'dependency_lock' AND r.reference_kind = 'dependency') + ) + """; var referenceIdSql = $"CASE WHEN {identityScopeConditionSql} THEN r.id ELSE 0 END"; var scopedResolutionStateSql = $"CASE WHEN {identityScopeConditionSql} THEN {resolutionStateSql} ELSE NULL END"; var identityScopedSql = $"CASE WHEN {identityScopeConditionSql} THEN 1 ELSE 0 END"; - var csharpNameEdgePredicate = hasResolvedReferenceTargets - ? "(snc.source_lang != 'csharp' OR snc.identity_scoped = 0)" - : "1 = 1"; + var identityNameEdgePredicate = hasResolvedReferenceTargets + ? "snc.identity_scoped = 0" + : "snc.source_lang <> 'dependency_lock'"; var resolvedIdentityLimitSql = lang == "csharp" ? " LIMIT @sourceCandidateLimit" : string.Empty; - var resolvedCSharpEdgesSql = hasResolvedReferenceTargets + var resolvedIdentityEdgesSql = hasResolvedReferenceTargets ? @" SELECT resolved.source_path, resolved.target_path, @@ -272,8 +277,7 @@ JOIN symbol_reference_candidates candidate JOIN target_files scoped_target ON scoped_target.target_path = target_file.path AND scoped_target.target_lang = target_file.lang - WHERE lrp.source_lang = 'csharp' - AND lrp.identity_scoped = 1 + WHERE lrp.identity_scoped = 1 AND lrp.resolution_state IN ('resolved', 'resolved_group') AND lrp.source_path != target_file.path" : string.Empty; @@ -281,13 +285,13 @@ AND lrp.resolution_state IN ('resolved', 'resolved_group') { AppendDependencySymbolFilter( cmd, - ref resolvedCSharpEdgesSql, + ref resolvedIdentityEdgesSql, "lrp.symbol_name", dependencySymbols, dependencySymbolFamilies, suppressDependencyNoise, "resolvedDependency"); - resolvedCSharpEdgesSql += @" + resolvedIdentityEdgesSql += @" ORDER BY lrp.source_path, lrp.symbol_name, lrp.reference_id" + resolvedIdentityLimitSql + @" ) resolved GROUP BY resolved.source_path, resolved.target_path, resolved.symbol_name @@ -676,7 +680,7 @@ JOIN symbols s GROUP BY tf.target_lang, tf.symbol_name, tf.symbol_segment_count ), edges AS ( - " + resolvedCSharpEdgesSql + @" + " + resolvedIdentityEdgesSql + @" SELECT snc.source_path, tf.target_path, tf.symbol_name, @@ -693,7 +697,7 @@ LEFT JOIN target_ambiguity ta AND ta.symbol_name = snc.symbol_name AND ta.symbol_segment_count = snc.symbol_segment_count WHERE snc.source_path != tf.target_path - AND " + csharpNameEdgePredicate + @" + AND " + identityNameEdgePredicate + @" -- All metadata references ([Foo] / @Foo) and their synthetic C# -- suffix aliases must only match class-like target kinds; otherwise -- a metadata reference would spuriously depend on any file that diff --git a/src/CodeIndex/Database/DbReader.GraphQueries.cs b/src/CodeIndex/Database/DbReader.GraphQueries.cs index e66ffed1b..420100b83 100644 --- a/src/CodeIndex/Database/DbReader.GraphQueries.cs +++ b/src/CodeIndex/Database/DbReader.GraphQueries.cs @@ -106,7 +106,7 @@ WITH logical_references AS ( FROM symbol_references r JOIN files f ON r.file_id = f.id" + referenceLineJoin + @" WHERE " + callerContainerPredicate + @" - AND " + GetCallableReferenceKindPredicateSql("r.reference_kind", referenceKind) + @" + AND " + GetCallableReferenceKindPredicateSql("r.reference_kind", referenceKind, "f.lang") + @" AND " + supportedLangPredicate; if (targetSymbolId != null && HasTable("symbol_reference_candidates")) { @@ -294,7 +294,7 @@ FROM symbol_references r WHERE " + BuildCallerContainerPredicate("f", "r"); groupedSql += $" AND {BuildGraphSupportedLanguagePredicate(cmd, "f", "graphLang")}"; - groupedSql += $" AND {GetCallableReferenceKindPredicateSql("r.reference_kind", referenceKind)}"; + groupedSql += $" AND {GetCallableReferenceKindPredicateSql("r.reference_kind", referenceKind, "f.lang")}"; var allowSqlLeafFallback = AllowSqlLeafFallbackForQuery(query); var allowCSharpQualifiedContextMatch = SqlNameResolver.HasQualifier(query) && !HasQualifiedSymbolDefinition(query, lang, pathPatterns, excludePathPatterns, excludeTests); @@ -429,7 +429,7 @@ FROM symbol_references r WHERE " + BuildCallerContainerPredicate("f", "r"); groupedSql += $" AND {BuildGraphSupportedLanguagePredicate(cmd, "f", "graphLang")}"; - groupedSql += $" AND {GetCallableReferenceKindPredicateSql("r.reference_kind", referenceKind)}"; + groupedSql += $" AND {GetCallableReferenceKindPredicateSql("r.reference_kind", referenceKind, "f.lang")}"; if (targetSymbolId != null && HasTable("symbol_reference_candidates")) { groupedSql += _referenceColumns.Contains("resolution_state") @@ -593,7 +593,7 @@ WITH logical_references AS ( FROM symbol_references r JOIN files f ON r.file_id = f.id WHERE r.container_name IS NOT NULL - AND {GetCallableReferenceKindPredicateSql("r.reference_kind", referenceKind)} + AND {GetCallableReferenceKindPredicateSql("r.reference_kind", referenceKind, "f.lang")} AND {BuildGraphSupportedLanguagePredicate(cmd, "f", "graphLang")}"; if (sourceSymbolId != null && _referenceColumns.Contains("source_symbol_id")) sql += " AND r.source_symbol_id = @sourceSymbolId"; @@ -758,7 +758,7 @@ FROM symbol_references r WHERE r.container_name IS NOT NULL"; groupedSql += $" AND {BuildGraphSupportedLanguagePredicate(cmd, "f", "graphLang")}"; - groupedSql += $" AND {GetCallableReferenceKindPredicateSql("r.reference_kind", referenceKind)}"; + groupedSql += $" AND {GetCallableReferenceKindPredicateSql("r.reference_kind", referenceKind, "f.lang")}"; var allowSqlLeafFallback = AllowSqlLeafFallbackForQuery(query); var allowQualifiedLeafFallback = HasSingleQualifiedSymbolDefinition(query, lang, pathPatterns, excludePathPatterns, excludeTests); var useSqlQualifiedContainerMatch = SqlNameResolver.HasQualifier(query); @@ -881,7 +881,7 @@ FROM symbol_references r WHERE r.container_name IS NOT NULL"; groupedSql += $" AND {BuildGraphSupportedLanguagePredicate(cmd, "f", "graphLang")}"; - groupedSql += $" AND {GetCallableReferenceKindPredicateSql("r.reference_kind", referenceKind)}"; + groupedSql += $" AND {GetCallableReferenceKindPredicateSql("r.reference_kind", referenceKind, "f.lang")}"; if (sourceSymbolId != null && _referenceColumns.Contains("source_symbol_id")) groupedSql += " AND r.source_symbol_id = @sourceSymbolId"; var allowSqlLeafFallback = AllowSqlLeafFallbackForQuery(query); diff --git a/src/CodeIndex/Database/DbReader.cs b/src/CodeIndex/Database/DbReader.cs index ab9efb2e3..fbe7ac284 100644 --- a/src/CodeIndex/Database/DbReader.cs +++ b/src/CodeIndex/Database/DbReader.cs @@ -244,10 +244,12 @@ ELSE 0 END"; private const string EventReferenceKindsSql = "('subscribe', 'unsubscribe', 'razor_event_binding')"; private const string ImpactAnchorReferenceKindsSql = "('call', 'instantiate', 'subscribe', 'unsubscribe', 'razor_event_binding', 'capture')"; - // Default callers/callees expose only executable edges. Broader dependency kinds remain - // available to impact/hotspot queries and through an explicit reference-kind filter. - // 既定の callers/callees は実行可能な edge だけを公開する。より広い依存 kind は - // impact/hotspot query と明示的な reference-kind filter から引き続き利用できる。 + // Default callers/callees expose executable edges plus dependency-lock ownership edges. + // Broader dependency kinds remain available to impact/hotspot queries and through an + // explicit reference-kind filter. + // 既定の callers/callees は実行可能な edge と dependency-lock の所有関係 edge を公開する。 + // より広い依存 kind は impact/hotspot query と明示的な reference-kind filter から + // 引き続き利用できる。 private const string CallableReferenceKindsSql = "('call', 'instantiate', 'subscribe', 'unsubscribe', 'razor_event_binding')"; // Reference kinds that participate in dependency-oriented graph traversal and hotspot ranking. // Default callers/callees use the narrower CallableReferenceKindsSql allowlist above. Metadata @@ -351,9 +353,15 @@ internal static string GetLogicalReferenceKindSql(string referenceKindSql) // filter semantics と output projection を分離する。canonical `subscribe` filter は raw event // variant 全体を選び、明示された raw variant はその kind だけを選ぶ。公開 label を // canonical / raw のどちらにするかは呼び出し側で別途決める。 - private static string GetCallableReferenceKindPredicateSql(string referenceKindSql, string? referenceKind) + private static string GetCallableReferenceKindPredicateSql( + string referenceKindSql, + string? referenceKind, + string? sourceLanguageSql = null) => referenceKind switch { + null when sourceLanguageSql is not null => + $"({referenceKindSql} IN {CallableReferenceKindsSql} OR " + + $"({sourceLanguageSql} = 'dependency_lock' AND {referenceKindSql} = 'dependency'))", null => $"{referenceKindSql} IN {CallableReferenceKindsSql}", "subscribe" => $"{referenceKindSql} IN {EventReferenceKindsSql}", _ => $"{referenceKindSql} = @referenceKind", diff --git a/src/CodeIndex/Database/DbWriter.References.cs b/src/CodeIndex/Database/DbWriter.References.cs index f039b9412..86151ae26 100644 --- a/src/CodeIndex/Database/DbWriter.References.cs +++ b/src/CodeIndex/Database/DbWriter.References.cs @@ -443,6 +443,7 @@ THEN r.symbol_name_folded || 'attribute' END AND (source_file.lang <> 'ambiguous_m' OR source_file.id = target_file.id)) OR (source_file.lang = 'ambiguous_m' AND target_file.lang IN ('matlab', 'objc')) ) + AND (source_file.lang <> 'dependency_lock' OR s.file_id = r.file_id) AND {CSharpTypeReferenceCandidatePredicateSql} AND r.target_qualifier IS NOT NULL AND r.target_qualifier NOT LIKE char(31) || 'receiver:%' @@ -564,6 +565,7 @@ THEN r.symbol_name_folded || 'attribute' END ) AND {CSharpTypeReferenceCandidatePredicateSql} AND r.target_qualifier IS NULL + AND (source_file.lang <> 'dependency_lock' OR s.file_id = r.file_id) AND source.container_qualified_name IS NOT NULL AND source.container_qualified_name <> '' AND s.container_qualified_name = source.container_qualified_name COLLATE NOCASE @@ -615,6 +617,7 @@ THEN r.symbol_name_folded || 'attribute' END ) AND {CSharpTypeReferenceCandidatePredicateSql} AND r.target_qualifier IS NULL + AND (source_file.lang <> 'dependency_lock' OR s.file_id = r.file_id) AND source.container_name IS NOT NULL AND source.container_name <> '' AND s.container_name = source.container_name COLLATE NOCASE @@ -717,7 +720,8 @@ OR source_file.lang IN ( AND NOT EXISTS ( SELECT 1 FROM symbol_reference_candidates AS existing WHERE existing.reference_id = r.id - ); + ) + AND (source_file.lang <> 'dependency_lock' OR target.file_id = r.file_id); INSERT INTO symbol_reference_candidates(reference_id, symbol_id, scope_rank) SELECT r.id, target.id, 5 diff --git a/src/CodeIndex/Indexer/DependencyPackageExtractor.JsonLocations.cs b/src/CodeIndex/Indexer/DependencyPackageExtractor.JsonLocations.cs new file mode 100644 index 000000000..1d85a5f48 --- /dev/null +++ b/src/CodeIndex/Indexer/DependencyPackageExtractor.JsonLocations.cs @@ -0,0 +1,204 @@ +using System.Text; +using System.Text.Json; + +namespace CodeIndex.Indexer; + +internal static partial class DependencyPackageExtractor +{ + private sealed class JsonObjectLocations + { + public List Properties { get; } = []; + } + + private sealed class JsonPropertyLocation(string name, int line, int column) + { + public string Name { get; } = name; + public int Line { get; } = line; + public int Column { get; } = column; + public int EndLine { get; set; } = line; + public JsonObjectLocations? ObjectValue { get; set; } + } + + private readonly record struct JsonLocationContainer( + JsonObjectLocations? ObjectValue, + JsonPropertyLocation? OwningProperty); + + private static JsonObjectLocations? TryParseJsonObjectLocations(string content) + { + try + { + var utf8 = Encoding.UTF8.GetBytes(content); + var locator = new Utf8JsonSourceLocator(utf8); + var reader = new Utf8JsonReader( + utf8, + new JsonReaderOptions + { + MaxDepth = MaxJsonLockParseDepth, + }); + var containers = new Stack(); + JsonObjectLocations? root = null; + JsonPropertyLocation? pendingProperty = null; + + while (reader.Read()) + { + switch (reader.TokenType) + { + case JsonTokenType.StartObject: + { + var objectLocations = new JsonObjectLocations(); + if (pendingProperty is not null) + { + pendingProperty.ObjectValue = objectLocations; + } + + root ??= objectLocations; + containers.Push(new JsonLocationContainer(objectLocations, pendingProperty)); + pendingProperty = null; + break; + } + + case JsonTokenType.EndObject: + { + if (containers.Count == 0) + { + return null; + } + + var container = containers.Pop(); + if (container.OwningProperty is not null) + { + container.OwningProperty.EndLine = locator.GetLine(reader.TokenStartIndex); + } + + pendingProperty = null; + break; + } + + case JsonTokenType.StartArray: + containers.Push(new JsonLocationContainer(null, pendingProperty)); + pendingProperty = null; + break; + + case JsonTokenType.EndArray: + { + if (containers.Count == 0) + { + return null; + } + + var container = containers.Pop(); + if (container.OwningProperty is not null) + { + container.OwningProperty.EndLine = locator.GetLine(reader.TokenStartIndex); + } + + pendingProperty = null; + break; + } + + case JsonTokenType.PropertyName: + { + if (containers.Count == 0 || containers.Peek().ObjectValue is not { } parent) + { + return null; + } + + var (line, column) = locator.GetLocation(reader.TokenStartIndex); + var property = new JsonPropertyLocation(reader.GetString() ?? string.Empty, line, column); + parent.Properties.Add(property); + pendingProperty = property; + break; + } + + case JsonTokenType.String: + case JsonTokenType.Number: + case JsonTokenType.True: + case JsonTokenType.False: + case JsonTokenType.Null: + if (pendingProperty is not null) + { + pendingProperty.EndLine = locator.GetLine(reader.TokenStartIndex); + } + + pendingProperty = null; + break; + } + } + + return containers.Count == 0 ? root : null; + } + catch (JsonException) + { + return null; + } + } + + private static JsonPropertyLocation? FindLocatedProperty( + JsonObjectLocations? objectLocations, + string name) + { + if (objectLocations is null) + { + return null; + } + + return objectLocations.Properties.FirstOrDefault( + property => string.Equals(property.Name, name, StringComparison.Ordinal)); + } + + private static IEnumerable<(JsonProperty Property, JsonPropertyLocation? Location)> EnumerateLocatedProperties( + JsonElement element, + JsonObjectLocations? objectLocations) + { + var locations = objectLocations?.Properties; + var locationIndex = 0; + foreach (var property in element.EnumerateObject()) + { + JsonPropertyLocation? location = null; + if (locations is not null) + { + while (locationIndex < locations.Count) + { + var candidate = locations[locationIndex++]; + if (string.Equals(candidate.Name, property.Name, StringComparison.Ordinal)) + { + location = candidate; + break; + } + } + } + + yield return (property, location); + } + } + + private sealed class Utf8JsonSourceLocator + { + private readonly byte[] _utf8; + private readonly List _lineStarts = [0]; + + public Utf8JsonSourceLocator(byte[] utf8) + { + _utf8 = utf8; + for (var index = 0; index < utf8.Length; index++) + { + if (utf8[index] == (byte)'\n') + { + _lineStarts.Add(index + 1); + } + } + } + + public (int Line, int Column) GetLocation(long byteOffset) + { + var offset = checked((int)byteOffset); + var search = _lineStarts.BinarySearch(offset); + var lineIndex = search >= 0 ? search : ~search - 1; + var lineStart = _lineStarts[lineIndex]; + var column = Encoding.UTF8.GetCharCount(_utf8, lineStart, offset - lineStart) + 1; + return (lineIndex + 1, column); + } + + public int GetLine(long byteOffset) => GetLocation(byteOffset).Line; + } +} diff --git a/src/CodeIndex/Indexer/DependencyPackageExtractor.cs b/src/CodeIndex/Indexer/DependencyPackageExtractor.cs index ddd6ca003..1d34026af 100644 --- a/src/CodeIndex/Indexer/DependencyPackageExtractor.cs +++ b/src/CodeIndex/Indexer/DependencyPackageExtractor.cs @@ -17,9 +17,10 @@ internal readonly record struct DependencyPackageInfo( string? Role, int Line, int Column, + int EndLine, string Signature); -internal static class DependencyPackageExtractor +internal static partial class DependencyPackageExtractor { internal const int MaxJsonLockParseBytes = 16 * 1024 * 1024; internal const int MaxJsonLockParseDepth = 64; @@ -64,7 +65,7 @@ public static List ExtractSymbols(long fileId, string content, str Line = package.Line, StartLine = package.Line, StartColumn = Math.Max(0, package.Column - 1), - EndLine = package.Line, + EndLine = package.EndLine, Signature = package.Signature, ContainerKind = package.Scope == null ? null : "project", ContainerName = package.Scope, @@ -188,7 +189,12 @@ private static List ExtractLockDependencyReferences( { var fileName = Path.GetFileName(path ?? string.Empty); if (string.Equals(fileName, "packages.lock.json", StringComparison.OrdinalIgnoreCase)) - ExtractNuGetLockDependencyReferences(fileId, document.RootElement, lines, references); + ExtractNuGetLockDependencyReferences( + fileId, + document.RootElement, + lines, + TryParseJsonObjectLocations(content), + references); else if (string.Equals(fileName, "package-lock.json", StringComparison.OrdinalIgnoreCase) || string.Equals(fileName, "npm-shrinkwrap.json", StringComparison.OrdinalIgnoreCase)) ExtractNpmLockDependencyReferences(fileId, document.RootElement, lines, references); @@ -201,27 +207,37 @@ private static void ExtractNuGetLockDependencyReferences( long fileId, JsonElement root, string[] lines, + JsonObjectLocations? rootLocations, List references) { if (!root.TryGetProperty("dependencies", out var frameworks) || frameworks.ValueKind != JsonValueKind.Object) return; - foreach (var framework in frameworks.EnumerateObject()) + var frameworkLocations = FindLocatedProperty(rootLocations, "dependencies")?.ObjectValue; + foreach (var (framework, frameworkLocation) in EnumerateLocatedProperties(frameworks, frameworkLocations)) { if (framework.Value.ValueKind != JsonValueKind.Object) continue; - var packageSearchLine = FindJsonProperty(lines, framework.Name).Line; - foreach (var package in framework.Value.EnumerateObject()) + foreach (var (package, packageLocation) in EnumerateLocatedProperties( + framework.Value, + frameworkLocation?.ObjectValue)) { if (package.Value.ValueKind != JsonValueKind.Object || !package.Value.TryGetProperty("dependencies", out var dependencies) || dependencies.ValueKind != JsonValueKind.Object) continue; - var packageLocation = FindJsonProperty(lines, package.Name, packageSearchLine); - packageSearchLine = packageLocation.Line + 1; - if (!AddPackageDependencyReferences(fileId, dependencies, package.Name, packageLocation.Line, lines, references)) + var dependencyLocations = FindLocatedProperty(packageLocation?.ObjectValue, "dependencies")?.ObjectValue; + if (!AddPackageDependencyReferences( + fileId, + dependencies, + package.Name, + packageLocation?.Line ?? 1, + lines, + references, + dependencyLocations, + targetQualifier: framework.Name)) return; } } @@ -265,12 +281,16 @@ private static bool AddPackageDependencyReferences( string parentName, int parentLine, string[] lines, - List references) + List references, + JsonObjectLocations? dependencyLocations = null, + string? targetQualifier = null) { var dependencySearchLine = parentLine; - foreach (var dependency in dependencies.EnumerateObject()) + foreach (var (dependency, dependencyLocation) in EnumerateLocatedProperties(dependencies, dependencyLocations)) { - var location = FindJsonProperty(lines, dependency.Name, dependencySearchLine); + var location = dependencyLocation == null + ? FindJsonProperty(lines, dependency.Name, dependencySearchLine) + : (dependencyLocation.Line, dependencyLocation.Column); dependencySearchLine = location.Line + 1; if (!ReferenceExtractor.TryAddReference( references, @@ -284,6 +304,7 @@ private static bool AddPackageDependencyReferences( Context = GetContext(lines, location.Line), ContainerKind = "package", ContainerName = parentName, + TargetQualifier = targetQualifier, })) return false; } @@ -426,7 +447,12 @@ private static void ExtractJsonLock( using (document) { if (string.Equals(fileName, "packages.lock.json", StringComparison.OrdinalIgnoreCase)) - ExtractNuGetPackagesLock(document.RootElement, lines, packages, seen); + ExtractNuGetPackagesLock( + document.RootElement, + lines, + TryParseJsonObjectLocations(content), + packages, + seen); else if (string.Equals(fileName, "package-lock.json", StringComparison.OrdinalIgnoreCase) || string.Equals(fileName, "npm-shrinkwrap.json", StringComparison.OrdinalIgnoreCase)) ExtractNpmPackageLock(document.RootElement, lines, packages, seen); @@ -436,24 +462,30 @@ private static void ExtractJsonLock( private static void ExtractNuGetPackagesLock( JsonElement root, string[] lines, + JsonObjectLocations? rootLocations, List packages, HashSet seen) { if (!root.TryGetProperty("dependencies", out var dependencies) || dependencies.ValueKind != JsonValueKind.Object) return; - foreach (var framework in dependencies.EnumerateObject()) + var frameworkLocations = FindLocatedProperty(rootLocations, "dependencies")?.ObjectValue; + foreach (var (framework, frameworkLocation) in EnumerateLocatedProperties(dependencies, frameworkLocations)) { if (framework.Value.ValueKind != JsonValueKind.Object) continue; - foreach (var package in framework.Value.EnumerateObject()) + foreach (var (package, packageLocation) in EnumerateLocatedProperties( + framework.Value, + frameworkLocation?.ObjectValue)) { var packageObject = package.Value; var role = GetStringProperty(packageObject, "type"); var resolved = GetStringProperty(packageObject, "resolved"); var requested = GetStringProperty(packageObject, "requested"); - var location = FindJsonProperty(lines, package.Name); + var location = packageLocation == null + ? FindJsonProperty(lines, package.Name) + : (packageLocation.Line, packageLocation.Column); var normalizedRole = NormalizeRole(role); AddPackage( @@ -467,7 +499,8 @@ private static void ExtractNuGetPackagesLock( scope: framework.Name, role: normalizedRole, location.Line, - location.Column); + location.Column, + endLine: packageLocation?.EndLine); } } } @@ -709,7 +742,8 @@ private static void AddPackage( string? scope, string? role, int line, - int column) + int column, + int? endLine = null) { var trimmedName = name.AsSpan().Trim(); if (trimmedName.IsEmpty) @@ -718,6 +752,7 @@ private static void AddPackage( name = TrimDependencyField(name, trimmedName); line = Math.Max(1, line); column = Math.Max(1, column); + endLine = Math.Max(line, endLine ?? line); version = NormalizeEmpty(version); requestedVersion = NormalizeEmpty(requestedVersion); scope = NormalizeEmpty(scope); @@ -746,6 +781,7 @@ private static void AddPackage( role, line, column, + endLine.Value, BuildSignature(name, version, requestedVersion, sourceKind, scope, role))); } diff --git a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.Contracts.cs b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.Contracts.cs index 23451b9b2..e8b56e06a 100644 --- a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.Contracts.cs +++ b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.Contracts.cs @@ -18,6 +18,7 @@ public static partial class SymbolExtractor public const int ScientificNativeGraphContractVersion = 4; public const int RepositoryMetadataContractVersion = 2; public const int ApplicationManifestContractVersion = 3; + public const int DependencyLockContractVersion = 3; private static readonly string[] ExplicitReferenceGraphContractLanguages = ["crystal", "groovy", "tcl", "prolog", "ambiguous_pl"]; @@ -68,7 +69,8 @@ public static int GetContractVersion(string? lang) "ada" or "ambiguous_m" or "cython" or "d" or "julia" or "matlab" or "nim" or "objc" => ScientificNativeGraphContractVersion, "config" or "dockerignore" or "editorconfig" or "gitattributes" or "gitignore" or "jsonl" or "toml" => RepositoryMetadataContractVersion, "app_manifest" => ApplicationManifestContractVersion, - "cmake" or "dependency_lock" or "dependency_manifest" or "graphql" or "html" or "json" or "justfile" or "markdown" or "msbuild" or "solution" or "yaml" => ExpandedLanguageContractVersion, + "dependency_lock" => DependencyLockContractVersion, + "cmake" or "dependency_manifest" or "graphql" or "html" or "json" or "justfile" or "markdown" or "msbuild" or "solution" or "yaml" => ExpandedLanguageContractVersion, _ => DefaultContractVersion, }; } diff --git a/tests/CodeIndex.Tests/DatabaseTests.cs b/tests/CodeIndex.Tests/DatabaseTests.cs index 6077cea8c..2480a0694 100644 --- a/tests/CodeIndex.Tests/DatabaseTests.cs +++ b/tests/CodeIndex.Tests/DatabaseTests.cs @@ -5482,6 +5482,29 @@ public void GetUnchangedFileId_ReturnsNullWhenLanguageExtractorVersionIsStale() Assert.Null(id); } + [Fact] + public void GetUnchangedFileId_InvalidatesPriorDependencyLockContract_Issue4845() + { + const string language = "dependency_lock"; + const int previousContractVersion = 2; + var modified = new DateTime(2025, 1, 1, 0, 0, 0, DateTimeKind.Utc); + var file = new FileRecord + { + Path = "packages.lock.json", + Lang = language, + Size = 50, + Lines = 5, + Modified = modified, + }; + _writer.UpsertFile(file); + _writer.SetMeta( + DbContext.GetSymbolExtractorVersionMetaKey(language), + previousContractVersion.ToString(CultureInfo.InvariantCulture)); + + Assert.True(SymbolExtractor.GetContractVersion(language) > previousContractVersion); + Assert.Null(_writer.GetUnchangedFileId(file.Path, modified, language: language)); + } + [Theory] [InlineData("crystal", 2)] [InlineData("groovy", 2)] diff --git a/tests/CodeIndex.Tests/DbReaderInspectCandidateTests.cs b/tests/CodeIndex.Tests/DbReaderInspectCandidateTests.cs index bd17a57d7..e13083252 100644 --- a/tests/CodeIndex.Tests/DbReaderInspectCandidateTests.cs +++ b/tests/CodeIndex.Tests/DbReaderInspectCandidateTests.cs @@ -61,4 +61,214 @@ private void QueryOnly() { } Assert.Contains(queryBundle.Callees, callee => callee.CalleeName == "QueryOnly"); Assert.DoesNotContain(queryBundle.Callees, callee => callee.CalleeName == "IndexOnly"); } + + [Fact] + public void AnalyzeSymbol_DependencyLockUsesFileTargetOwnerAndNodeIdentity_Issue4845() + { + InsertIndexedFile("src/A/packages.lock.json", "dependency_lock", """ + { + "version": 1, + "dependencies": { + "net8.0": { + "Microsoft.Data.Sqlite": { + "type": "Direct", + "resolved": "8.0.0", + "dependencies": { + "Microsoft.Data.Sqlite.Core": "8.0.0" + } + }, + "Microsoft.Data.Sqlite.Core": { + "type": "Transitive", + "resolved": "8.0.0" + } + }, + "net8.0/win-x64": { + "Microsoft.Data.Sqlite": { + "type": "Direct", + "resolved": "8.0.0", + "dependencies": { + "Microsoft.Data.Sqlite.Core": "8.0.0" + } + }, + "Microsoft.Data.Sqlite.Core": { + "type": "Transitive", + "resolved": "8.0.0" + } + }, + "net8.0/linux-x64": { + "Microsoft.Data.Sqlite": { + "type": "Direct", + "resolved": "8.0.0", + "dependencies": { + "Microsoft.Data.Sqlite.Core": "8.0.0" + } + }, + "Microsoft.Data.Sqlite.Core": { + "type": "Transitive", + "resolved": "8.0.0" + } + } + } + } + """); + InsertIndexedFile("src/B/packages.lock.json", "dependency_lock", """ + { + "version": 1, + "dependencies": { + "net8.0": { + "Microsoft.Data.Sqlite": { + "type": "Direct", + "resolved": "9.0.0", + "dependencies": { + "Microsoft.Data.Sqlite.Core": "9.0.0" + } + }, + "Microsoft.Data.Sqlite.Core": { + "type": "Transitive", + "resolved": "9.0.0" + } + } + } + } + """); + + Assert.Empty(_reader.GetFileDependencies(lang: "dependency_lock")); + + var childAnalysis = _reader.AnalyzeSymbol( + "Microsoft.Data.Sqlite.Core", + limit: 10, + lang: "dependency_lock", + exact: true); + var childBundles = Assert.IsType>( + childAnalysis.CandidateBundles); + Assert.Equal(4, childBundles.Count); + + var aNet8Child = Assert.Single( + childBundles, + bundle => bundle.Definition.Path == "src/A/packages.lock.json" + && bundle.Definition.ContainerName == "net8.0"); + Assert.Equal(12, aNet8Child.Definition.Line); + var aNet8Caller = Assert.Single(aNet8Child.Callers); + Assert.Equal("src/A/packages.lock.json", aNet8Caller.Path); + Assert.Equal("Microsoft.Data.Sqlite", aNet8Caller.CallerName); + Assert.Equal(9, aNet8Caller.FirstLine); + + var aRidChild = Assert.Single( + childBundles, + bundle => bundle.Definition.Path == "src/A/packages.lock.json" + && bundle.Definition.ContainerName == "net8.0/win-x64"); + Assert.Equal(25, aRidChild.Definition.Line); + var aRidCaller = Assert.Single(aRidChild.Callers); + Assert.Equal("src/A/packages.lock.json", aRidCaller.Path); + Assert.Equal("Microsoft.Data.Sqlite", aRidCaller.CallerName); + Assert.Equal(22, aRidCaller.FirstLine); + + var aLinuxRidChild = Assert.Single( + childBundles, + bundle => bundle.Definition.Path == "src/A/packages.lock.json" + && bundle.Definition.ContainerName == "net8.0/linux-x64"); + Assert.Equal(38, aLinuxRidChild.Definition.Line); + var aLinuxRidCaller = Assert.Single(aLinuxRidChild.Callers); + Assert.Equal("src/A/packages.lock.json", aLinuxRidCaller.Path); + Assert.Equal("Microsoft.Data.Sqlite", aLinuxRidCaller.CallerName); + Assert.Equal(35, aLinuxRidCaller.FirstLine); + + var bNet8Child = Assert.Single( + childBundles, + bundle => bundle.Definition.Path == "src/B/packages.lock.json" + && bundle.Definition.ContainerName == "net8.0"); + Assert.Equal(12, bNet8Child.Definition.Line); + var bNet8Caller = Assert.Single(bNet8Child.Callers); + Assert.Equal("src/B/packages.lock.json", bNet8Caller.Path); + Assert.Equal("Microsoft.Data.Sqlite", bNet8Caller.CallerName); + Assert.Equal(9, bNet8Caller.FirstLine); + + Assert.All( + childBundles, + bundle => Assert.All( + bundle.Callers, + caller => Assert.Equal(bundle.Definition.Path, caller.Path))); + + var parentAnalysis = _reader.AnalyzeSymbol( + "Microsoft.Data.Sqlite", + limit: 10, + lang: "dependency_lock", + exact: true); + var parentBundles = Assert.IsType>( + parentAnalysis.CandidateBundles); + Assert.Equal(4, parentBundles.Count); + + AssertCallee(parentBundles, "src/A/packages.lock.json", "net8.0", definitionLine: 5, calleeLine: 9); + AssertCallee(parentBundles, "src/A/packages.lock.json", "net8.0/win-x64", definitionLine: 18, calleeLine: 22); + AssertCallee(parentBundles, "src/A/packages.lock.json", "net8.0/linux-x64", definitionLine: 31, calleeLine: 35); + AssertCallee(parentBundles, "src/B/packages.lock.json", "net8.0", definitionLine: 5, calleeLine: 9); + } + + [Fact] + public void AnalyzeSymbol_DependencyLockDoesNotResolveNpmDependenciesAcrossFiles_Issue4845() + { + InsertIndexedFile("src/npm-a/package-lock.json", "dependency_lock", """ + { + "lockfileVersion": 3, + "packages": { + "node_modules/left-pad": { + "version": "1.3.0", + "dependencies": { + "is-number": "7.0.0" + } + } + } + } + """); + InsertIndexedFile("src/npm-b/package-lock.json", "dependency_lock", """ + { + "lockfileVersion": 3, + "packages": { + "node_modules/is-number": { + "version": "7.0.0" + }, + "node_modules/right-pad": { + "version": "1.0.1", + "dependencies": { + "is-number": "7.0.0" + } + } + } + } + """); + + var analysis = _reader.AnalyzeSymbol( + "is-number", + limit: 10, + lang: "dependency_lock", + exact: true); + var bundle = Assert.Single( + Assert.IsType>( + analysis.CandidateBundles)); + + Assert.Equal("src/npm-b/package-lock.json", bundle.Definition.Path); + var caller = Assert.Single(bundle.Callers); + Assert.Equal("src/npm-b/package-lock.json", caller.Path); + Assert.Equal("right-pad", caller.CallerName); + Assert.Equal(10, caller.FirstLine); + Assert.Equal(1, caller.ReferenceCount); + } + + private static void AssertCallee( + List bundles, + string path, + string target, + int definitionLine, + int calleeLine) + { + var bundle = Assert.Single( + bundles, + candidate => candidate.Definition.Path == path + && candidate.Definition.ContainerName == target); + Assert.Equal(definitionLine, bundle.Definition.Line); + var callee = Assert.Single(bundle.Callees); + Assert.Equal(path, callee.Path); + Assert.Equal("Microsoft.Data.Sqlite.Core", callee.CalleeName); + Assert.Equal(calleeLine, callee.FirstLine); + } } diff --git a/tests/CodeIndex.Tests/DbReaderTests.cs b/tests/CodeIndex.Tests/DbReaderTests.cs index 65871f2ee..8cd0a8896 100644 --- a/tests/CodeIndex.Tests/DbReaderTests.cs +++ b/tests/CodeIndex.Tests/DbReaderTests.cs @@ -826,10 +826,10 @@ private void InsertIndexedFile(string path, string lang, string content, DateTim Content = normalized, }]); - var symbols = SymbolExtractor.Extract(fileId, lang, normalized); + var symbols = SymbolExtractor.Extract(fileId, lang, normalized, filePath: path); SymbolExtractor.ApplyFamilyScope(symbols, familyScopeKey ?? FileIndexer.DeriveFallbackFamilyScopeKey(path)); _writer.InsertSymbols(symbols); - _writer.InsertReferences(ReferenceExtractor.Extract(fileId, lang, normalized, symbols)); + _writer.InsertReferences(ReferenceExtractor.Extract(fileId, lang, normalized, symbols, path: path)); } private string ExplainQueryPlan(string sql) diff --git a/tests/CodeIndex.Tests/DependencyPackageExtractorTests.cs b/tests/CodeIndex.Tests/DependencyPackageExtractorTests.cs index f37b89a09..e7efd2c01 100644 --- a/tests/CodeIndex.Tests/DependencyPackageExtractorTests.cs +++ b/tests/CodeIndex.Tests/DependencyPackageExtractorTests.cs @@ -185,25 +185,31 @@ public void Extract_DependencyLock_EmitsResolvedSymbolsAndParentPackageReference var symbols = SymbolExtractor.Extract(10, "dependency_lock", content, filePath: "packages.lock.json"); var references = ReferenceExtractor.Extract(10, "dependency_lock", content, symbols, path: "packages.lock.json"); - Assert.Contains(symbols, symbol => - symbol.Kind == "package" - && symbol.SubKind == "lock_direct_dependency" - && symbol.Name == "Newtonsoft.Json" - && symbol.ContainerName == "net8.0" - && symbol.Signature?.Contains("role=direct", StringComparison.Ordinal) == true - && symbol.Signature?.Contains("resolved=13.0.3", StringComparison.Ordinal) == true - && symbol.Signature?.Contains("requested=[13.0.3, )", StringComparison.Ordinal) == true); - Assert.Contains(symbols, symbol => - symbol.Kind == "package" - && symbol.SubKind == "lock_transitive_dependency" - && symbol.Name == "Serilog" - && symbol.Signature?.Contains("role=transitive", StringComparison.Ordinal) == true); - Assert.Contains(references, reference => - reference.SymbolName == "Serilog" - && reference.ReferenceKind == "dependency" - && reference.ContainerKind == "package" - && reference.ContainerName == "Newtonsoft.Json" - && reference.Context.Contains("Serilog", StringComparison.Ordinal)); + var directPackage = Assert.Single(symbols, symbol => symbol.Name == "Newtonsoft.Json"); + Assert.Equal("package", directPackage.Kind); + Assert.Equal("lock_direct_dependency", directPackage.SubKind); + Assert.Equal("net8.0", directPackage.ContainerName); + Assert.Equal(5, directPackage.Line); + Assert.Equal(12, directPackage.EndLine); + var directSignature = Assert.IsType(directPackage.Signature); + Assert.Contains("role=direct", directSignature); + Assert.Contains("resolved=13.0.3", directSignature); + Assert.Contains("requested=[13.0.3, )", directSignature); + + var transitivePackage = Assert.Single(symbols, symbol => symbol.Name == "Serilog"); + Assert.Equal("lock_transitive_dependency", transitivePackage.SubKind); + Assert.Equal(13, transitivePackage.Line); + Assert.Equal(16, transitivePackage.EndLine); + Assert.Contains("role=transitive", Assert.IsType(transitivePackage.Signature)); + + var dependencyReference = Assert.Single(references); + Assert.Equal("Serilog", dependencyReference.SymbolName); + Assert.Equal("dependency", dependencyReference.ReferenceKind); + Assert.Equal("package", dependencyReference.ContainerKind); + Assert.Equal("Newtonsoft.Json", dependencyReference.ContainerName); + Assert.Equal("net8.0", dependencyReference.TargetQualifier); + Assert.Equal(10, dependencyReference.Line); + Assert.Contains("Serilog", dependencyReference.Context, StringComparison.Ordinal); Assert.DoesNotContain(references, reference => reference.SymbolName == "Newtonsoft.Json"); } diff --git a/tests/CodeIndex.Tests/IndexCommandRunnerUpdateTests.cs b/tests/CodeIndex.Tests/IndexCommandRunnerUpdateTests.cs index b89280428..e069b4a18 100644 --- a/tests/CodeIndex.Tests/IndexCommandRunnerUpdateTests.cs +++ b/tests/CodeIndex.Tests/IndexCommandRunnerUpdateTests.cs @@ -197,6 +197,77 @@ UPDATE symbol_references } } + [Fact] + public void Run_UpdateMode_NoOpRepairsPriorReferenceIdentityContract_Issue4845() + { + var projectRoot = CreateTempProject(); + try + { + File.WriteAllText( + Path.Combine(projectRoot, "Caller.cs"), + "public class Caller { public void Run() { Target.Execute(); } }\n"); + File.WriteAllText( + Path.Combine(projectRoot, "Target.cs"), + "public static class Target { public static void Execute() { } }\n"); + + var (initialExitCode, _) = RunAndCaptureJson([projectRoot, "--json"]); + Assert.Equal(CommandExitCodes.Success, initialExitCode); + + var dbPath = Path.Combine(projectRoot, ".cdidx", "codeindex.db"); + using (var connection = new SqliteConnection($"Data Source={dbPath}")) + { + connection.Open(); + using var command = connection.CreateCommand(); + command.CommandText = """ + DELETE FROM symbol_reference_candidates; + UPDATE symbol_references + SET target_symbol_id = NULL, + target_symbol_key = NULL, + resolution_state = NULL, + resolution_candidate_count = 0; + UPDATE codeindex_meta + SET value = @priorVersion + WHERE key = @key; + """; + command.Parameters.AddWithValue( + "@priorVersion", + (DbContext.ReferenceIdentityContractVersion - 1).ToString(CultureInfo.InvariantCulture)); + command.Parameters.AddWithValue("@key", DbContext.ReferenceIdentityContractVersionMetaKey); + command.ExecuteNonQuery(); + } + + var (updateExitCode, updateJson) = RunAndCaptureJson( + [projectRoot, "--files", "Caller.cs", "--json"]); + + Assert.Equal(CommandExitCodes.Success, updateExitCode); + Assert.Equal(0, updateJson.GetProperty("summary").GetProperty("updated").GetInt32()); + Assert.Equal(1, updateJson.GetProperty("summary").GetProperty("skipped").GetInt32()); + + using var verification = new SqliteConnection($"Data Source={dbPath}"); + verification.Open(); + using var markerCommand = verification.CreateCommand(); + markerCommand.CommandText = "SELECT value FROM codeindex_meta WHERE key = @key"; + markerCommand.Parameters.AddWithValue("@key", DbContext.ReferenceIdentityContractVersionMetaKey); + Assert.Equal( + DbContext.ReferenceIdentityContractVersion.ToString(CultureInfo.InvariantCulture), + Convert.ToString(markerCommand.ExecuteScalar(), CultureInfo.InvariantCulture)); + + using var candidateCommand = verification.CreateCommand(); + candidateCommand.CommandText = """ + SELECT COUNT(*) + FROM symbol_reference_candidates candidate + JOIN symbol_references reference ON reference.id = candidate.reference_id + WHERE reference.resolution_state IN ('resolved', 'resolved_group') + """; + Assert.True(Convert.ToInt32(candidateCommand.ExecuteScalar(), CultureInfo.InvariantCulture) > 0); + } + finally + { + SqliteConnection.ClearAllPools(); + DeleteDirectory(projectRoot); + } + } + [Fact] public void Run_UpdateMode_RefreshesMutualRecursionOncePerBatchIncludingDeleteOnly() { diff --git a/tests/CodeIndex.Tests/SymbolExtractorTests.cs b/tests/CodeIndex.Tests/SymbolExtractorTests.cs index 902b6160c..bb0644323 100644 --- a/tests/CodeIndex.Tests/SymbolExtractorTests.cs +++ b/tests/CodeIndex.Tests/SymbolExtractorTests.cs @@ -12620,6 +12620,8 @@ public void GetContractVersion_LanguageSpecificKinds_UseDedicatedVersions() Assert.True(SymbolExtractor.DockerfileContractVersion > SymbolExtractor.DefaultContractVersion); Assert.Equal(SymbolExtractor.MakefileContractVersion, SymbolExtractor.GetContractVersion("makefile")); Assert.True(SymbolExtractor.MakefileContractVersion > SymbolExtractor.DefaultContractVersion); + Assert.Equal(SymbolExtractor.DependencyLockContractVersion, SymbolExtractor.GetContractVersion("dependency_lock")); + Assert.True(SymbolExtractor.DependencyLockContractVersion > SymbolExtractor.ExpandedLanguageContractVersion); Assert.Equal(SymbolExtractor.StyleAndXamlContractVersion, SymbolExtractor.GetContractVersion("sass")); Assert.Equal(SymbolExtractor.StyleAndXamlContractVersion, SymbolExtractor.GetContractVersion("stylus")); Assert.True(SymbolExtractor.StyleAndXamlContractVersion > SymbolExtractor.DefaultContractVersion);