Fix recall computation for beta-filtered search in benchmark#1238
Fix recall computation for beta-filtered search in benchmark#1238magdalendobson wants to merge 23 commits into
Conversation
Codecov Report❌ Patch coverage is
❌ Your patch status has failed because the patch coverage (78.26%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #1238 +/- ##
==========================================
- Coverage 91.01% 90.03% -0.99%
==========================================
Files 501 504 +3
Lines 95572 95983 +411
==========================================
- Hits 86988 86415 -573
- Misses 8584 9568 +984
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
This PR fixes recall computation for beta-filtered (and other range-groundtruth) search benchmarks by ensuring the correct GroundTruthMode is threaded into the KNN runner earlier in the call stack, avoiding the previous hardcoded mode mismatch.
Changes:
- Add
groundtruth_mode: GroundTruthModetoSearchSteps(defaulting toFixed) and pass it throughKnn::search_all. - Update KNN runner implementations to accept a
groundtruth_modeparameter instead of hardcodingFixed/Flexible. - Set
GroundTruthMode::Flexibleat beta-filtered / multihop / inline-filter benchmark call sites that load range groundtruth.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| diskann-benchmark/src/index/search/knn.rs | Threads GroundTruthMode via SearchSteps into the KNN search runner/aggregator. |
| diskann-benchmark/src/index/inmem/spherical.rs | Marks beta-filtered/multihop/inline-filter in-memory phases as GroundTruthMode::Flexible. |
| diskann-benchmark/src/index/benchmarks.rs | Marks beta-filtered/multihop/inline-filter benchmark phases as GroundTruthMode::Flexible. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
It turns out that when we implemented the fix to recall computation for filtered search in #1069, the fix did not propagate to beta search. This happened because beta search used the same runner as regular knn search in
search/knn.rs, so the wrongGroundTruthModewas passed in. This PR fixes that problem by includingGroundTruthModeinSearchStepsso it can be passed in earlier in the stack.