Skip to content

Fix recall computation for beta-filtered search in benchmark#1238

Open
magdalendobson wants to merge 23 commits into
mainfrom
users/magdalen/fix_recall_beta_search
Open

Fix recall computation for beta-filtered search in benchmark#1238
magdalendobson wants to merge 23 commits into
mainfrom
users/magdalen/fix_recall_beta_search

Conversation

@magdalendobson

@magdalendobson magdalendobson commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

It turns out that when we implemented the fix to recall computation for filtered search in #1069, the fix did not propagate to beta search. This happened because beta search used the same runner as regular knn search in search/knn.rs, so the wrong GroundTruthMode was passed in. This PR fixes that problem by including GroundTruthMode in SearchSteps so it can be passed in earlier in the stack.

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 78.26087% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.03%. Comparing base (ee7dbaa) to head (8c5f07e).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
diskann-benchmark/src/index/search/knn.rs 83.33% 3 Missing ⚠️
diskann-benchmark/src/index/benchmarks.rs 60.00% 2 Missing ⚠️

❌ Your patch status has failed because the patch coverage (78.26%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1238      +/-   ##
==========================================
- Coverage   91.01%   90.03%   -0.99%     
==========================================
  Files         501      504       +3     
  Lines       95572    95983     +411     
==========================================
- Hits        86988    86415     -573     
- Misses       8584     9568     +984     
Flag Coverage Δ
miri 90.03% <78.26%> (-0.99%) ⬇️
unittests 89.69% <78.26%> (-1.29%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
diskann-benchmark/src/index/inmem/spherical.rs 100.00% <ø> (ø)
diskann-benchmark/src/index/benchmarks.rs 66.66% <60.00%> (ø)
diskann-benchmark/src/index/search/knn.rs 78.84% <83.33%> (+1.06%) ⬆️

... and 47 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@magdalendobson magdalendobson marked this pull request as ready for review July 8, 2026 13:22
@magdalendobson magdalendobson requested review from a team and Copilot July 8, 2026 13:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes recall computation for beta-filtered (and other range-groundtruth) search benchmarks by ensuring the correct GroundTruthMode is threaded into the KNN runner earlier in the call stack, avoiding the previous hardcoded mode mismatch.

Changes:

  • Add groundtruth_mode: GroundTruthMode to SearchSteps (defaulting to Fixed) and pass it through Knn::search_all.
  • Update KNN runner implementations to accept a groundtruth_mode parameter instead of hardcoding Fixed/Flexible.
  • Set GroundTruthMode::Flexible at beta-filtered / multihop / inline-filter benchmark call sites that load range groundtruth.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
diskann-benchmark/src/index/search/knn.rs Threads GroundTruthMode via SearchSteps into the KNN search runner/aggregator.
diskann-benchmark/src/index/inmem/spherical.rs Marks beta-filtered/multihop/inline-filter in-memory phases as GroundTruthMode::Flexible.
diskann-benchmark/src/index/benchmarks.rs Marks beta-filtered/multihop/inline-filter benchmark phases as GroundTruthMode::Flexible.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants