Import historical RFCs#1239
Open
arrayka wants to merge 1 commit into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR imports RFCs 00100 through 00109 into rfcs/ to preserve historical design context alongside the DiskANN codebase, including some RFCs that carry accompanying reference implementations/bench harnesses.
Changes:
- Add 10 historical RFC markdown documents covering topics like distance functions, testing, diverse search, quantization layouts, and unsafe usage guidelines.
- Import RFC-associated reference Rust crates (e.g.,
00105-label-format-parser,00101-rayon-util,00100-distance-functions) including benches and lockfiles. - Add a label data format RFC describing JSONL metadata/query formats for filtered search datasets.
Reviewed changes
Copilot reviewed 29 out of 31 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| rfcs/00109-unsafe-rust.md | Documents guidance/criteria for introducing unsafe Rust in DiskANN. |
| rfcs/00108-scalar-quantization-disk-layout.md | Specifies proposed on-disk layouts + schema options for scalar quantization. |
| rfcs/00107-rust-cpp-dynamic-inmem-parity.md | Defines a runbook-based process/metrics for Rust vs C++ dynamic in-mem parity. |
| rfcs/00106-quantized-range-query-correction.md | Proposes statistical correction for range queries under quantization + reranking. |
| rfcs/00105-label-format-parser/src/query_filter.pest | Pest grammar for MongoDB-style query filter JSON. |
| rfcs/00105-label-format-parser/src/pest_parser.rs | Pest-based parser converting JSON-string filters into an AST (has compile issues). |
| rfcs/00105-label-format-parser/src/parser.rs | “Original” JSON-based parser implementation for the same AST. |
| rfcs/00105-label-format-parser/src/optimized.rs | Alternate implementation focusing on evaluation/path lookup changes. |
| rfcs/00105-label-format-parser/src/main.rs | Minimal binary entrypoint for the RFC crate. |
| rfcs/00105-label-format-parser/src/lib.rs | Library module wiring and re-exports for the RFC parser crate. |
| rfcs/00105-label-format-parser/src/focused_opt.rs | Alternate evaluator focusing on reducing splits for dot-path lookup. |
| rfcs/00105-label-format-parser/src/evaluator.rs | Baseline evaluator for AST against JSON labels (has numeric-compare bug). |
| rfcs/00105-label-format-parser/src/ast.rs | Defines QueryExpr + CompareOp AST for filters. |
| rfcs/00105-label-format-parser/README.md | Usage/bench instructions and summary for the RFC label filter parser crate. |
| rfcs/00105-label-format-parser/Cargo.toml | Manifest for the RFC label filter parser crate (deps/benches). |
| rfcs/00105-label-format-parser/Cargo.lock | Lockfile for the RFC label filter parser crate. |
| rfcs/00105-label-format-parser/benches/main.rs | Criterion bench entrypoint for the RFC crate. |
| rfcs/00105-label-format-parser/benches/benchmarks/parser_bench.rs | Parser benchmarks comparing implementations and dynamic cases. |
| rfcs/00105-label-format-parser/benches/benchmarks/mod.rs | Bench module wiring. |
| rfcs/00105-label-format-parser/benches/benchmarks/evaluator_bench.rs | Evaluator benchmarks comparing implementations. |
| rfcs/00105-label-data-format-rfc.md | Defines JSONL label/query/ground-truth formats for filtered search datasets. |
| rfcs/00104-diverse-search.md | Design doc for adding diverse similarity search in DiskANN Rust. |
| rfcs/00103-diskann-core-testing.md | RFC describing pedantic providers + cached generated test results approach. |
| rfcs/00102-SSD-index-convergence.md | Proposal to converge SSD index onto diskann_async.rs architecture. |
| rfcs/00101-rayon-util/src/lib.rs | Reference code/tests illustrating Rayon deadlock scenarios + mitigation ideas. |
| rfcs/00101-rayon-util/Cargo.toml | Manifest for the rayon-util RFC crate. |
| rfcs/00101-rayon-util.md | RFC describing a scoped Rayon threadpool API and _in_pool iterator methods. |
| rfcs/00100-distance-functions/src/lib.rs | Reference crate implementing proposed distance-function traits/types. |
| rfcs/00100-distance-functions/Cargo.toml | Manifest for the distance RFC crate. |
| rfcs/00100-distance-functions/Cargo.lock | Lockfile for the distance RFC crate. |
| rfcs/00100-distance-functions.md | RFC describing proposed Rust distance-function API and rationale. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1239 +/- ##
==========================================
- Coverage 91.04% 90.03% -1.01%
==========================================
Files 504 504
Lines 95966 95966
==========================================
- Hits 87370 86401 -969
- Misses 8596 9565 +969
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Imports a collection of 10 previously-authored RFCs into
rfcs/(numbered00100through00109) to preserve the record of design decisions made earlier in DiskANN's development.Keeping these RFCs alongside the code provides context for the decisions that shaped this repository.