feat(theta): add jaccard similarity#142
Conversation
|
We also miss |
tisonkun
left a comment
There was a problem hiding this comment.
We must not have a unit struct ThetaJaccardSimilarity for creating JaccardSimilarity.
Let me consider the API and review the helper functions location.
Typically, here are a few API issues that should be resolved:
- Should have
JaccardSimilarity::between/JaccardSimilarity::between_with_seed, rather thanThetaJaccardSimilarity::jaccard/ThetaJaccardSimilarity::jaccard_with_seed. JaccardSimilarityshould expose its internal fields with getter, not pub fields, because the fields are immutatble.
…/datasketches-rust into feat/theta-jaccard-similarity
|
Addressed the API feedback from @tisonkun:
The tests have been updated to use the new API. |
…to feat/theta-jaccard-similarity
|
@ZENOTME I have two questions:
|
Java has a specific update implementation, AlphaSketch, where theta can be reduced without immediately rebuilding the internal cache. This means some cached entries may become invalid under the new theta but remain in the cache until a later rebuild. Therefore, getRetainedEntries(true) returns the valid entries used for estimation, while getRetainedEntries(false) returns the raw internal cache count, including possibly invalid entries.
We don't have ThetaUnion so far. I think it maybe better to implement it first. I send a PR for it. #145 cc @tisonkun @hawkingrei |
Summary
ThetaJaccardSimilarityfor Theta sketches.theta_jaccard_similarity_test.cppcases.Motivation
This closes one small P0 parity gap with
apache/datasketches-cpp: Rust had Theta sketching and intersection support, but no Jaccard similarity API.Implementation Notes
ThetaIntersectionfor the intersection side.ThetaUnionAPI in this PR.Tests
cargo check -p datasketches --features thetacargo test -p datasketches --features theta --test theta_jaccard_similarity_testcargo test -p datasketches --features theta --lib --test theta_intersection_test --test theta_jaccard_similarity_test --test theta_sketch_testFull
cargo test -p datasketches --features thetacurrently reaches unrelated failures intheta_serialization_testbecause localdatasketches/tests/serialization_test_data/...files are missing.