[FIP-40] merge fluss-rust#3401
Open
fresh-borzoni wants to merge 322 commits into
Open
Conversation
…CompactedRow::from_bytes(), these fields can be skipped as current rust client implementation does not check schema and already passes row_type (apache#190)
* [cpp] Limit scan * add test
* Add client metrics and prometheus example * improvements * address reviews
…r client so start/stop share one runtime (apache#624)
- Update fluss_api.proto with all 1.x message types (ACLs, KV snapshots,
producer offsets, cluster config, rebalance, server tags, etc.)
- Add optional fields: rack, remote_data_dir, leader_epoch, agg_mode, etc.
- Register 24 new ApiKey variants (1023-1064) in api_key.rs
- Update build.rs prost bytes config for new proto fields
- Add None defaults in convert.rs and partition.rs
- Update pre-existing message wrappers that reference renamed proto fields /
ApiKey variants so the crate still builds:
* create_partition.rs: ignore_if_exists -> ignore_if_not_exists
* get_latest_lake_snapshot.rs: ApiKey::GetLatestLakeSnapshot -> GetLakeSnapshot
* list_databases.rs: populate new include_summary field
* lookup.rs: PbLookupReqForBucket.key -> keys; new LookupRequest fields
Co-authored-by: warmbupt <warmbupt@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix: allow unknown bootstrap server type * test: cover tablet bootstrap server endpoint * fix: align unknown server type mapping with Java client * docs: align cpp/python binding docs with Unknown server type Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Add 9 RPC message wrapper types: - alter_database, alter_table (DDL operations) - get_table_stats (table statistics) - list_database_summaries (database listing with summaries) - create_acls, list_acls, drop_acls (ACL management) - describe_cluster_configs, alter_cluster_configs (cluster configuration) Each wrapper follows the standard pattern: a request struct wrapping the proto-generated type, implementing RequestBody (tying to ApiKey and ResponseBody), WriteType, and ReadType. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…apache#611) * [c++] Add MAP/ROW types, unify complex types under Value handle * address review * add int family tests for nested cases
…ases in write path (apache#635) The write path (sender, accumulator, bucket_assigner) and RPC messages (produce_log, put_kv) used raw i64/i32 for table/partition/bucket IDs instead of the semantic type aliases defined in lib.rs. This makes the code inconsistent with newer modules that already use the aliases. Changes: - cluster.rs: get_table_id() returns Option<TableId>, get_table_id_by_path() returns &HashMap<TablePath, TableId>, get_bucket_locations() takes TableId - produce_log.rs, put_kv.rs: new() takes TableId - sender.rs: build_write_request/send_and_handle_response/handle_write_response take TableId; BucketResponse trait returns BucketId - bucket_assigner.rs: BucketAssigner trait uses BucketId throughout - accumulator.rs: test helper uses BucketId - remote_log.rs: test helper uses TableId/BucketId Co-authored-by: warmbupt <warmbupt@gmail.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* [rust] Add RPC message wrappers for extended operations Add message wrappers for the remaining 1.x RPC APIs: - KV snapshot lifecycle: acquire/release/drop lease, list, metadata, latest snapshots, lake snapshot - Server management: add/remove server tag, rebalance + progress + cancel, get cluster health, list remote log manifests - Producer offsets: register/get/delete - ScanKv (API 1061): full KV-table bucket scan request/response * Add GoalType and ServerTag domain enums for RPC wrappers Replace raw i32 with proper Rust enums matching the Java Fluss definitions: GoalType (ReplicaDistribution/LeaderDistribution/RackAware) and ServerTag (PermanentOffline/TemporaryOffline). Addresses reviewer feedback on PR apache#630. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Apply TableId/PartitionId/BucketId type aliases to RPC messages Use the i64/i32 type aliases from lib.rs instead of raw primitives in the new RPC message wrappers introduced by this PR. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: warmbupt <warmbupt@gmail.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* [elixir] feat: Add get_table_info admin call * Include schema in table_info and add get_table_schema admin call
* [rust] Add Fluss 1.x protocol support to the admin client Add 27 new admin methods to FlussAdmin: - Database/table extensions: list_database_summaries, alter_database, alter_table, get_table_stats - KV snapshot operations: get_latest_kv_snapshots, get_kv_snapshot_metadata, create_kv_snapshot_lease, get_lake_snapshot - ACL management: create_acls, list_acls, drop_acls - Cluster configuration: describe_cluster_configs, alter_cluster_configs - Server management: add_server_tag, remove_server_tag, rebalance, list_rebalance_progress, cancel_rebalance - Producer offsets: register_producer_offsets, get_producer_offsets, delete_producer_offsets - Monitoring: get_cluster_health, list_remote_log_manifests - KV snapshots: list_kv_snapshots, release_kv_snapshot_lease, drop_kv_snapshot_lease * Address reviewer feedback: add domain enums, type aliases, expose readable - Add ClusterHealthStatus enum (Green/Yellow/Red/Unknown) to cluster_health.rs - Add RebalanceStatus enum (NotStarted..Timeout) to rebalance.rs - Replace raw i32/i64 with BucketId/TableId/PartitionId aliases in kv_snapshot.rs, lake_snapshot.rs, rebalance.rs, producer_offsets.rs - Expose readable parameter in admin.get_lake_snapshot() instead of hardcoding None Addresses reviewer feedback on PR apache#631. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Apply type aliases to remaining metadata types (table_stats, kv_snapshot_lease) Extends the BucketId/TableId/PartitionId alias consistency fix to table_stats.rs (BucketStatsRequest, BucketStats) and kv_snapshot_lease.rs (KvSnapshotLeaseForBucket, KvSnapshotLeaseForTable). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update admin.rs to use GoalType/ServerTag enums after rebase Now that pr/3 provides GoalType and ServerTag enums in the RPC wrappers, update the admin client methods to use them in their public signatures too. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Apply TableId/PartitionId/BucketId type aliases to admin methods Update the new admin method signatures introduced by this PR to use the i64/i32 type aliases from lib.rs instead of raw primitives, matching the underlying RPC message wrappers. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Address reviewer feedback round 2 - Add `RegisterProducerResult` enum mirroring Java's `RegisterResult` (Created=0, AlreadyExists=1); `register_producer_offsets` now returns `Option<RegisterProducerResult>` instead of raw `Option<i32>`. - Expose `comment: Option<&str>` on `alter_database` and the `AlterDatabaseRequest` wrapper (was hardcoded to `None`). - Replace fully-qualified paths with imports: `BucketStatsRequest` in admin.rs, `PbDatabaseSummary` in metadata/database.rs, and `AlterConfig` self-reference in metadata/table_change.rs. - Rename `create_acls(acl)` to `create_acls(acls)` and `drop_acls(acl_filter)` to `drop_acls(acl_filters)` for plural consistency with the `Vec` argument type. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: warmbupt <warmbupt@gmail.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
|
@fresh-borzoni , please rebase this pr. |
loserwang1024
approved these changes
Jun 29, 2026
loserwang1024
left a comment
loserwang1024
left a comment
Contributor
There was a problem hiding this comment.
LGTM, please rebase this PR
Contributor
|
Thanks for the pr. |
Contributor
|
I'll have a review in next few days |
fresh-borzoni
force-pushed
the
fip-40-merge-rust
branch
from
c232100 to
af7dd10
Compare
Member
Author
|
@luoyuxia @loserwang1024 @beryllw I brought a new sub-tree here, PTAL |
beryllw
reviewed
Jul 3, 2026
fresh-borzoni
force-pushed
the
fip-40-merge-rust
branch
from
af7dd10 to
b6c71c6
Compare
fresh-borzoni
force-pushed
the
fip-40-merge-rust
branch
from
66f94bc to
f81e98b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
closes #3407
Brings the Rust, Python, C++, Elixir clients (apache/fluss-rust) into this repo under fluss-rust/, per FIP-40 (https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=421957144)