[lake/paimon] Support custom Paimon path#3523
Conversation
There was a problem hiding this comment.
Pull request overview
Enables configuring a custom Paimon table path via table custom properties, and adds server-side validation to prevent changing paimon.path once a table is datalake-enabled (plus regression coverage for the new behavior).
Changes:
- Allow
paimon.pathto be set by removingCoreOptions.PATHfrom Paimon’s “unsettable options” list. - Extend alter-table property validation to consider custom property keys and reject
paimon.pathchanges when datalake is enabled. - Add IT coverage for unsettable Paimon options and
paimon.pathalter validation.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| fluss-server/src/main/java/org/apache/fluss/server/utils/TableDescriptorValidation.java | Adds validation for altering paimon.path via custom properties when datalake is enabled. |
| fluss-server/src/main/java/org/apache/fluss/server/coordinator/MetadataManager.java | Passes custom-keys-to-change into alter validation during table property updates. |
| fluss-lake/fluss-lake-paimon/src/main/java/org/apache/fluss/lake/paimon/utils/PaimonConversions.java | Makes Paimon path user-settable by removing it from the unsettable-options list. |
| fluss-lake/fluss-lake-paimon/src/test/java/org/apache/fluss/lake/paimon/LakeEnabledTableCreateITCase.java | Adds regression tests for unsettable options and paimon.path alter restrictions. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
luoyuxia
left a comment
There was a problem hiding this comment.
@wzx140 Thanks for the pr.
IIUC, paimon.path is intended to customize the physical location only when the Paimon table is created. Allowing it to be changed after the table has already been created looks unsafe to me, because changing this option does not migrate the existing Paimon metadata/data from the old path to the new one.
The current check only rejects changing paimon.path while table.datalake.enabled is true. However, a table can be enabled once, then disabled later, while the Paimon table still exists.
In that state, altering paimon.path would still be allowed. Could we make paimon.path a create-time-only option, or reject changing it once the lake table has ever been created?
Thanks for the review! Good catch. I’ll update the implementation accordingly. |
5641661 to
f07558d
Compare
|
@luoyuxia I’ve updated the implementation to reject changes to paimon.path once the lake table has been created. Could you please review it again? |
Summary
Test Plan