[lake/iceberg] Reserved write-mode properties can be set and reset through their `iceberg.` key
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 82/100
Research direction
Start in applyPropertyChanges in fluss-lake/fluss-lake-iceberg-common/.../IcebergLakeCatalog.java, where the RESERVED_PROPERTIES check runs on the Fluss key before convertFlussPropertyKeyToIceberg strips the iceberg. prefix; the ResetOption branch needs the same check. Extend IcebergLakeCatalogTest.alterReservedTableProperties to also pass each reserved name with the iceberg. prefix. Done when prefixed set and reset calls fail with the existing messages, the bare-key cases still fail, and the Iceberg table is left unchanged.
Written by the indexing model from the issue text.
Description
Search before asking
- I searched the issues and found nothing similar.
Fluss version
1.0.0 (latest release). The same code is on main at 25306d8f1.
Please describe the bug
IcebergLakeCatalog reserves three Iceberg table properties: write.merge.mode,
write.update.mode and write.delete.mode (RESERVED_PROPERTIES). A Fluss table sets these to
merge-on-read when it creates a primary-key table, and alterTable is meant to refuse a change
to any of them.
The check runs on the Fluss key, before the iceberg. prefix is removed:
// fluss-lake/fluss-lake-iceberg-common/src/main/java/org/apache/fluss/lake/iceberg/IcebergLakeCatalog.java
// applyPropertyChanges
checkArgument(
!RESERVED_PROPERTIES.contains(option.getKey()), // "iceberg.write.merge.mode"
"Cannot set table property '%s'",
option.getKey());
updateProperties.set(
convertFlussPropertyKeyToIceberg(option.getKey()), // -> "write.merge.mode"
option.getValue());
A key arrives with the prefix, so iceberg.write.merge.mode passes the check, and
convertFlussPropertyKeyToIceberg turns it into write.merge.mode, which is then set. The bare
key write.merge.mode is rejected, but a bare key would map to fluss.write.merge.mode and could
never reach the Iceberg property. As a result, the check never blocks a change that would take
effect. ResetOption has the same problem, and a reset removes the property, so the table falls
back to Iceberg's default, copy-on-write.
IcebergLakeCatalogTest.alterReservedTableProperties passes each bare name from
RESERVED_PROPERTIES to alterTable, so it does not cover the prefixed key.
Steps to reproduce
We ran these against apache/fluss:1.0.0 and apache/fluss-gateway:1.0.0, with Iceberg 1.10.1 on
a REST catalog and MinIO.
-
Create a primary-key table with
table.datalake.enabled = trueand
table.datalake.format = iceberg. The Iceberg table holdswrite.merge.mode,
write.update.modeandwrite.delete.mode, allmerge-on-read. -
Set
iceberg.write.merge.modetocopy-on-write. Through the gateway:PATCH /v1/clusters/iceberg/databases/q4/tables/t4 {"changes":[{"kind":"set_config","key":"iceberg.write.merge.mode","value":"copy-on-write"}]}The request returns 204. The Iceberg table property
write.merge.modebecomes
copy-on-write. The Flink SQL form,
ALTER TABLE t4 SET ('iceberg.write.merge.mode' = 'copy-on-write'), reaches the same
alterTablecall; we did not run it. -
Reset it with
{"kind":"reset_config","key":"iceberg.write.merge.mode"}. The request returns
204, andwrite.merge.modeis removed from the Iceberg table. -
For comparison, set the bare key
write.merge.mode. It fails with
Cannot set table property 'write.merge.mode'.
iceberg.write.update.mode and iceberg.write.delete.mode behave the same way.
Expected
Setting or resetting iceberg.write.merge.mode, iceberg.write.update.mode or
iceberg.write.delete.mode fails with Cannot set table property or
Cannot reset table property, and the Iceberg table is unchanged.
Solution
Check both the key as given and the Iceberg key it converts to. Checking only the converted key
would stop rejecting the bare write.merge.mode, which the existing test expects to fail.
String icebergKey = convertFlussPropertyKeyToIceberg(option.getKey());
checkArgument(
!isReservedProperty(option.getKey(), icebergKey),
"Cannot set table property '%s'",
option.getKey());
updateProperties.set(icebergKey, option.getValue());
private static boolean isReservedProperty(String flussKey, String icebergKey) {
return RESERVED_PROPERTIES.contains(flussKey) || RESERVED_PROPERTIES.contains(icebergKey);
}
ResetOption takes the same check. Extend alterReservedTableProperties to pass each reserved
name both bare and with the iceberg. prefix, and expect both the set and the reset to fail.
Are you willing to submit a PR?
- I'm willing to submit a PR!
- Dominant language
- Java
- Stars
- 2.2k
- Forks
- 656
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 108
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/fluss
-
[CI] NettyClientTest.testServerDisconnection may fail with DisconnectExceptionPossibly taken @yangshangqing95 claimed this 1 day ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
Maintainers usually reply within 1 day
-
[common] Decimal#hashCode is inconsistent with Decimal#equalsPossibly taken @firasbouzazi claimed this 15 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/fluss#4538 · 1 comment ·
Maintainers usually reply within 1 day
-
[security] PlainSaslServerProvider conflicts with Hadoop SASL provider causing authentication failurePossibly taken @xinzhang-hello claimed this 20 days ago. Open
Difficulty 1/5 Under an hour Newbie friendliness 88/100
Maintainers usually reply within 1 day
-
[server] KV snapshot scheduling stops permanently after a transient snapshot-init failurePossibly taken @zhang-arvin claimed this 20 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
apache/fluss#4489 · 1 comment ·
Maintainers usually reply within 1 day
-
[docs] Document advertised-resource and ACL principal ignore-case optionsPossibly taken @EchoLee5 claimed this 32 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Maintainers usually reply within 1 day
Similar issues
-
enhancement Improvement
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
zowe/zowe-client-java-sdk#647 ·
-
NullPointerException when a `DeserializationProblemHandler` returns `null` for `byte[]` (single value unwrapped) and `char[]` elementsPossibly taken @Marti2203 claimed this today. Open
Difficulty 2/5 Half a day Newbie friendliness 88/100
FasterXML/jackson-databind#6300 ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
MustacheCase/zanadir#254 ·
Maintainers usually reply within 1 day
-
content good first issue
Difficulty 1/5 Under an hour Newbie friendliness 95/100
cpinitiative/usaco-guide#6699 ·
Maintainers usually reply within 1 day
-
BC remittance upload: a claim MSP refused (S03) is saved with a blank status instead of refused (F)Possibly taken A pull request linked to this issue is open or already merged. Opentype: bug
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
carlos-emr/carlos#4933 ·
Maintainers usually reply within 1 day