[Feature] Support user-specified characterEncoding in JDBC Catalog URL
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 78/100
Research direction
Start in fe/fe-core/src/main/java/org/apache/doris/catalog/JdbcResource.java and inspect checkAndSetJdbcParam, especially the characterEncoding call. Verify the JDBC URL behavior when characterEncoding is already present with a non-UTF-8 value and when it is absent. Done means user-specified values are preserved without duplication and the UTF-8 default is added only when missing.
Written by the indexing model from the issue text.
Description
Is your feature request related to a problem?
Doris currently hardcodes characterEncoding=utf-8 for MySQL/OceanBase JDBC catalogs. Users connecting to legacy MySQL tables stored in latin1 (or gbk, etc.) cannot read their data correctly, because Doris always forces UTF-8 and even produces a duplicated parameter when the user specifies their own value:
jdbc:mysql://host:port/db?characterEncoding=latin1
becomes
jdbc:mysql://host:port/db?characterEncoding=latin1&characterEncoding=utf-8
Describe the solution you'd like
Respect the characterEncoding value explicitly provided by the user in jdbc_url. Concretely, in fe/fe-core/src/main/java/org/apache/doris/catalog/JdbcResource.java, the injection logic:
newJdbcUrl = checkAndSetJdbcParam(dbType, newJdbcUrl, "characterEncoding", "utf-8");
should skip injection when characterEncoding is already present (with any value), and only append characterEncoding=utf-8 as the default when it is absent.
Describe alternatives you've considered
- Keep forcing UTF-8 but at least replace (not append) any user-specified value — however this would still not serve latin1 users.
- Add a new catalog property (e.g.
jdbc_character_encoding) to control it — heavier, and redundant sincejdbc_urlalready carries the parameter.
Additional context
- Root cause:
checkAndSetJdbcParamchecksjdbcUrl.contains("characterEncoding=utf-8")literally, so any other value (latin1/gbk/...) triggers appending a duplicate. - Legacy latin1 tables are common in older systems; this limitation blocks migration to Doris's JDBC federated queries.
- Current behavior exists on both branch-2.x and master.
- Dominant language
- Java
- Stars
- 16k
- Forks
- 4k
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 569
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/doris
-
Difficulty 1/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
[Bug] Empty array/map field in Hive text is read as NULL (or errors) instead of an empty collection Open
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
infinispan/infinispan#18150 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
untriaged
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
opensearch-project/k-NN#3597 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100