[Feature] Support user-specified characterEncoding in JDBC Catalog URL

Open Beginner friendly
#66,993 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Feature
Clarity
Clearly specified
Activity status
Active
Tech stack
java, mysql
Domain
databases

Research direction

Start in fe/fe-core/src/main/java/org/apache/doris/catalog/JdbcResource.java and inspect checkAndSetJdbcParam, especially the characterEncoding call. Verify the JDBC URL behavior when characterEncoding is already present with a non-UTF-8 value and when it is absent. Done means user-specified values are preserved without duplication and the UTF-8 default is added only when missing.

Written by the indexing model from the issue text.

Description

Is your feature request related to a problem?

Doris currently hardcodes characterEncoding=utf-8 for MySQL/OceanBase JDBC catalogs. Users connecting to legacy MySQL tables stored in latin1 (or gbk, etc.) cannot read their data correctly, because Doris always forces UTF-8 and even produces a duplicated parameter when the user specifies their own value:

jdbc:mysql://host:port/db?characterEncoding=latin1

becomes

jdbc:mysql://host:port/db?characterEncoding=latin1&characterEncoding=utf-8

Describe the solution you'd like

Respect the characterEncoding value explicitly provided by the user in jdbc_url. Concretely, in fe/fe-core/src/main/java/org/apache/doris/catalog/JdbcResource.java, the injection logic:

newJdbcUrl = checkAndSetJdbcParam(dbType, newJdbcUrl, "characterEncoding", "utf-8");

should skip injection when characterEncoding is already present (with any value), and only append characterEncoding=utf-8 as the default when it is absent.

Describe alternatives you've considered

  1. Keep forcing UTF-8 but at least replace (not append) any user-specified value — however this would still not serve latin1 users.
  2. Add a new catalog property (e.g. jdbc_character_encoding) to control it — heavier, and redundant since jdbc_url already carries the parameter.

Additional context

  • Root cause: checkAndSetJdbcParam checks jdbcUrl.contains("characterEncoding=utf-8") literally, so any other value (latin1/gbk/...) triggers appending a duplicate.
  • Legacy latin1 tables are common in older systems; this limitation blocks migration to Doris's JDBC federated queries.
  • Current behavior exists on both branch-2.x and master.
Dominant language
Java
Stars
16k
Forks
4k
Avg merge
2d 12h
Merged PRs (30d)
569

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/doris

All issues in apache/doris

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.