最新版本读取kafka数据sink到ES，如果kafka数据是嵌套json，字段值在es中为null
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 20/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- java
- Domain
- data-engineering
Research direction
Start by locating the Kafka-to-Elasticsearch sink and its nested JSON mapping path, then reproduce the issue with the payload shown in the report. Done means identifying why the lib object becomes null and preserving its nested fields in Elasticsearch, with a regression test if the relevant test area is found.
Written by the indexing model from the issue text.
Description
1、kafka中的数据格式为：
{
"distinct_id": "16c6a9eeabd205-08b9c3c13daa25-36f661a-269280-16c6a9eeabe1fd",
"lib": {
"$lib": "js",
"$lib_method": "code",
"$lib_version": "1.14.2"
}
}
2、sink到ES中的数据为
{
"_index": "flink2es",
"_type": "mytype_1",
"_id": "16c6a9eeabd205-08b9c3c13daa25-36f661a-269280-16c6a9eeabe1fd",
"_score": 1,
"_source": {
"lib": null,
"distinct_id": "16c6a9eeabd205-08b9c3c13daa25-36f661a-269280-16c6a9eeabe1fd"
}
}
请问嵌套格式的数据为什么是null？（这里指的：lib字段）
- Dominant language
- Java
- Stars
- 2k
- Forks
- 913
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from DTStack/flinkStreamSQL
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
DTStack/flinkStreamSQL#467 · 1 comment ·
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
DTStack/flinkStreamSQL#437 · 3 comments ·
-
Difficulty 3/5 1-2 days Newbie friendliness 38/100
DTStack/flinkStreamSQL#432 · 1 comment ·
-
Difficulty 5/5 Over a week Newbie friendliness 15/100
DTStack/flinkStreamSQL#431 ·
-
有无对sql 语法校验的方法 Open
Difficulty 5/5 Over a week Newbie friendliness 15/100
DTStack/flinkStreamSQL#430 · 1 comment ·
All issues in DTStack/flinkStreamSQL
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
elastic/gradle-plugins#157 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
cryptomator/hub#497 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
johanhaleby/occurrent#1120 ·