apache/pinot

Choose which fields get flattened by complexType

Open

#10,549 opened on 2023年4月5日

GitHub で見る
 (1 comment) (0 reactions) (1 assignee)Java (1,234 forks)batch import
featurehelp wanted

Repository metrics

Stars
 (4,937 stars)
PR merge metrics
 (平均マージ 6d 7h) (30d で 186 merged PRs)

説明

Say we have the following event structure:

{
	"...",
	"auditFields": {
		"a": "b",
		"c": "d"
	},
	"results": [
		{
			"name": "result1",
			"value": 33.2
		},
		{
			"name": "result2",
			"value": 45.6
		}
	]
}

We want to create the following columns:

results.name
results.value
auditFields

At the moment I don't think that's possible as auditFields will be flattened into:

audiFields.a
auditFields.c

By this line of code - https://github.com/apache/pinot/blob/master/pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/ComplexTypeTransformer.java#L176

Can we make the columns that get flattened configurable, like the fields to unnest can be configured?

コントリビューターガイド