apache/pinot

Choose which fields get flattened by complexType

Open

#10 549 ouverte le 5 avr. 2023

Voir sur GitHub
 (1 commentaire) (0 réactions) (1 assigné)Java (1 234 forks)batch import
featurehelp wanted

Métriques du dépôt

Stars
 (4 937 stars)
Métriques de merge PR
 (Merge moyen 6j 7h) (186 PRs mergées en 30 j)

Description

Say we have the following event structure:

{
	"...",
	"auditFields": {
		"a": "b",
		"c": "d"
	},
	"results": [
		{
			"name": "result1",
			"value": 33.2
		},
		{
			"name": "result2",
			"value": 45.6
		}
	]
}

We want to create the following columns:

results.name
results.value
auditFields

At the moment I don't think that's possible as auditFields will be flattened into:

audiFields.a
auditFields.c

By this line of code - https://github.com/apache/pinot/blob/master/pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/ComplexTypeTransformer.java#L176

Can we make the columns that get flattened configurable, like the fields to unnest can be configured?

Guide contributeur