apache/pinot

Choose which fields get flattened by complexType

Open

#10.549 aperta il 5 apr 2023

Vedi su GitHub
 (1 commento) (0 reazioni) (1 assegnatario)Java (1234 fork)batch import
featurehelp wanted

Metriche repository

Star
 (4937 star)
Metriche merge PR
 (Merge medio 6g 7h) (186 PR mergiate in 30 g)

Descrizione

Say we have the following event structure:

{
	"...",
	"auditFields": {
		"a": "b",
		"c": "d"
	},
	"results": [
		{
			"name": "result1",
			"value": 33.2
		},
		{
			"name": "result2",
			"value": 45.6
		}
	]
}

We want to create the following columns:

results.name
results.value
auditFields

At the moment I don't think that's possible as auditFields will be flattened into:

audiFields.a
auditFields.c

By this line of code - https://github.com/apache/pinot/blob/master/pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/ComplexTypeTransformer.java#L176

Can we make the columns that get flattened configurable, like the fields to unnest can be configured?

Guida contributor