apache/pinot

Choose which fields get flattened by complexType

Aperta

#10.549 aperta il 5 apr 2023

 (1 commento) (0 reazioni) (1 assegnatario)Java (1234 fork)batch import
featurehelp wanted

Metriche repository

Star
 (4937 stelle)
Metriche merge PR
 (Metriche PR in attesa)

Descrizione

Say we have the following event structure:

{
	"...",
	"auditFields": {
		"a": "b",
		"c": "d"
	},
	"results": [
		{
			"name": "result1",
			"value": 33.2
		},
		{
			"name": "result2",
			"value": 45.6
		}
	]
}

We want to create the following columns:

results.name
results.value
auditFields

At the moment I don't think that's possible as auditFields will be flattened into:

audiFields.a
auditFields.c

By this line of code - https://github.com/apache/pinot/blob/master/pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/ComplexTypeTransformer.java#L176

Can we make the columns that get flattened configurable, like the fields to unnest can be configured?

Guida contributor