apache/pinot

Choose which fields get flattened by complexType

Aberta

#10.549 aberto em 5 de abr. de 2023

 (1 comentário) (0 reação) (1 responsável)Java (1.234 forks)batch import
featurehelp wanted

Métricas do repositório

Stars
 (4.937 estrelas)
Métricas de merge de PR
 (Métricas PR pendentes)

Description

Say we have the following event structure:

{
	"...",
	"auditFields": {
		"a": "b",
		"c": "d"
	},
	"results": [
		{
			"name": "result1",
			"value": 33.2
		},
		{
			"name": "result2",
			"value": 45.6
		}
	]
}

We want to create the following columns:

results.name
results.value
auditFields

At the moment I don't think that's possible as auditFields will be flattened into:

audiFields.a
auditFields.c

By this line of code - https://github.com/apache/pinot/blob/master/pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/ComplexTypeTransformer.java#L176

Can we make the columns that get flattened configurable, like the fields to unnest can be configured?

Guia do colaborador