Kotlin DataFrame plugin: `unfold` and `toDataFrame` with properties give an empty schema

Abierto
  • ¿Algo no está bien?
#2,115 1 comentario 0 reacciones 2 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

@koperagen ya está trabajando en esto.

Desde el 29/9/2026.

Evaluación

Este issue todavía no se ha evaluado.

Descripción

When DataFrame.unfold or Iterable.toDataFrame get a list of properties, the plugin derives an empty
schema for the result. At runtime the result has columns, and none of them can be accessed through the
generated accessors.

Reproduce
data class Name(val firstName: String, val lastName: String)

data class Score(val subject: String, val value: Int)

data class Student(val name: Name, val age: Int, val scores: List<Score>)

val df = dataFrameOf(
    "id" to listOf(1, 2),
    "student" to listOf(
        Student(Name("Alice", "Cooper"), 15, listOf(Score("math", 4), Score("biology", 3))),
        Student(Name("Bob", "Marley"), 20, listOf(Score("music", 5))),
    ),
    "year" to listOf(2021, 2022),
)

val unfolded = df.unfold(Student::name, Student::age) { student }
unfolded.compileTimeSchema() // empty
unfolded.schema()            // id: Int, student: { name: Name, age: Int }, year: Int

val fromList = listOf(Student(Name("Alice", "Cooper"), 15, emptyList())).toDataFrame(Student::age)
fromList.compileTimeSchema() // empty
fromList.schema()            // age: Int

Without properties the schemas match: df.unfold(maxDepth = 1) { student } and toDataFrame() give the same
schema at compile time and at runtime. The tests for both cases are in Kotlin/dataframe,
samples/src/test/kotlin/org/jetbrains/kotlinx/dataframe/plugin/UnfoldCompileTimeSchemaTests.kt
(issue 1, case 5): unfold with roots … and toDataFrame with properties …, both @Ignored.

What we see in the source
  • DataFrameUnfold (kotlin-dataframe.k2/.../impl/api/unfold.kt) declares val Arguments.properties by ignore(),
    but the parameter of DataFrame.unfold is called roots.
  • ToDataFrame (.../impl/api/toDataFrame.kt) does not declare the props parameter of
    Iterable.toDataFrame(vararg props: KProperty<*>, maxDepth: Int) at all.

We think the interpreters cannot match the vararg argument, so the interpretation fails and the plugin falls
back to an empty schema. We did not confirm this in the plugin itself.

Suggested fix

Read the property references the way Exclude1 and Preserve1 already do, and keep only the columns for
those properties:

class DataFrameUnfold : AbstractSchemaModificationInterpreter() {
    val Arguments.receiver: PluginDataFrameSchema by dataFrame()
    val Arguments.roots: FirVarargArgumentsExpression by arg(lens = Interpreter.Id)
    val Arguments.maxDepth: Int by arg(defaultValue = Present(0))
    val Arguments.columns: ColumnsResolver by arg()

    override fun Arguments.interpret(): PluginDataFrameSchema {
        val rootNames = roots.arguments
            .filterIsInstance<FirCallableReferenceAccess>()
            .map { it.calleeReference.name.identifier }
        return receiver.modify(impliedColumnsResolver = columns) {
            replace { columns }.with {
                val column = it.asSimpleColumn() as? SimpleDataColumn
                if (column == null || !column.type.coneType.canBeUnfolded(session)) {
                    it
                } else {
                    val unfolded = toDataFrame(maxDepth, column.type.coneType, TraverseConfiguration.EMPTY).columns()
                    val kept = if (rootNames.isEmpty()) unfolded else unfolded.filter { col -> col.name in rootNames }
                    SimpleColumnGroup(it.name(), kept).asDataColumn()
                }
            }
        }
    }
}

This is a sketch: we did not compile it. Two things to check:

  • how a missing vararg argument is passed (the default is no roots);
  • that the column name is the property name. It is not for @ColumnName properties and for getters like
    getX(), so it is safer to filter by the property symbol than by the name.

ToDataFrame needs the same change for props.

Changes to make together with Kotlin/dataframe #2114

When the library fixes land, the plugin has to change in the same release, or the compile-time schema will
differ from the runtime one:

  • roots on a column of simple values (issue 1, case 1): when roots are given, do not return the column
    unchanged just because canBeUnfolded is false for its type.
  • a column of lists of objects (issue 1, case 3): derive a SimpleFrameColumn from the element type
    instead of a column group with size.
  • maps (issue 1, case 4): isValueType matches StandardClassIds.Map / MutableMap exactly. Match
    subtypes of Map, so that it stays aligned with the library's KClass.isValueType.
Lenguaje dominante
Kotlin
Estrellas
1.1k
Forks
87
Merge medio
2 d 21 h
PR fusionados (30 d)
40

Preparar el entorno

Abrir en Codespaces

Inicia el contenedor de desarrollo del proyecto en tu navegador, con tu propia cuenta de GitHub.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de Kotlin/dataframe

Todos los issues de Kotlin/dataframe

Issues similares

Más issues de Kotlin

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.