polynote/polynote

Nested data from Spark Dataset is not displayed in Viz Cell

Open

#1,406 opened on Aug 24, 2023

View on GitHub
 (2 comments) (0 reactions) (0 assignees)Scala (368 forks)batch import
help wanted

Repository metrics

Stars
 (4,205 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

Description When visualizing the contents of a Spark Dataset (even with explicit schema provided in the form of a Java data class) in a Viz Cell, the data in nested fields is not resolved and a placeholder text is displayed.

Reproduction Steps to reproduce the behavior:

  1. Read a dataset from a source (table, file)
  2. Cast it to a defined data class (with nested elements)
  3. Display the loaded contents in a Viz Cell

Expectation The contents of a nested field are visible, preferably with the option to collapse in order to save screen space.

Environment

Otherwise, please provide the following information:

  • Polynote Version:
    • Example: 0.5.1
  • Operating System:
    • Example: Windows 10
  • JVM Version:
    • Example: 17
  • Scala Version:
    • Example: 2.13
  • How you installed Polynote:
    • Custom made Docker file
  • The command you are using to start Polynote:

  • Any other details that might be useful, such as whether you are running Polynote as a service, using a custom script, etc.
    • Polynote is running in an Openshift environment

Configuration If applicable, please attach or paste your Polynote configuration (config.yml)

Notebook If applicable, please attach or paste the notebook file (.ipynb) you were trying to run when you see this bug.

Logs If applicable, please attach or paste the Polynote log

Screenshots If applicable, please attach screenshots to help explain your problem.

Additional context Add any other context about the problem here. polynoteStructProblem

Contributor guide