File and column descriptions from dataset-metadata.json are still not saved (after #1055)

Open
#1,207 0 comments 1 reaction 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
45/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
python
Domain
api, backend

Research direction

Start with docs/datasets_metadata.md and trace datasets create, datasets version, and metadata --update through DatasetApiService.UpdateDatasetMetadata. Reproduce with the supplied dataset-metadata.json, inspect GetDatasetMetadata, and verify that file and column descriptions survive each command and appear in the Data Explorer.

Written by the indexing model from the issue text.

Description

bug dependencies

Summary

File and column descriptions in dataset-metadata.json are accepted without error but never saved. This happens on datasets create, datasets version and datasets metadata --update. #447 and #248 were closed by #1055, but that PR changed only how the CLI builds the request. The server still drops the descriptions. Descriptions entered in the web editor are saved as expected.

The documentation says this should work:

  • docs/datasets_metadata.md lists resources (with path, description and schema.fields[].description) as supported for kaggle datasets create and kaggle datasets version.
  • The same page says kaggle datasets metadata --update "supports all fields mentioned above for kaggle datasets version".

Environment

  • kaggle 2.2.4, kagglesdk 0.1.37
  • Python 3.14.3, Ubuntu (GitHub Actions ubuntu-latest)
  • Authenticated with KAGGLE_API_TOKEN

Steps to reproduce

  1. Make a folder with data.csv:

    year,category,grants
    1977-78,Refugee,9326
    1978-79,Refugee,12750
    
  2. Add dataset-metadata.json beside it, following the documented format:

    {
      "title": "Metadata description test",
      "id": "<your-username>/metadata-description-test",
      "licenses": [{"name": "CC0-1.0"}],
      "resources": [
        {
          "path": "data.csv",
          "description": "Rows of the table, as CSV.",
          "schema": {
            "fields": [
              {"name": "year", "description": "Financial year.", "type": "string"},
              {"name": "category", "description": "Visa category.", "type": "string"},
              {"name": "grants", "description": "Visas granted.", "type": "number"}
            ]
          }
        }
      ]
    }
    
  3. Create the dataset, wait until it is ready, then upload the same files as a new version:

    kaggle datasets create -p . -u
    kaggle datasets status <your-username>/metadata-description-test   # until "ready"
    kaggle datasets version -p . -m "second version with the same resources"
    
  4. Send the same file again as a metadata update:

    kaggle datasets metadata <your-username>/metadata-description-test --update -p .
    
  5. Open the dataset page and select data.csv in the Data Explorer.

Expected

"About this file" shows "Rows of the table, as CSV.", and each column shows its description. The usability rating ticks File Description and Column Description.

Actual

  • "About this file" says "This file does not have a description yet", and no column has a description.
  • The usability rating still crosses File Description and Column Description.
  • Every command exits 0. metadata --update prints "successfully updated dataset metadata".
  • The other fields in the same update are saved: keywords, expectedUpdateFrequency, userSpecifiedSources and the cover image.

What else we tried

All of these were accepted with an empty errors list, and none changed the page:

  • resources with the documented types (string, number, datetime), and with integer.
  • Calling DatasetApiService.UpdateDatasetMetadata through kagglesdk directly, with settings.data set to DatasetSettingsFile entries in four forms: description only; description and totalBytes; those plus columns; and all three files of the dataset.
  • Three dataset versions uploaded with resources present (create once, then version twice).

GetDatasetMetadata returns info.data as an empty list after each update, as #410 also notes.

Example

A public dataset that shows the problem: https://www.kaggle.com/datasets/nationaldigitalau/au-humanitarian-program-grants. Version 3 was uploaded with resources for all three files and then sent the same settings through metadata --update.

Dominant language
Python
Stars
7.7k
Forks
1.5k
Avg merge
1d 6h
Merged PRs (30d)
17

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Kaggle/kaggle-cli

All issues in Kaggle/kaggle-cli

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.