join_spatialelement_table crashes when obs index name matches an existing column

Open Beginner friendly
#1,099 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
84/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
pandas, python
Domain
data

Research direction

Read spatialdata/_core/query/relational_query.py, focusing on _inner_join_spatialelement_table around line 390 and _left_join_spatialelement_table around line 471. Run the minimal AnnData reproduction with an obs index named EntityID and verify that both inner and left joins complete without the reset_index collision.

Written by the indexing model from the issue text.

Description

needs: triage priority: medium

Bug description

_inner_join_spatialelement_table and _left_join_spatialelement_table in spatialdata/_core/query/relational_query.py call table.obs.reset_index() (lines 390 and 471) without handling the case where the obs index name already exists as a column. This raises:

ValueError: cannot insert EntityID, already exists

How it manifests

In spatialdata-plot, render_shapes(color=...) calls join_spatialelement_table(..., how="inner"), which hits this crash. Users with Merfish data are affected because their tables have EntityID as both the obs index name and an obs column — a state that spatialdata's own validation allows.

Reported in https://github.com/scverse/spatialdata-plot/issues/441.

Minimal reproduction

import pandas as pd
from anndata import AnnData
from spatialdata.models import TableModel

obs = pd.DataFrame({
    "region": pd.Categorical(["shapes"] * 5),
    "EntityID": [0, 1, 2, 3, 4],
    "cell_type": ["A", "B", "C", "A", "B"],
})
table = AnnData(obs=obs)
table = TableModel.parse(table, region="shapes", region_key="region", instance_key="EntityID")

# Simulate the state found in real Merfish data loaded from disk
table.obs.index = pd.Index([0, 1, 2, 3, 4], name="EntityID")

# This is what join_spatialelement_table does internally — crashes here:
table.obs.reset_index()
# ValueError: cannot insert EntityID, already exists

Suggested fix

In _inner_join_spatialelement_table (line 390) and _left_join_spatialelement_table (line 471), handle the collision before calling reset_index(). For example, drop the index name when it already exists as a column:

obs = table.obs
if obs.index.name is not None and obs.index.name in obs.columns:
    obs = obs.reset_index(drop=True)
else:
    obs = obs.reset_index()
Dominant language
Python
Stars
394
Forks
95
Avg merge
3d 9h
Merged PRs (30d)
5

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from scverse/spatialdata

All issues in scverse/spatialdata

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.