[Data] repartition() warns that shuffle is ignored before rejecting the same argument

Open Beginner friendly
  • Is something wrong?
#67,604 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 2 days

@cryptus-neoxys is already working on this.

Since Nov 10, 2026.

  • #68020 by @cryptus-neoxys — open
  • #68062 by @Yin-1022 — open

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
74/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
data-engineering

Research direction

Start with Dataset.repartition() and the target_num_rows_per_block validation described in the issue; run the minimal reproduction to confirm the warning and error. Add a regression test showing that shuffle=True raises the expected ValueError without emitting the contradictory warning.

Written by the indexing model from the issue text.

Description

community-backlog data docs usability

Problem / Description

Dataset.repartition() gives contradictory feedback when target_num_rows_per_block is used together with shuffle=True.

The method first emits:

UserWarning: `shuffle` is ignored when `target_num_rows_per_block` is set.

but then immediately raises:

ValueError: `shuffle` must be False when `target_num_rows_per_block` is set.

This makes the public API contract unclear: the warning tells users that the argument is accepted but ignored, while the validation rejects the same configuration.

The current repartition() documentation describes target_num_rows_per_block as exclusive with shuffle, so the ValueError behavior appears consistent with the documented contract.

Minimal Reproduction

import warnings
import ray

ray.init(include_dashboard=False, num_cpus=2)

ds = ray.data.range(10)

with warnings.catch_warnings(record=True) as caught:
    warnings.simplefilter("always")

    try:
        ds.repartition(
            target_num_rows_per_block=3,
            shuffle=True,
        )
    except Exception as e:
        print(type(e).__name__, str(e))

    for warning in caught:
        print(type(warning.message).__name__, str(warning.message))

Output:

ValueError `shuffle` must be False when `target_num_rows_per_block` is set.
UserWarning `shuffle` is ignored when `target_num_rows_per_block` is set.

Expected behavior

repartition() should communicate one consistent contract for this argument combination.

Since target_num_rows_per_block is documented as exclusive with shuffle, passing shuffle=True should raise the existing ValueError without first emitting a warning that says the argument will be ignored.

Proposal

Remove the shuffle ignored warning from the target_num_rows_per_block branch and keep the existing validation:

if target_num_rows_per_block is not None and shuffle:
    raise ValueError(
        "`shuffle` must be False when `target_num_rows_per_block` is set."
    )

Add a regression test confirming that this invalid argument combination raises the expected ValueError without emitting the contradictory warning.

Dominant language
Python
Stars
44.3k
Forks
8.3k
Avg merge
4d 1h
Merged PRs (30d)
414

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ray-project/ray

All issues in ray-project/ray

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.