Make the sparse path observable and controllable (.is_sparse, densify warning, per-call sparse=)
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- python
- Domain
- backend-api-design, performance
Research direction
Start by reading the sparse paths at linopy/expressions.py:2570, 3291, 3319, 2606-2609 and linopy/constraints.py:1427, then inspect Grid and the existing sparse_groupby option. Define how repr(), shape, sizes, coords, dims and isnull avoid densifying before implementing the public state, warnings and per-call controls. Done means sparse fallbacks are observable and controllable without changing default behavior.
Written by the indexing model from the issue text.
Description
[!NOTE]
The following content was generated by AI.
Describe the feature you'd like to see
Make the sparse path observable and controllable. Child of #756.
Right now a user cannot tell whether their build stayed sparse. There is no .is_sparse property, no marker in the repr, and no warning when the backing is dropped. Four separate code paths fall back to dense by returning None, all silently: _sparse_matmul (linopy/expressions.py:2570), _try_csr_merge (3319), _aligned (3291) and csr_rhs (linopy/constraints.py:1427).
Combined with the fact that reading repr() or .sizes destroys the backing in place (filed separately as a bug), this makes the feature very hard to use deliberately on anything larger than a toy model.
Three parts:
- Expose the state. A public
.is_sparseonLinearExpression, and a marker in__repr__— provided the repr itself stops densifying. - Explain the fallbacks. An opt-in warning, for example
options["warn_on_densify"], naming the operation that dropped the backing. Off by default, so nothing changes for existing users. - Per-call control. A
sparse=flag on@/dotandmerge, mirroringgroupby(...).sum(sparse=True). Today@contracts sparsely but returns a dense object unless the input was already CSR-backed or the globaloptions["sparse_groupby"]is set (expressions.py:2606-2609), which is a surprising coupling between an unrelated option and the result's representation. #961 explicitly deferred this flag.
Implementation ideas
Serving shape, sizes, coords, dims and isnull from the existing Grid is a prerequisite for a non-densifying repr, and is worth doing on its own: Grid already holds all of it.
- Dominant language
- Python
- Stars
- 257
- Forks
- 87
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 32
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from PyPSA/linopy
-
bug solver interface
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
documentation sparse
Difficulty 3/5 1-2 days Newbie friendliness 68/100
-
enhancement sparse
Difficulty 4/5 3-5 days Newbie friendliness 48/100
-
enhancement performance sparse
Difficulty 4/5 3-5 days Newbie friendliness 68/100
-
enhancement performance sparse
Difficulty 4/5 3-5 days Newbie friendliness 52/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
anthropics/skills#1811 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
speaches-ai/speaches#678 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
datalayer/mcp-compose#42 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
conda-forge/spacy-feedstock#177 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
UKGovernmentBEIS/inspect_evals#2523 ·