Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

Meta: separate dense and sparse stores from the expression and constraint frontend

オープン
  • 何か問題がありますか？
#1,019 コメント 2 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

@Rishabhkanhaiya がすでに取り組んでいます。

2026年10月7日 から。

  • #1034 @FabianHofmann による — オープン

評価

難易度
5/5
見積もり時間
1週間以上
初心者へのやさしさ
25/100
issue の種類
リファクタリング
明瞭さ
明確に書かれている
活発さ
活発
技術スタック
numpy, pandas, python
領域
backend

調査の方向性

This is a meta issue; start with child #1020 for the first step, and read the referenced sections in linopy/expressions.py and linopy/csr.py to understand the current dispatch and sparse operations. The plan also identifies test/test_csr.py, constraints.py, matrices.py, and model.py for later steps. Done means completing the staged store refactor and conformance tests; several steps depend on other work, including legacy-semantics removal.

索引モデルが issue の本文から書いたものです。

説明

discussion refactor sparse

[!NOTE]
The following content was generated by AI.

Describe the feature you'd like to see

Meta issue. With #972 the sparse (CSR) backing is complete and observable, and Model(sparse=True) makes dense and sparse two equivalent ways to hold the same model. The code does not reflect that yet: sparse support is layered onto the dense classes by overrides and isinstance switches. This issue fixes the target architecture and the order of the refactoring. Child issues are spawned per step. Code links point to master at 69dd5c97.

Target

One LinearExpression class and one Constraint class, each holding either a dense or a sparse store behind the same small interface. Every conversion between stores passes through one chokepoint that names the operation. Model, export, persistent diff and solvers never branch on the store.

Where the coupling is today
Layer State Reference
LinearExpression one class, 34 explicit branch points on _csr / is_sparse / _try_csr_* / _sparse_*; every non-overridden method falls back implicitly by reading .data, which converts the receiver in place and for good with a generic reason linopy/expressions.py:2443-3100, groupby :485-669, _try_csr_merge :3740, _densify_all :3698
CSRLinearExpression already has the shape of a store interface, mirroring the frontend one to one (summed, scaled, added, contracted, taken, reindexed, filled, renamed, with_const); the dense equivalent is inline xarray code in BaseExpression, mixed with alignment and semantics checks linopy/csr.py:262-694
Constraints Constraint and CSRConstraint are siblings under ConstraintBase(ABC) with about twenty abstract methods each implements; CSRConstraint stores a raw csr_array plus seven side arrays, not a CSRLinearExpression, so lhs and data rebuild a Dataset on every access and from_csr / from_dense are separate conversions; _csr means different things on the two classes linopy/constraints.py:145, :762, :797-812, :1229, :1595, :1664, :1739
Model the read-only sparse key is clean; three reads remain (groupby sum, @, add_constraints fast path) linopy/model.py:572, :1375; expressions.py:615, :2888
Export and solvers mostly polymorphic through ConstraintBase; MatrixAccessor and the dual write-back still switch on type; Model.copy densifies model-level expressions linopy/matrices.py:48-55, :289; model.py:2533; io.py:1424
Tests equivalence by twin builds (twin_models, assert_sparse_matches, assert_frozen_equal) and ten hand-picked sparse parametrizations; no backend fixture, no conformance suite over the store operations test/test_csr.py

Reproduction of the implicit fallback:

import warnings
import numpy as np, pandas as pd, linopy

linopy.options["semantics"] = "v1"
linopy.options["warn_on_densify"] = True
i, t = pd.RangeIndex(20, name="i"), pd.RangeIndex(5, name="t")
m = linopy.Model(sparse=True)
x = m.add_variables(0, 1, coords=[i, t], name="x")
g = pd.Series(np.arange(20) % 3, index=i, name="g")

for op in ["shift(t=1)", "diff('t')", "coeffs", "to_quadexpr()", "isnull()"]:
    s = x.groupby(g).sum()
    assert s.is_sparse
    with warnings.catch_warnings(record=True) as w:
        warnings.simplefilter("always")
        eval(f"s.{op}")
    msgs = [str(x.message) for x in w if "dens" in str(x.message).lower()]
    print(f"{op:16} sparse after: {s.is_sparse!s:5}  warning: {msgs}")

Output on master (69dd5c97):

shift(t=1)       sparse after: False  warning: ['Sparse (CSR) backing densified: `.data` read, e.g. by an operation without a sparse path.']
diff('t')        sparse after: False  warning: ['Sparse (CSR) backing densified: `.data` read, e.g. by an operation without a sparse path.']
coeffs           sparse after: False  warning: ['Sparse (CSR) backing densified: `.data` read, e.g. by an operation without a sparse path.']
to_quadexpr()    sparse after: False  warning: ['Sparse (CSR) backing densified: `.data` read, e.g. by an operation without a sparse path.']
isnull()         sparse after: True   warning: []
Plan
Step Content Depends on
0: legacy removal Make v1 the default in the next release, then delete legacy semantics (linopy/semantics.py and the is_v1() gates). Not part of this issue, but the prerequisite for steps 1, 3 and 4: those steps rewrite exactly the methods that carry the gates, so doing them first means doing them twice. #845, #846
1: chokepoint One _delegate(name, ...) on LinearExpression: an operation either has a sparse implementation or converts with reason=name. The 34 branches become a dispatch table mapping frontend methods to CSRLinearExpression counterparts. .data becomes a plain read through the same chokepoint. The receiver keeps its backing; the converted copy is the operand of the fallback. Partial fallbacks (_selected, _sparse_matmul, _try_csr_merge, csr_rhs) return a reason instead of calling _densify_all. The table is the authoritative list of backing-preserving operations: #971 documents it, #1011 adds rows. 0
2: conformance tests A backend fixture (dense, sparse) and one parametrized suite that runs every table entry on both stores and compares cells, terms and isnull, plus a test that every fallback warns with its own operation name. Replace twin builds where the fixture covers them. Only needs require_v1 and a list of operations, so it can land before step 0 and then guards the legacy removal as well. –
3: constraint store CSRConstraint holds a CSRLinearExpression plus sign, rhs, scaling and labels. lhs returns a sparse LinearExpression without a rebuild, from_csr becomes a constructor, from_dense goes through CSRLinearExpression.from_dense. Then Constraint and CSRConstraint collapse into one class with a frozen flag whose storage is its lhs; ConstraintBase goes away. MatrixAccessor and the dual write-back lose their isinstance branches. 0, 1, 2
4: dense store Move the xarray Dataset manipulation out of BaseExpression into a DenseLinearExpression implementing the same protocol as CSRLinearExpression. LinearExpression keeps only the user API, alignment and semantics checks, and delegates to its store. Align the store method names with the frontend names in this step. 0, 3

Ordering notes:

  • Step 2 first, at any time. Step 0 after the release. Steps 1, 3 and 4 back to back after step 0.
  • Step 1 before #1011 would absorb its new branches, but #1011 is user-facing and may land first; the four extra hand-written branches are then absorbed by step 1.
  • #1010 is internal to csr.py and independent of this plan.
  • Step 4 depends on step 0 because the sparse form is valid only under v1 (linopy/csr.py:1-27): a store protocol that one backend implements only under one semantics is worse than the current branches.

Rejected:

  • A lazy expression graph (already rejected in #972).
  • A sparse store for variables and one-term expressions (no gain in the #972 benchmark).
  • Two public classes (DenseLinearExpression / SparseLinearExpression) at the frontend. Users should not pick a class; the model key picks the store.
Children
  • #845, #846 (step 0: v1 default, then legacy semantics removal)
  • #1020 (step 1: densification chokepoint and dispatch table)
  • step 2: backend-parametrized conformance suite
  • step 3: CSRConstraint on a CSRLinearExpression, single Constraint class
  • step 4: DenseLinearExpression store
主要言語
Python
スター
260
フォーク
89
平均マージ
1日 1時間
マージ済み PR（30日）
24

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

PyPSA/linopy のほかの issue

PyPSA/linopy の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。