Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

Store protocol for LinearExpression: DenseStore and CSRStore behind one interface, fallback in the base class

Đã đóng
  • Có gì không ổn?
#1,020 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
35/100
Loại issue
Tái cấu trúc
Độ rõ ràng
Đặc tả rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
numpy, pandas, python
Lĩnh vực
backend

Hướng nghiên cứu

Start with linopy/expressions.py, especially _densify, the .data property, and the listed CSR branches; read the dispatch-table notes in issues #971 and #1011 for context. Review the partial fallbacks in linopy/constraints.py as well, then run the proposed parametrized tests in test/test_csr.py. Done means dispatch entries preserve CSR backing under no_densify(), fallbacks warn with the operation name without densifying the receiver, and the existing twin-build tests pass.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

refactor sparse

[!NOTE]
The following content was generated by AI.

Describe the feature you'd like to see

Child of #1019 (step 2, store protocol). Rewritten on 2026-10-22: the legacy semantics are gone (#1041), so this is unblocked, and the design changed from a dispatch table on the frontend to a store protocol. Revised the same day after a review against the code: fallbacks are per argument case, so store operations return a reason instead of the base class providing per-method defaults. Internal refactor of linopy/expressions.py and linopy/csr.py; no change to results. Code links point to master at f6da2877.

Problem

A CSR-backed LinearExpression carries two fields, _data: Dataset and _csr: CSRLinearExpression | None, and falls back to dense in two ways. Explicit overrides call self._densify(reason) with a hand-written string. Every method without an override reads self.data (linopy/expressions.py:2193), which converts the receiver in place and for good through _densify_all (:3440) with the generic reason ".data read". About a hundred lines in expressions.py branch on _csr, and the implicit set (coeffs, vars, to_quadexpr, simplify, loc, densify_terms, the const setter, ...) is nowhere listed.

import warnings
import numpy as np, pandas as pd, linopy

linopy.options["warn_on_densify"] = True
i, t = pd.RangeIndex(20, name="i"), pd.RangeIndex(5, name="t")
m = linopy.Model(sparse=True)
x = m.add_variables(0, 1, coords=[i, t], name="x")
g = pd.Series(np.arange(20) % 3, index=i, name="g")

for op in ["coeffs", "to_quadexpr()", "isnull()"]:
    s = x.groupby(g).sum()
    assert s.is_sparse
    with warnings.catch_warnings(record=True) as w:
        warnings.simplefilter("always")
        eval(f"s.{op}")
    msgs = [str(x.message) for x in w if "dens" in str(x.message).lower()]
    print(f"{op:16} sparse after: {s.is_sparse!s:5}  warning: {msgs}")

The receiver is converted in place and the warning does not name the operation.

Design
  • A Store base class with two implementations. DenseStore wraps the xarray Dataset and receives the term-storage part of the code that today sits inline in BaseExpression. CSRStore is today's CSRLinearExpression (linopy/csr.py:263), renamed. Both expose .grid; the store abstracts term storage on a fixed Grid, not the Dataset. Both are immutable value objects: every operation returns a new store.
  • LinearExpression always holds one store, never None, and is mutable only by store swap (the const and data setters replace the store). It keeps what both stores need and what runs on Grid datasets today: __init__ normalization, _align_constant, fill_missing_coords, enforce_merge_dims, enforce_aux_conflict, override detection, absorb_absence, groupby key resolution. Each frontend method then calls the store once.
  • Fallbacks are per argument case, not per method. Every CSR override today is partial: _selected (linopy/expressions.py:2335), reindex on method/tolerance/fill_value (:2761), where on other/drop (:2423), shift on fill_value (:2397), sum on kwargs and non-grid dims (:2454), fillna scalar vs array (:2493), _sparse_matmul with four reasons (:2690). So a store operation returns Store | str, the pattern _aligned (:3489), _concatenated (:3542) and csr_rhs (linopy/constraints.py:1673) already use; DenseStore never returns a reason. LinearExpression turns a reason into store.to_dense(reason) followed by the dense call on the converted copy; the receiver keeps its store.
  • Store.to_dense(reason) is the single chokepoint and the only place that emits the densify notice (_densify_notice, linopy/csr.py:958). No cached dense twin: netcdf (linopy/io.py:1108) and Model.copy (:1410) stop reading .data and go through the store.
  • Multi-operand operations (merge :3549, _combined_with_constant :2295, _multiply_by_linear_expression :992, @) go through one free function over stores. The target-store rule and Store.to_sparse(reason) are step 3 of #1019; in this step the function keeps today's behaviour (a sparse operand pulls sparse, from_dense for the dense ones, densify-all for Dataset or quadratic operands).
  • The five to_linexpr(_dense=True) sites (:2611; linopy/variables.py:484, :505, :834, :874: quadratic products, rolling, cumsum) become explicit DenseStore constructions without a notice. Quadratic expressions use DenseStore only.
  • Expected warning after the change: Sparse (CSR) backing densified by reindex(method=...), one reason per argument case.

Out of scope, handled by step 3 of #1019: Model.store_cls, the store constructors as the construction seam, Store.to_sparse(reason).

Tests
  • The conformance suite of #1019 step 1 enumerates (operation, argument case, kept or reason) and runs it on both backends with assert_equivalent. One test asserts that every Store method has a CSRStore override or a documented reason.
  • Existing test/test_csr.py tests pass unchanged; the twin-build helpers are deleted in step 4b.
Ngôn ngữ chính
Python
Star
262
Fork
89
Merge trung bình
2 ngày 11 giờ
Pull request đã merge (30 ngày)
28

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của PyPSA/linopy

Tất cả issue của PyPSA/linopy

Issue tương tự

Thêm issue về Python

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.