CSC/ CSR classes

Aperta
#433 6 commenti 3 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 2 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
25/100
Tipo di issue
Funzionalità
Chiarezza
Da chiarire
Stato di attività
Ferma
Stack tecnologico
numpy, python
Ambito
data, performance

Direzione di ricerca

L'issue non indica file di implementazione né test. Inizia esaminando la gerarchia esistente dei tipi di array sparsi e le controparti CSR e CSC di scipy.sparse, quindi determina i requisiti relativi all'API pubblica, all'ereditarietà e alle prestazioni prima di implementare qualsiasi cosa. Il lavoro è completo quando esistono classi CSR e CSC esplicite con comportamento simile a ndarray e prestazioni comparabili alle controparti di scipy.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

enhancement
Is your feature request related to a problem? Please describe.

Could explicit CSR and CSC classes be added to this library?

I would love to stop using scipy.sparse.spmatrix classes. Their matrix API is a source of constant frustration. Lack of CSC and CSR arrays with an np.ndarray API has been a barrier to adoption not only of this library, but of xarray and dask for the scanpy/ anndata projects

Describe the solution you'd like

I would like there to be explicit CSR and CSC classes in this package. These should perform at least as well as scipy.sparse.spmatrix counterparts.

While higher dimensional algorithms are developed, performant implementations for these cases can be taken from scipy.sparse or developed from reference implementations. I think this is a low effort/ high pay-off path towards performant implementations for common formats. They wouldn't be n-dimensional, but there's a lot of value in being able to start consolidating efforts to this library before an n-dimensional generalization has the features and performance of the common 2d cases.

Describe alternatives you've considered
No explicit CSC/ CSR classes. These are subsets of more generic structures so why special case them?

I think this is the wrong approach due to how common these structures are. Even a flexible framework like taco exports these formats (pytaco.csc and pytaco.csr).

We could also consider what code looks like if these classes are not defined. All dispatch here could change from isinstance(x, sparse.csr_matrix) to something like isinstance(x, sparse.GXCS) and x.compressed_axes == (0,) and x.ndims == 2), but something closer to the first case is probably an easier sell.

CSC/ CSR classes as thin wrappers of an n-dimensional format

I think this is a great goal. However, it requires an implementation of that format which is at least as performant in most (probably all) cases as the scipy equivalents for broad adoption. As mentioned above, performant implementations can be (1) sourced from a current dependency (scipy.sparse) or (2) easily adapted from existing reference implementations.

Code specialized for these cases could gradually be replaced with more generic versions as they are implemented. However, performant implementation for these common cases would not be blocked by implementation of n-dimensional algorithms.

Additional context
I could expand on why I think this is the right way to go

I could go on about how important I think it is for there to be CSR and CSC classes in this library, and why it's important to implement these separately from higher dimensional generalizations. I find that makes this post too long. I'd be happy to add these arguments if asked.

Type hierarchy

I don't know what the appropriate type hierarchy is here. Should issubclass(CSR, GXCS) work? Or should there be an abstract CompressedDimSparseArray parent class for all of these? What is the result type of np.stack(: list[CSR]).

These are things that could be decided on later, once the n-dimensional compressed representation is public API.

Lingua principale
Python
Stelle
670
Fork
167
Merge medio
3g 4h
PR unite (30g)
7

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di pydata/sparse

Tutte le issue di pydata/sparse

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.