Re-implementing rowsum/colsum for HDF5Matrix

Open

#41 opened on Feb 25, 2019 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty: 5/5
Estimated time: Over a week
Newbie friendliness: 35/100

Issue type: Feature
Clarity: Mostly clear
Activity status: Stale
Tech stack: r
Domain: data

Research direction

Start by tracing the existing rowsum() and colsum() methods and the historical rowsum,HDF5Matrix-method mentioned in the issue. Review how HDF5RealizationSink and RealizationSink are used, then define how results should be written incrementally, including support for writing separate operations to the same sink. Done means both operations can realize results to a RealizationSink without requiring the full result in memory.

Written by the indexing model from the issue text.

Description

I just noticed that bsseq has been broken in devel for a while due to the removal of the rowsum,HDF5Matrix-method (which I okayed back in https://github.com/Bioconductor/DelayedArray/commit/1a6a44f0119d81ff5eae975a37b4eb4efe85efab#comments).

I now remember why I had this method: if the data are so large that they need to be stored in HDF5 then the result of rowsum()/colsum() may itself be so large that it can't be stored in memory.
Therefore, the method I had allowed for this by allowing the result to be written to an HDF5RealizationSink (it also allowed me some extra control over the result, such as writing 2 separate rowsum()s to the same file as in https://github.com/PeteHaitch/DelayedMatrixStats/commit/27712d726a78350819961d4c10e50ef900118ef7).

Would it be possible to have a rowsum()/colsum() that supports realizing the result to a RealizationSink as it goes?
I will try to find time to do this if you're okay with it.

Dominant language: R
Stars: 29
Forks: 12
PR merge metrics: No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

Read the whole issue, then the project's contributing guide.
Comment on the issue to say you are picking it up — it saves two people doing the same work.
Fork the repository and make your change on a branch.
Open a pull request that references the issue number.

Similar issues

print.check_outliers() mislabels thresholds when method is a combination given in non-canonical order Open

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

easystats/performance#950 · Opened Sep 23, 2026
Adjust the cohort counts dissemination - only include cohorts in the manifest Open

Difficulty 2/5 1-3 hours Newbie friendliness 70/100

OHDSI/Picard#129 · Opened Sep 23, 2026
Unable to use 'start_time' 'end_time' params for get_airnow_monitors() Open

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

briandconnelly/airnow#9 · Opened Sep 23, 2026
Copy cohorts to keep old cohorts Open

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

OHDSI/CohortConstructor#774 · Opened Sep 23, 2026
[PRE REVIEW]: fastml: An R package for leakage-aware machine learning workflows with fold-local preprocessing Open

pre-review R TeX Track: 5 (DSAIS)

Difficulty 1/5 Under an hour Newbie friendliness 60/100

openjournals/joss-reviews#11330 · 7 comments · Opened Sep 23, 2026

More R issues