Size of DNAStringSet object not compatible with `nchar()`
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Idoneità per principianti
- 25/100
- Tipo di issue
- Bug
- Chiarezza
- Da chiarire
- Stato di attività
- Ferma
- Stack tecnologico
- r
- Ambito
- performance
Direzione di ricerca
Inizia eseguendo il reprex R fornito e confrontando obj_size() con nchar() per gli oggetti DNAStringSet genome, promoters e promoters_subset. Traccia il modo in cui il subsetting di DNAStringSet conserva l'archiviazione delle sequenze, quindi determina se le dimensioni osservate sono previste o indicano un bug; il lavoro deve includere una spiegazione documentata oppure una correzione verificata con copertura di regressione.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Hi all,
For a long time, I've noticed that {Biostrings} is very weird in terms of memory usage. For instance, when one subsets a DNAStringSet object to create a much smaller DNAStringSet object, object sizes do not change much (if at all). Here is a demonstration, comparing a DNAStringSet object with an entire plant genome to another (much smaller) DNAStringSet object with promoter sequences only:
suppressPackageStartupMessages({
library(Biostrings)
library(GenomicRanges)
library(lobstr)
})
options(timeout = 1e4)
# Load genome and annotation
genome <- readDNAStringSet(
"https://ftp.ebi.ac.uk/ensemblgenomes/pub/release-61/plants/fasta/hordeum_vulgare/dna/Hordeum_vulgare.MorexV3_pseudomolecules_assembly.dna_rm.toplevel.fa.gz"
)
names(genome) <- gsub(" .*", "", names(genome))
annot <- rtracklayer::import(
"https://ftp.ebi.ac.uk/ensemblgenomes/pub/release-61/plants/gff3/hordeum_vulgare/Hordeum_vulgare.MorexV3_pseudomolecules_assembly.61.gff3.gz"
)
# Keep genes only and add seqlengths
annot <- annot[annot$type == "gene"]
sl <- setNames(width(genome), names(genome))
sl <- sl[seqlevels(annot)]
seqlengths(annot) <- sl
# Extract promoter sequences
prom_ranges <- trim(promoters(annot, 1000, 200))
prom_seqs <- BSgenome::getSeq(genome, prom_ranges)
names(prom_seqs) <- prom_ranges$gene_id
# Select promoters of random genes
sel_genes <- sample(names(prom_seqs), 1e4, replace = FALSE)
prom_seqs_subset <- prom_seqs[sel_genes]
# Inspect objects
size_df <- data.frame(
name = c("genome", "promoters", "promoters_subset"),
size = c(
obj_size(genome),
obj_size(prom_seqs),
obj_size(prom_seqs_subset)
),
Mbp = c(
sum(nchar(genome)) / 1e6,
sum(nchar(prom_seqs)) / 1e6,
sum(nchar(prom_seqs_subset)) / 1e6
)
)
size_df
#> name size Mbp
#> 1 genome 4.23 GB 4225.57752
#> 2 promoters 4.25 GB 42.98556
#> 3 promoters_subset 4.25 GB 11.99864
# Does removing `genome` do anything?
rm(genome)
size_df2 <- data.frame(
name = c("promoters", "promoters_subset"),
size = c(
obj_size(prom_seqs),
obj_size(prom_seqs_subset)
),
Mbp = c(
sum(nchar(prom_seqs)) / 1e6,
sum(nchar(prom_seqs_subset)) / 1e6
)
)
size_df2
#> name size Mbp
#> 1 promoters 4.25 GB 42.98556
#> 2 promoters_subset 4.25 GB 11.99864
sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#> setting value
#> version R version 4.4.1 (2024-06-14)
#> os Ubuntu 22.04.4 LTS
#> system x86_64, linux-gnu
#> ui X11
#> language (EN)
#> collate en_US.UTF-8
#> ctype en_US.UTF-8
#> tz Europe/Brussels
#> date 2025-08-07
#> pandoc 3.2 @ /usr/lib/rstudio/resources/app/bin/quarto/bin/tools/x86_64/ (via rmarkdown)
#>
#> ─ Packages ───────────────────────────────────────────────────────────────────
#> package * version date (UTC) lib source
#> abind 1.4-5 2016-07-21 [1] CRAN (R 4.4.1)
#> Biobase 2.64.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.1)
#> BiocGenerics * 0.50.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.1)
#> BiocIO 1.14.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.1)
#> BiocParallel 1.38.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.1)
#> Biostrings * 2.72.1 2024-06-02 [1] Bioconductor 3.19 (R 4.4.1)
#> bitops 1.0-7 2021-04-24 [1] CRAN (R 4.4.1)
#> BSgenome 1.72.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.1)
#> cli 3.6.3 2024-06-21 [1] CRAN (R 4.4.1)
#> codetools 0.2-20 2024-03-31 [1] CRAN (R 4.4.1)
#> crayon 1.5.3 2024-06-20 [1] CRAN (R 4.4.1)
#> curl 5.2.1 2024-03-01 [1] CRAN (R 4.4.1)
#> DelayedArray 0.30.1 2024-05-07 [1] Bioconductor 3.19 (R 4.4.1)
#> digest 0.6.36 2024-06-23 [1] CRAN (R 4.4.1)
#> evaluate 0.24.0 2024-06-10 [1] CRAN (R 4.4.1)
#> fastmap 1.2.0 2024-05-15 [1] CRAN (R 4.4.1)
#> fs 1.6.4 2024-04-25 [1] CRAN (R 4.4.1)
#> GenomeInfoDb * 1.40.1 2024-05-24 [1] Bioconductor 3.19 (R 4.4.1)
#> GenomeInfoDbData 1.2.12 2024-07-24 [1] Bioconductor
#> GenomicAlignments 1.40.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.1)
#> GenomicRanges * 1.56.1 2024-06-12 [1] Bioconductor 3.19 (R 4.4.1)
#> glue 1.8.0 2024-09-30 [1] https://cran.r-universe.dev (R 4.4.1)
#> htmltools 0.5.8.1 2024-04-04 [1] CRAN (R 4.4.1)
#> httr 1.4.7 2023-08-15 [1] CRAN (R 4.4.1)
#> IRanges * 2.38.1 2024-07-03 [1] Bioconductor 3.19 (R 4.4.1)
#> jsonlite 1.8.8 2023-12-04 [1] CRAN (R 4.4.1)
#> knitr 1.48 2024-07-07 [1] CRAN (R 4.4.1)
#> lattice 0.22-6 2024-03-20 [1] CRAN (R 4.4.1)
#> lifecycle 1.0.4 2023-11-07 [1] CRAN (R 4.4.1)
#> lobstr * 1.1.2 2022-06-22 [1] CRAN (R 4.4.1)
#> Matrix 1.7-0 2024-04-26 [1] CRAN (R 4.4.1)
#> MatrixGenerics 1.16.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.1)
#> matrixStats 1.3.0 2024-04-11 [1] CRAN (R 4.4.1)
#> prettyunits 1.2.0 2023-09-24 [1] CRAN (R 4.4.1)
#> R6 2.5.1 2021-08-19 [1] CRAN (R 4.4.1)
#> RCurl 1.98-1.16 2024-07-11 [1] CRAN (R 4.4.1)
#> reprex 2.1.1 2024-07-06 [1] CRAN (R 4.4.1)
#> restfulr 0.0.15 2022-06-16 [1] CRAN (R 4.4.1)
#> rjson 0.2.21 2022-01-09 [1] CRAN (R 4.4.1)
#> rlang 1.1.4 2024-06-04 [1] CRAN (R 4.4.1)
#> rmarkdown 2.27 2024-05-17 [1] CRAN (R 4.4.1)
#> Rsamtools 2.20.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.1)
#> rstudioapi 0.16.0 2024-03-24 [1] CRAN (R 4.4.1)
#> rtracklayer 1.64.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.1)
#> S4Arrays 1.4.1 2024-05-20 [1] Bioconductor 3.19 (R 4.4.1)
#> S4Vectors * 0.42.1 2024-07-03 [1] Bioconductor 3.19 (R 4.4.1)
#> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.4.1)
#> SparseArray 1.4.8 2024-05-24 [1] Bioconductor 3.19 (R 4.4.1)
#> SummarizedExperiment 1.34.0 2024-05-01 [1] Bioconductor 3.19 (R 4.4.1)
#> UCSC.utils 1.0.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.1)
#> withr 3.0.0 2024-01-16 [1] CRAN (R 4.4.1)
#> xfun 0.51 2025-02-19 [1] CRAN (R 4.4.1)
#> XML 3.99-0.17 2024-06-25 [1] CRAN (R 4.4.1)
#> XVector * 0.44.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.1)
#> yaml 2.3.9 2024-07-05 [1] CRAN (R 4.4.1)
#> zlibbioc 1.50.0 2024-04-30 [1] Bioconductor 3.19 (R 4.4.1)
#>
#> [1] /home/faalm/R/x86_64-pc-linux-gnu-library/4.4
#> [2] /usr/local/lib/R/site-library
#> [3] /usr/lib/R/site-library
#> [4] /usr/lib/R/library
#>
#> ──────────────────────────────────────────────────────────────────────────────
Created on 2025-08-07 with reprex v2.1.1
Note that the number of sequences is dramatically different, while object sizes remain pretty much the same. Is that something you are aware of? Why does this happen?
As a side comment, I've also noticed that, for small sets of sequences, it's more efficient to store gzipped FASTA files compared to binary .rds files with XStringSet objects. This is not true for large XStringSet objects, though. It seems like subsetting a large XStringSet object preserves something of the original object in the subset. Does that make sense?
All the best,
Fabricio
- Lingua principale
- R
- Stelle
- 72
- Fork
- 17
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Preparare l'ambiente
Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di Bioconductor/Biostrings
-
low level utilities packageAperta
Difficoltà 5/5 Più di una settimana Idoneità per principianti 25/100
Bioconductor/Biostrings#142 · 1 commento ·
-
Difficoltà 5/5 Più di una settimana Idoneità per principianti 25/100
Bioconductor/Biostrings#141 ·
-
Difficoltà 3/5 1-2 giorni Idoneità per principianti 48/100
Bioconductor/Biostrings#140 · 2 commenti ·
-
Single-pass file readingAperta
Difficoltà 4/5 3-5 giorni Idoneità per principianti 45/100
Bioconductor/Biostrings#138 ·
-
Improve error message for `subseq` when called on a vectorForse di nuovo libera @ahl27 l’ha presa 307 giorni fa e non c’è nessuna pull request aperta. Aperta
Bioconductor/Biostrings#135 · 1 assegnatario ·
Tutte le issue di Bioconductor/Biostrings
Issue simili
-
bug
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 66/100
nmfs-ost/stockplotr#518 ·
I maintainer di solito rispondono entro 2 giorni
-
bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 64/100
nf-core/ampliseq#1146 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
HelenaLC/spatialdataR#295 ·
-
R cheat sheetAperta
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 68/100
KUBDatalab/R-PUFF-SEMS#170 ·
-
size-time-budget
Difficoltà 2/5 1-3 ore Idoneità per principianti 69/100
kapelner/EDI#66 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno