[refactor]: merge multiple models into singular file for standard output

Abierto
  • ¿Algo no está bien?
#457 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 2 días

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
48/100
Tipo de issue
Refactorización
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
r
Área
data

Línea de trabajo

Start by reading the standard-output code that creates the tables and figures, and trace where it currently passes a list of data frames; the issue does not name specific files or tests. Check whether those data frames share a schema and how the outputs consume them. Done means the outputs use a single data frame and the package's relevant tests pass.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

We should feed in a single df to all tables and figures rather than a list. Especially to aid in memory issues. See rationale below:

Memory wise, it is more efficient to have a dataframe rather than a list. I am thinking that if we actually move to parquet files/databases that the list of dataframes would be the least efficient way to store this information in R. And, according to Gemini

A single consolidated data frame is significantly more memory-efficient and performant than a list of many small data frames, assuming the data frames share the same schema. Every R object carries structural overhead (class definitions, attributes, dimensions, and names). If you store thousands of small data frames in a list, R maintains thousands of separate attribute vectors and column names. Consolidating into one table collapses that metadata into a single set. R's vector engine operates on contiguous blocks of memory. Iterating over elements in a list (e.g., via lapply() or purrr::map()) incurs call-stack overhead and traverses fragmented pointers across the heap, whereas column operations on a single data frame utilize SIMD vectorization and cache locality. The R garbage collector tracks every allocated node (Vcells and Ncells). Having a large list of separate data frames balloons the total count of distinct objects, forcing the garbage collector to run frequently and cause execution pauses.
A list of data frames is only preferable if:

  1. The structures differ: The tables do not share common columns or types, making a combined table sparse or full of NA padding.
  2. Independent lifecycle: You frequently delete, update, or process chunks independently and want to let R selectively garbage-collect individual pieces without rewriting a massive master table.

Originally posted by @kellijohnson-NOAA in #455

Lenguaje dominante
R
Estrellas
20
Forks
10
Merge medio
4 d 48 min
PR fusionados (30 d)
25

Preparar el entorno

Abrir en Codespaces

Inicia el contenedor de desarrollo del proyecto en tu navegador, con tu propia cuenta de GitHub.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de nmfs-ost/stockplotr

Todos los issues de nmfs-ost/stockplotr

Issues similares

Más issues de R

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.