Render AE drill-down listings lazily to shrink the ae_forestly() widget
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 45/100
- Issue type
- Refactor
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- javascript, r
- Domain
- data-visualization, performance
Research direction
The work is in the ae_forestly() function, which builds a widget. The drill-down listings are currently built as nested reactable tables. The first step is to understand the existing details function and the inst/js/search-filter.js file. The change depends on an external lt.js extension from the lt repo. Once that is available, embed AE listing data as JSON and replace the R-side details renderer with a client-side JS callback. Verify features by testing sorting, search, and filtering in the drill-down.
Written by the indexing model from the issue text.
Description
Background
Follow-up to #147 (running-time / output-size optimization). #157 removed the duplicated per-nested-table search/filter JS (widget ~1.4 GB → ~350 MB at 1,000 subjects). Profiling the remainder shows the drill-down listing tables are now ~70%+ of the widget: ae_forestly() builds one nested reactable per main-table row (44,865 rows in the profiled case), and each carries ~3 KB of scaffolding regardless of content. All of them are serialized eagerly into the self-contained HTML even though a user only ever expands a few.
Measured at 300 subjects: 13,673 rows, 59.9 MB total; per-row drill-down ≈ 3,135 B.
Proposal
Render the drill-down lazily on the client, and embed the listing data once:
- Embed the AE listing data a single time as compact, column-oriented JSON (a page global keyed by parameter + AE term), instead of one pre-built table per row.
- Replace the R-side
details = function(index) { reactable(...) }with a client-sidedetails = JS("function(rowInfo){ … }")(reactable supports aJS()details renderer). On expand, the callback slices the embedded data for that term and renders an interactive table into the expanded container. - Render that table with a lightweight interactive
lttable (sort + search/filter + paging), via the plannedlt.jsinteractive extension.
Expected impact
- Drill-down scaffolding (~141 MB at 1,000 subj) collapses to one shared renderer + the listing data embedded once → projected total ~340 MB → ~60–100 MB.
- The per-row R-side
reactableconstruction disappears, so most of the ~605sae_buildstage (this box; single-core) goes away too.
Behavior to preserve
- Drill-down keeps column sorting, table-wide search, per-column filters, and paging (current
reactablefeatures). - Search semantics unchanged: substring,
!negation, andx-expression (currently ininst/js/search-filter.js). - Main table and crosstalk (param selector + incidence slider) are unchanged — crosstalk wraps only the main table; the drill-down uses none.
Dependency / blocker
Requires the interactive lt.js extension (sort/search/filter/paging as an opt-in add-on). Implementation plan lives in the lt repo (TODO.md). This forestly change is blocked until that extension is published.
Tasks
- Interactive
ltextension available (seeltrepo). - Emit the AE listing once as a compact JSON global in
ae_forestly(). - Switch
detailsto a client-sideJS()renderer calling theltextension. - Attach the
lt(+ interactive extension) JS/CSS dependencies. - Verify drill-down features (sort/search/filter/paging) and search semantics.
- Re-measure widget size and build time; update #147.
Refs #147, #157.
- Dominant language
- R
- Stars
- 22
- Forks
- 6
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 10
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from Merck/forestly
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
format_ae_listing(): AE duration set to NA (not "Unknown") for outcomes outside two hardcoded values Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
bug
Difficulty 3/5 1-2 days Newbie friendliness 65/100
Similar issues
-
Affects Web App documentation PRIORITY LOW
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 1/5 Under an hour Newbie friendliness 80/100
hubverse-org/hubCI#36 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 85/100
-
Release 1.4.0 Open
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
pharmaverse/pharmaverseadam#170 ·