`input$table_rows_selected` Returns Incorrect Indexes with Large Data & Slow Connection when Filter is set by default

Open
  • Is something wrong?
#1,087 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
38/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Stale
Tech stack
javascript, r
Domain
frontend

Research direction

Start with inst/htmlwidgets/datatables.js around line 325 and reproduce the provided Shiny app with a throttled Slow 3G connection. Inspect how the dataSrc callback updates DT_rows_current across repeated calls and how it determines input$table_rows_selected. Done means selecting rows after the default filter consistently reports the correct row IDs for large data.

Written by the indexing model from the issue text.

Description

input$table_rows_selected Returns Incorrect Indexes with Large Data & Slow Connection when Filter is set by default

Description:

When a default filter is set on a DT::datatable (options -> searchCols -> search), the input$table_rows_selected can return incorrect row indexes, especially with large data frames (10k+ rows) and a slow internet connection (e.g., Slow 3G).

Steps to Reproduce:

Use the provided R code to set up a Shiny app with a DT::datatable.
Throttle the connection to "Slow 3G" in browser DevTools.
Refresh the page.
Click on any row in the table.

Expected Behavior:

The R console should print the correct row ID of the selected row.

Actual Behavior:

The R console prints incorrect rows_selected IDs.

Code to Reproduce:

library(shiny)
library(DT)

odd_even <- data.frame(is_odd = c(1:50000) %% 2 == 1)
ui <- fluidPage(dataTableOutput(outputId = "table"))
server <- function(input, output) {
  output$table <- renderDataTable({datatable(
    odd_even, filter = "top",
    options = list(searchCols = c(list(NULL), list(list(search = '["false"]'))))
  )})
  observe(print(input$table_rows_selected))
}
shinyApp(ui = ui, server = server)

err1Untitled

xfun::session_info('DT')

> xfun::session_info('DT')
R version 4.2.3 (2023-03-15 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19045), RStudio 2022.2.3.492

Locale: LC_COLLATE=English_United Kingdom.utf8  LC_CTYPE=English_United Kingdom.utf8    LC_MONETARY=English_United Kingdom.utf8 LC_NUMERIC=C                            LC_TIME=English_United Kingdom.utf8    

Package version:
  base64enc_0.1.3   bslib_0.5.1       cachem_1.0.8      cli_3.6.1         crosstalk_1.2.0   digest_0.6.29     DT_0.29.2         ellipsis_0.3.2    evaluate_0.21     fastmap_1.1.1     fontawesome_0.5.2
  fs_1.6.3          glue_1.6.2        graphics_4.2.3    grDevices_4.2.3   highr_0.10        htmltools_0.5.5   htmlwidgets_1.6.2 httpuv_1.6.5      jquerylib_0.1.4   jsonlite_1.8.7    knitr_1.44       
  later_1.3.0       lazyeval_0.2.2    lifecycle_1.0.3   magrittr_2.0.3    memoise_2.0.1     methods_4.2.3     mime_0.12         promises_1.2.0.1  R6_2.5.1          rappdirs_0.3.3    Rcpp_1.0.8.3     
  rlang_1.1.1       rmarkdown_2.25    sass_0.4.7        stats_4.2.3       stringi_1.7.12    stringr_1.5.0     tinytex_0.46      tools_4.2.3       utils_4.2.3       vctrs_0.6.3       xfun_0.40        
  yaml_2.3.7

Where is the problem?

As I understand the problem happens because of a race condition (see datatables.js Line 325 https://github.com/rstudio/DT/blob/main/inst/htmlwidgets/datatables.js#L325)

HTMLWidgets.widget({
  name: "datatables",
  type: "output",
  renderOnNullValue: true,
  initialize: function(el, width, height) {
    ...
  },
  renderValue: function(el, data, instance) {
    ...
    // use the dataSrc function to pre-process JSON data returned from R
    var DT_rows_all = [], DT_rows_current = [];
    if (server && HTMLWidgets.shinyMode && typeof options.ajax === 'object' &&
        /^session\/[\da-z]+\/dataobj/.test(options.ajax.url) && !options.ajax.dataSrc) {
      options.ajax.dataSrc = function(json) {
        DT_rows_all = $.makeArray(json.DT_rows_all);
        // !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
        DT_rows_current = $.makeArray(json.DT_rows_current);
        // !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
        var data = json.data;
        if (!colReorderEnabled()) return data;
        var table = $table.DataTable(), order = table.colReorder.order(), flag = true, i, j, row;
        for (i = 0; i < order.length; ++i) if (order[i] !== i) flag = false;
        if (flag) return data;
        for (i = 0; i < data.length; ++i) {
          row = data[i].slice();
          for (j = 0; j < order.length; ++j) data[i][j] = row[order[j]];
        }
        return data;
      };
    }
   ...
})

The line DT_rows_current = $.makeArray(json.DT_rows_current); get called 3 times, and depending on the internet speed the output can differ. (DT_rows_current is directly used for determining indexes of input$table_rows_selected)

Below you can see DT_rows_current values with slow and normal connection:

image

By filing an issue to this repo, I promise that

  • I have fully read the issue guide at https://yihui.org/issue/.
  • I have provided the necessary information about my issue.
    • If I'm asking a question, I have already asked it on Stack Overflow or RStudio Community, waited for at least 24 hours, and included a link to my question there.
    • If I'm filing a bug report, I have included a minimal, self-contained, and reproducible example, and have also included xfun::session_info('DT'). I have upgraded all my packages to their latest versions (e.g., R, RStudio, and R packages), and also tried the development version: remotes::install_github('rstudio/DT').
    • If I have posted the same issue elsewhere, I have also mentioned it in this issue.
  • I have learned the Github Markdown syntax, and formatted my issue correctly.

I understand that my issue may be closed if I don't fulfill my promises.

Dominant language
JavaScript
Stars
621
Forks
177
No merged PRs in 30d

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from rstudio/DT

All issues in rstudio/DT

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.