as.data.table() recurses without end on a survival::Surv object (or any data.frame carrying one)

Open Beginner friendly
#7,887 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
88/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
r
Domain
data

Research direction

Start in R/as.data.table.R at as.data.table.default and the referenced data.frame/list dispatch sections around lines 169, 256-260, and 271-275. Run the supplied survival::Surv reproducer, then check the existing matrix method and regression cases listed in the issue. Done means Surv objects and data.frames containing Surv columns convert without recursion and the listed cases still pass.

Written by the indexing model from the issue text.

Description

Summary

as.data.table() never terminates on a Surv object, or on any data.frame that
carries a Surv column. It exhausts a stack limit instead. A Surv column is the
normal way survival data is carried, so this is reachable by an ordinary
as.data.table(df) on a survival dataset.

Reproducer
library(data.table)   # 1.18.4
library(survival)     # 3.8.11

s <- Surv(c(5, 10, 15), c(1, 0, 1))

as.data.table(s)
#> Error: node stack overflow

d <- data.frame(a = 1:3)
d$y <- s
as.data.table(d)
#> Error: node stack overflow

A plain matrix column is fine; the trigger is specifically a classed matrix whose
as.data.frame method returns a data.frame that still contains it:

d2 <- data.frame(a = 1:3); d2$m <- matrix(1:6, nrow = 3)
as.data.table(d2)     # OK, 3 x 3
Cause

Surv is a matrix carrying class(x) == "Surv", so S3 dispatch selects
as.data.table.default, not as.data.table.matrix:

# R/as.data.table.R#9-11
as.data.table.default = function(x, ...){
  as.data.table(as.data.frame(x, ...), ...)
}

as.data.frame(<Surv>) returns a one-column data.frame whose column is still a
Surv
(survival keeps the class deliberately, as emmeans does in #6874):

str(as.data.frame(s))
#> 'data.frame':	3 obs. of  1 variable:
#>  $ x: 'Surv' num [1:3, 1:2]  5  10+ 15
#>   ..- attr(*, "dimnames")=List of 2
#>   .. ..$ : NULL
#>   .. ..$ : chr [1:2] "time" "status"
#>   ..- attr(*, "type")= chr "right"

That data.frame's class is exactly "data.frame", so the guard added for #6874
at R/as.data.table.R#256-260 does not apply. Control then reaches

# R/as.data.table.R#271-275
if (any(cols_with_dims(x))) {
  # a data.frame with a column that is data.frame needs to be expanded; test 2013.4
  # x may be a class with [[ method that behaves differently, so as.list first for default [[, #4526
  return(as.data.table.list(as.list(x), keep.rownames=keep.rownames, key = key,...))
}

and in as.data.table.list:

# R/as.data.table.R#169
xi = x[[i]] = as.data.table(xi, keep.rownames=keep.rownames)  # we will never allow a matrix to be a column; always unpack the columns

which dispatches the Surv column straight back to as.data.table.default. Closed
loop.

Suggested patch

The atomic-vector methods (one chained definition, integer through ITime) already
carry an is.matrix() short-circuit at R/as.data.table.R#14-19;
as.data.table.default is the one method that lacks it:

 as.data.table.default = function(x, ...){
+  if (is.matrix(x)) return(as.data.table.matrix(x, ...))
   as.data.table(as.data.frame(x, ...), ...) # we cannot assume as.data.frame will do copy, thus setDT changed to as.data.table #3230
 }

Applied to 1.18.4 and tested. The recursion is gone and the result is what the
matrix method would have produced all along:

as.data.table(s)
#     time status
#    <num>  <num>
# 1:     5      1
# 2:    10      0
# 3:    15      1

as.data.table(d)
#        a y.time y.status
#    <int>  <num>    <num>
# 1:     1      5        1
# 2:     2     10        0
# 3:     3     15        1

Regression checks with the patch in place, all OK: data.frame, matrix, integer,
character, factor, list, table, Date, ts, NULL.

Related
  • #6874 (emmeans) — same family, different shape. Its guard covers
    as.data.frame(x) returning an object with the same class; here as.data.frame
    returns a genuine data.frame and it is the column that keeps the class, so the
    guard is never reached.
  • #6878 — closed as a duplicate of #6874.
  • #1204 (open since 2015, labelled bug / non-atomic column) — Surv columns in
    data.table. Different symptom ([.Surv error), same underlying "non-atomic column"
    area; linked for cross-reference.

I am aware of the position taken in #6874 that a looping as.data.frame method is the
downstream package's problem. Two reasons this one is worth guarding anyway: Surv
comes from a Recommended package, and its class-preserving as.data.frame is
deliberate and long-standing; and data.table's own vector methods already carry the
one-line guard that would prevent it.

Session info
R version 4.6.1 (2026-06-24 ucrt)
Platform: x86_64-w64-mingw32/x64
Running under: Windows 11 x64 (build 26200)

packages: data.table 1.18.4, survival 3.8.11
Dominant language
R
Stars
3.9k
Forks
1.1k
Avg merge
14h 4m
Merged PRs (30d)
4

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Rdatatable/data.table

All issues in Rdatatable/data.table

Similar issues

More R issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.