as.data.table() recurses without end on a survival::Surv object (or any data.frame carrying one)
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 88/100
Research direction
Start in R/as.data.table.R at as.data.table.default and the referenced data.frame/list dispatch sections around lines 169, 256-260, and 271-275. Run the supplied survival::Surv reproducer, then check the existing matrix method and regression cases listed in the issue. Done means Surv objects and data.frames containing Surv columns convert without recursion and the listed cases still pass.
Written by the indexing model from the issue text.
Description
Summary
as.data.table() never terminates on a Surv object, or on any data.frame that
carries a Surv column. It exhausts a stack limit instead. A Surv column is the
normal way survival data is carried, so this is reachable by an ordinary
as.data.table(df) on a survival dataset.
Reproducer
library(data.table) # 1.18.4
library(survival) # 3.8.11
s <- Surv(c(5, 10, 15), c(1, 0, 1))
as.data.table(s)
#> Error: node stack overflow
d <- data.frame(a = 1:3)
d$y <- s
as.data.table(d)
#> Error: node stack overflow
A plain matrix column is fine; the trigger is specifically a classed matrix whose
as.data.frame method returns a data.frame that still contains it:
d2 <- data.frame(a = 1:3); d2$m <- matrix(1:6, nrow = 3)
as.data.table(d2) # OK, 3 x 3
Cause
Surv is a matrix carrying class(x) == "Surv", so S3 dispatch selects
as.data.table.default, not as.data.table.matrix:
# R/as.data.table.R#9-11
as.data.table.default = function(x, ...){
as.data.table(as.data.frame(x, ...), ...)
}
as.data.frame(<Surv>) returns a one-column data.frame whose column is still a
Surv (survival keeps the class deliberately, as emmeans does in #6874):
str(as.data.frame(s))
#> 'data.frame': 3 obs. of 1 variable:
#> $ x: 'Surv' num [1:3, 1:2] 5 10+ 15
#> ..- attr(*, "dimnames")=List of 2
#> .. ..$ : NULL
#> .. ..$ : chr [1:2] "time" "status"
#> ..- attr(*, "type")= chr "right"
That data.frame's class is exactly "data.frame", so the guard added for #6874
at R/as.data.table.R#256-260 does not apply. Control then reaches
# R/as.data.table.R#271-275
if (any(cols_with_dims(x))) {
# a data.frame with a column that is data.frame needs to be expanded; test 2013.4
# x may be a class with [[ method that behaves differently, so as.list first for default [[, #4526
return(as.data.table.list(as.list(x), keep.rownames=keep.rownames, key = key,...))
}
and in as.data.table.list:
# R/as.data.table.R#169
xi = x[[i]] = as.data.table(xi, keep.rownames=keep.rownames) # we will never allow a matrix to be a column; always unpack the columns
which dispatches the Surv column straight back to as.data.table.default. Closed
loop.
Suggested patch
The atomic-vector methods (one chained definition, integer through ITime) already
carry an is.matrix() short-circuit at R/as.data.table.R#14-19;
as.data.table.default is the one method that lacks it:
as.data.table.default = function(x, ...){
+ if (is.matrix(x)) return(as.data.table.matrix(x, ...))
as.data.table(as.data.frame(x, ...), ...) # we cannot assume as.data.frame will do copy, thus setDT changed to as.data.table #3230
}
Applied to 1.18.4 and tested. The recursion is gone and the result is what the
matrix method would have produced all along:
as.data.table(s)
# time status
# <num> <num>
# 1: 5 1
# 2: 10 0
# 3: 15 1
as.data.table(d)
# a y.time y.status
# <int> <num> <num>
# 1: 1 5 1
# 2: 2 10 0
# 3: 3 15 1
Regression checks with the patch in place, all OK: data.frame, matrix, integer,
character, factor, list, table, Date, ts, NULL.
Related
- #6874 (emmeans) — same family, different shape. Its guard covers
as.data.frame(x)returning an object with the same class; hereas.data.frame
returns a genuinedata.frameand it is the column that keeps the class, so the
guard is never reached. - #6878 — closed as a duplicate of #6874.
- #1204 (open since 2015, labelled
bug/non-atomic column) —Survcolumns in
data.table. Different symptom ([.Surverror), same underlying "non-atomic column"
area; linked for cross-reference.
I am aware of the position taken in #6874 that a looping as.data.frame method is the
downstream package's problem. Two reasons this one is worth guarding anyway: Surv
comes from a Recommended package, and its class-preserving as.data.frame is
deliberate and long-standing; and data.table's own vector methods already carry the
one-line guard that would prevent it.
Session info
R version 4.6.1 (2026-06-24 ucrt)
Platform: x86_64-w64-mingw32/x64
Running under: Windows 11 x64 (build 26200)
packages: data.table 1.18.4, survival 3.8.11
- Dominant language
- R
- Stars
- 3.9k
- Forks
- 1.1k
- Avg merge
- 14h 4m
- Merged PRs (30d)
- 4
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from Rdatatable/data.table
-
consistency tests
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Rdatatable/data.table#7853 · 3 comments ·
-
internals
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
Rdatatable/data.table#6938 · 1 comment ·
-
encoding fread
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
Rdatatable/data.table#5179 · 8 comments ·
-
documentation programming
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Rdatatable/data.table#3199 · 3 comments ·
-
bug fread
Difficulty 3/5 1-2 days Newbie friendliness 68/100
Rdatatable/data.table#7896 · 1 reaction ·
All issues in Rdatatable/data.table
Similar issues
-
documentation pkg infrastructure
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
epiverse-trace/epiparameter#511 ·
-
function:write_dwc
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Urgent request: Due to vulnerabilities move to API version 12.6.1 (12.6.2 eventually) or 13.1.1 Open
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
jbkunst/highcharter#849 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
r-lib/pkgdepends#485 · 3 comments ·
-
Difficulty 1/5 Under an hour Newbie friendliness 92/100