Predict fails with "cannot allocate vector of size..." with large number of prediction points
#215 opened on Jun 8, 2017
Repository metrics
- Stars
- (17,686 stars)
- PR merge metrics
- (No merged PRs in 30d)
Description
Hi, I am currently trying to use Prophet to predict byte flows in a network. The data set has the following properties:
- 4 periods (1 period = 1 week)
- minute granularity
- 44355 data points
The first couple of lines look as follows, with y being the number of bytes monitored in the network:
| # | ds | y |
|---|---|---|
| 1 | 2016-08-18 19:00:00 | 8476391449 |
| 2 | 2016-08-18 19:01:00 | 6432109555 |
| 3 | 2016-08-18 19:02:00 | 4039378069 |
| 4 | 2016-08-18 19:03:00 | 4124949796 |
| 5 | 2016-08-18 19:04:00 | 6911448995 |
| 6 | 2016-08-18 19:05:00 | 4021931960 |
I wrote some simple R code to fit the model and predict a single period:
fit <- prophet(data)
future <- make_future_dataframe(fit, periods = 1)
forecast <- predict(fit, future)
plot(fit, forecast)
The model fitting works, however, during the prediction, R tries to allocate more and more memory until both Mem and Swp are fully allocated (I have a machine with 32GB of RAM, Debian 8.8 Jessie, R 3.3.3). Meanwhile, a single CPU core is at 100% load. The program will eventually crash with the following error:
Error: cannot allocate vector of size 242.4 Mb
Execution halted
Warning message:
system call failed: Cannot allocate memory
I've done the same thing with the Holt-Winters prediction that's built into R already which works flawlessly and uses around 300MB of RAM.
Any ideas what the problem is here? Could it be the granularity?