Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Let fits compress their own output CSVs

Open
  • Is something wrong?
#1,276 1 comment 0 reactions 1 assignee View on GitHub

Maintainers usually reply within 1 day

@VisruthSK is already working on this.

Since Sep 27, 2026.

  • #1311 by @VisruthSK — open

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
42/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
r
Domain
tooling

Research direction

Trace the fitting methods such as $sample() and $optimize(), along with $save_output_files(), and review how read_cmdstan_csv() and as_cmdstan_fit() store and consume paths. Define how the compress option should update fit-owned files, including latent dynamics, and how commands such as $cmdstan_summary(), $cmdstan_diagnose(), generate_quantities(), and laplace() handle compressed paths. Done should include working gzip and bzip2 output plus clear handling or documentation for affected commands and performance requirements.

Written by the indexing model from the issue text.

Description

feature

#1027/#1217 make read_cmdstan_csv() and as_cmdstan_fit() read gzip and bzip2 compressed CSVs, but there's no good way to get a fit object's own files compressed. If you gzip the files yourself, the fitted model object still points at the plain .csv paths and fit$draws() will break. When #1217 is merged, the advice for users is basically "finish with the fit, compress, then rebuild it with as_cmdstan_fit()", which is awkward.

But we could add a compress = c("none", "gzip", "bzip2") argument in two places to make this easier for users:

  • The fitting methods ($sample(), $optimize(), etc.), alongside output_dir and output_basename. Once CmdStan finishes, cmdstanr compresses the output files in place with gzfile()/bzfile() and records the new paths, so the fit works as usual with no extra call. I guess latent dynamics files should be compressed too when save_latent_dynamics = TRUE. We can probably skip small files (profile, metric, config).
  • $save_output_files() should support compressing after the fact.

This would break anything that gives file paths to a CmdStan binary like $cmdstan_summary(), $cmdstan_diagnose(), generate_quantities(fitted_params = fit), and laplace(mode = fit). Once #1217 is merged this already happens for a fit built with as_cmdstan_fit() from compressed files, since the fit keeps the .csv.gz paths and CmdStan rejects them. We could either error in those cases or we could decompress to a temp file when needed. Or maybe there's another option. Something to think about.

A few other things that aren't ideal, but can just be documented clearly so users are aware:

  • CmdStanMCMC reads the sampler diagnostics right after sampling to print warnings, so that first read requires a decompression. We could avoid by compressing after the fit is build (instead of right after run_cmdstan()) but that's a more complicated implementation and I think we should keep it simpler.
  • Every read of a compressed file is slower than the plain CSV. Just need to document this so users keep in mind the tradeoff.

I think reading in the compressed files needs gzip/bzip2 on the PATH (from RTools on Windows)

Dominant language
R
Stars
161
Forks
69
Avg merge
1d 12h
Merged PRs (30d)
45

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from stan-dev/cmdstanr

All issues in stan-dev/cmdstanr

Similar issues

More R issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.