Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

diagnose: the error argument is sent as epsilon; the output directory is never removed

Open Beginner friendly
  • Is something wrong?
#871 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
62/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
tooling

Research direction

The bug is in cmdstanpy/model.py, in CmdStanModel.diagnose: the error branch appends epsilon= instead of error=, and the output directory from tempfile.mkdtemp under _TMPDIR is never removed. Start by reading the log_prob tests, which check the logged command and are the model for a new test. Done means the command carries error=, and the entry count under _TMPDIR is unchanged after calls on both the success and failure paths. Check with #869 first, since the issue says the cleanup approach was agreed there.

Written by the indexing model from the issue text.

Description

Summary

CmdStanModel.diagnose (cmdstanpy develop at the time of writing, also 1.3.0) has two bugs:

  1. The error argument is sent as epsilon. CmdStan never receives error=, and when both arguments are given, epsilon appears twice on the command line.
  2. Each call leaves a temporary output directory under _TMPDIR, removed only at interpreter exit. This is the same pattern as #867, which is being fixed for log_prob in #869.
Where

cmdstanpy/model.py, CmdStanModel.diagnose, on develop:

if epsilon is not None:
    cmd.append(f"epsilon={epsilon}")
if error is not None:
    cmd.append(f"epsilon={error}")      # should be f"error={error}"
...
output_dir = tempfile.mkdtemp(prefix=self.name, dir=_TMPDIR)   # never removed

CmdStan 2.40.0's argument tree accepts both sub-arguments under diagnose test=gradient (output of <model> diagnose help-all):

gradient
  Valid subarguments: epsilon, error
  epsilon=<double>   Finite difference step size     Defaults to 1e-6
  error=<double>     Error threshold                 Defaults to 1e-6
Effect
  • diagnose(error=x) silently sets the finite-difference step to x, and the error threshold stays at its default 1e-6. The gradient check then judges against a threshold the caller did not ask for, using a step size the caller did not ask for.
  • diagnose(epsilon=a, error=b) passes epsilon=a epsilon=b.
  • The output directory accumulates in long-running processes, as described in #867.
Suggested fix
  • Send f"error={error}".
  • Scope the output directory with tempfile.TemporaryDirectory(prefix=self.name, dir=_TMPDIR), the approach agreed on #869.
  • Add a test asserting that the command carries error= (the logged command, as the log_prob tests do) and that _TMPDIR's entry count is unchanged after calls on both the success path and the failure path.

We can fold this into #869, or open a separate PR, whichever you prefer.

Versions

cmdstanpy develop (also 1.3.0), CmdStan 2.40.0, Python 3.12, Linux x86_64.

Dominant language
Python
Stars
198
Forks
82
Avg merge
4d 11h
Merged PRs (30d)
3

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from stan-dev/cmdstanpy

All issues in stan-dev/cmdstanpy

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.