[Bug]: [Python] WriteToJson silently rounds floats to 10 decimal places

Open Beginner friendly
  • Is something wrong?
#41,102 4 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@Abhi2939 is already working on this.

Since Nov 10, 2026.

  • #41154 by @Abhi2939 — open

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
62/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
json, pandas, python, yaml
Domain
data-engineering

Research direction

The rounding comes from the DataFrame path: WriteToJson in sdks/python/apache_beam/io/textio.py forwards kwargs to apache_beam.dataframe.io.to_json, which calls pandas to_json whose double_precision defaults to 10. Start by reproducing with the YAML pipeline in the issue, then check how textio.py builds the to_json call. Done means floats round-trip (a default double_precision or a json.dumps path for lines output) and a test in the textio or dataframe io tests covers 0.1234567890123.

Written by the indexing model from the issue text.

Description

io P2 python yaml
What happened?

beam.io.WriteToJson (and Beam YAML WriteToJson) silently loses float precision: values are rounded to 10 decimal places.

input written
0.1234567890123 0.123456789
3.3333333333333335 3.3333333333
12345678901.5 12345678901.5

There is no warning, so a JSON export doesn't round-trip and differs from the same data written with WriteToParquet, WriteToCsv or json.dumps.

Minimal reproducer (Beam YAML, Beam 2.75.0; WriteToJson on master @ 728ef1b4 has the same code path):

pipeline:
  type: chain
  transforms:
  - type: Create
    config:
      elements:
      - {id: 1, x: 0.1234567890123}
      - {id: 2, x: 3.3333333333333335}
  - type: WriteToJson
    config:
      path: /tmp/c5

python -m apache_beam.yaml.main --yaml_pipeline_file=c5.yaml writes:

{"id":1,"x":0.123456789}
{"id":2,"x":3.3333333333}
Cause

WriteToJson (sdks/python/apache_beam/io/textio.py) writes through the DataFrame API, apache_beam.dataframe.io.to_json -> pandas.DataFrame.to_json, and only forwards the caller's **kwargs. pandas' to_json defaults to double_precision=10, so every float is rounded to 10 decimal places.

Options:

  1. Default double_precision=15 (pandas' maximum) when the caller doesn't pass it. That fixes most cases but still isn't exact for 16-17 significant digits.
  2. Write JSON lines with json.dumps for orient='records', lines=True (the YAML default), which round-trips floats exactly.

At minimum the docstring should mention the rounding and the double_precision kwarg. Happy to send a PR for whichever approach maintainers prefer.

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components
  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam YAML
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Infrastructure
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Prism Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner
Dominant language
Java
Stars
8.7k
Forks
4.8k
Avg merge
2d 9h
Merged PRs (30d)
368

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/beam

All issues in apache/beam

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.