[BUG] CEA-708 .srt is written as Latin-1 and ignores --utf8/--latin1/--unicode
Maintainers usually reply within 7 days
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 65/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Domain
- backend, internationalization
Research direction
Start by reading src/rust/src/decoder/output.rs and the C writer ccx_decoders_708_output.c, then trace how global encoding flags reach the 608 writer and the 708 writer. Test a CEA-708 stream containing accented G1 characters with default settings and --utf8, --latin1, and --unicode; done means 708 output follows the selected encoding, with UTF-8 as the documented default.
Written by the indexing model from the issue text.
Description
CCExtractor version: 0.96.6 release AppImage (--version prints 0.96.5, commit 185631dc). The code below is unchanged on master (d7eb132).
Necessary information
- Is this a regression (i.e. did it work before)? Don't know
- What platform did you use? Linux
- What were the used arguments?
-o out.srt(same result with--service 1)
Additional information
The CEA-708 .srt is written as raw Latin-1 bytes, while --help says UTF-8 is the default ("--utf8: Encode subtitles in UTF-8 (no longer needed because UTF-8 is now the default)"). --utf8, --latin1 and --unicode don't change the 708 file at all.
On an MXF with 608 and 708 captions in Portuguese, one run gives a UTF-8 608 file and a Latin-1 708 file:
$ ccextractor in.mxf -o out.srt
$ file out.srt out.p0.svc01.srt
out.srt: Unicode text, UTF-8 text, with CRLF line terminators
out.p0.svc01.srt: ISO-8859 text, with CRLF line terminators
"você" in the 708 file is 76 6f 63 ea, the same with no flag, --utf8, --latin1 or --unicode. Only a per-service charset gives UTF-8: --service "1[ISO-8859-1]" writes 76 6f 63 c3 aa.
From the code, the 708 writer never looks at the global encoding. With no service charset there is no iconv descriptor, and write_char in src/rust/src/decoder/output.rs (and the C writer in ccx_decoders_708_output.c) writes the internal symbol bytes as they are, so G1 characters come out as their Latin-1 byte. I expected the 708 output to follow --utf8/--latin1/--unicode like the 608 output does, with UTF-8 as the default.
The input is a 395 KB MXF that I can share if it helps. Any 708 stream with G1 characters (accented letters) should show the same thing.
- Dominant language
- C
- Stars
- 913
- Forks
- 603
- Avg merge
- 6d 8h
- Merged PRs (30d)
- 33
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from CCExtractor/ccextractor
-
[BUG] MXF: update_tid_lut() writes past tracks[32] onto nb_tracks, turning a file-supplied track id into the write offsetPossibly taken @pranayr710 claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
CCExtractor/ccextractor#2457 ·
Maintainers usually reply within 7 days
-
[Security] Heap buffer overflow (OOB read past GPAC sample buffer) in process_avc_sample() at mp4.c:99 when parsing a crafted MP4 filePossibly taken @yasar710 claimed this 6 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
CCExtractor/ccextractor#2449 ·
Maintainers usually reply within 7 days
-
[BUG] DVB subtitle decoder: heap-buffer-overflow write in dvbsub_parse_clut_segment via unbounded entry_idPossibly taken @Aditya-Saroha claimed this 9 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
CCExtractor/ccextractor#2440 ·
Maintainers usually reply within 7 days
-
[BUG] Filename sequence expansion (`+`) panics on numeric values exceeding `i32::MAX`May be free again A pull request for this issue was closed without being merged. Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
CCExtractor/ccextractor#2402 ·
Maintainers usually reply within 7 days
-
[Security] NULL pointer dereference (SEGV at address 0x0) in sei_message() when parsing a crafted MP4 filePossibly taken @yasar710 claimed this 8 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 50/100
CCExtractor/ccextractor#2444 ·
Maintainers usually reply within 7 days
All issues in CCExtractor/ccextractor
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
tursodatabase/libsql#2312 ·
Maintainers usually reply within 11 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
follow-up Prio 📌
Difficulty 1/5 Under an hour Newbie friendliness 84/100
Maintainers usually reply within 1 day
-
area: test kind: bug preview-effort: small priority: low
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
Maintainers usually reply within 1 day