ciphers/base32.py and ciphers/base85.py encode Unicode code points instead of bytes: output diverges from the spec for non-ASCII, round-trips corrupt chars >= U+0400, arbitrary bytes crash

Open Beginner friendly
  • Is something wrong?
#15,636 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 2 days

@Nate-br is already working on this.

Since Oct 26, 2026.

  • #15646 by @Nate-br — closed without merging
  • #15647 by @Nate-br — open
  • #15648 by @Nate-br — open

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
88/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
developer-experience

Research direction

Read ciphers/base32.py:19 and ciphers/base85.py:25, where the bit string is built from data.decode("utf-8") and bin(ord(d)) instead of raw bytes. Compare with ciphers/base16.py and base64_cipher.py, which already operate on bytes correctly. Fix both encode and decode sides to consume and produce bytes, then verify with base64.b32encode/base64.a85encode and the reproduction script in the issue; done means byte-for-byte parity with the standard library and lossless round-trips for non-UTF-8 input, with doctests extended beyond ASCII.

Written by the indexing model from the issue text.

Description

Repository commit

6e03a48579ca212a08b36be00378f887dc41a003

Python version (python --version)

Python 3.13.5

Dependencies version (pip freeze)

None needed — both modules only use the standard library.

Expected behavior

Base32 and Ascii85 are byte encodings: base32_encode(data) should return the RFC 4648 encoding of the input bytes (matching base64.b32encode), and ascii85_encode(data) should match base64.a85encode. They should accept arbitrary bytes, as the type hints (data: bytes) suggest. The sibling modules base16.py and base64_cipher.py do exactly this and match the standard library for every input.

Actual behavior

ciphers/base32.py and ciphers/base85.py start with data.decode("utf-8") and then encode Unicode code points (bin(ord(d))) instead of bytes. All three consequences show up as soon as the input is not pure ASCII:

  1. Output diverges from the standard for non-ASCII text:
>>> from ciphers.base32 import base32_encode
>>> import base64
>>> base32_encode("héllo".encode())
b'NDUWY3DPA======='
>>> base64.b32encode("héllo".encode())
b'NDB2S3DMN4======'   # correct RFC 4648 result
  1. Encode→decode silently corrupts data for characters ≥ U+0400 (their code point is wider than 8 bits, shifting every following bit):
>>> from ciphers.base32 import base32_encode, base32_decode
>>> base32_decode(base32_encode("€100".encode()))
b'\xc2\x82\xc2\xb0\xc3\x84\xc3\x80\xc3\x80'   # should round-trip to b'\xe2\x82\xac100'

Same for ascii85_decode(ascii85_encode("€100".encode())) → b'\xc2\x82\xc2\xb0\xc3\x84\xc3\x80\xc3\x80'.

  1. Arbitrary non-UTF-8 bytes — perfectly valid input for a byte codec — raise UnicodeDecodeError:
>>> base32_encode(bytes([0xff, 0xfe, 0x01]))
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0

The doctests only use ASCII strings, so the test suite never exercises any of this.

Steps to reproduce

python - <<'EOF'
import base64
from ciphers.base32 import base32_encode, base32_decode
from ciphers.base85 import ascii85_encode, ascii85_decode
for data in ["héllo".encode(), "€100".encode(), bytes([0xff, 0xfe, 0x01])]:
    print(data)
    try: print("  b32:", base32_encode(data), "| stdlib:", base64.b32encode(data))
    except Exception as e: print("  b32 crash:", repr(e))
    try: print("  b32 roundtrip ok:", base32_decode(base32_encode(data)) == data)
    except Exception as e: print("  b32 roundtrip crash:", repr(e))
    try: print("  a85 match:", ascii85_encode(data) == base64.a85encode(data))
    except Exception as e: print("  a85 crash:", repr(e))
EOF

Suspected cause

ciphers/base32.py:19 and ciphers/base85.py:25 both build the bit string from "".join(bin(ord(d))[2:].zfill(8) for d in data.decode("utf-8")). Operating on the raw bytes (bin(byte)[2:].zfill(8) for byte in data, like base16.py does) and assembling the decode side from byte values instead of chr(int(chunk, 2)) would keep both codecs on-spec and lossless for every input.

Dominant language
Python
Stars
226k
Forks
51.3k
Avg merge
1d 13h
Merged PRs (30d)
32

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from TheAlgorithms/Python

All issues in TheAlgorithms/Python

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.