links table: 20,375 unreachable rows left over from the 2021 prefix-length change

Open
#2,292 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
45/100
Issue type
Refactor
Clarity
Mostly clear
Activity status
Quiet
Domain
database

Research direction

No source file or test is named. Start by reviewing the links table and the lookup behavior described here, then verify the 9-character-row filter and the matching 6-character twins before changing data. Done means preserving the 84,152 clicks, removing only unreachable rows, and confirming normal short-link lookups remain intact.

Written by the indexing model from the issue text.

Description

Written by Claude (an LLM), at Matt's request. Low priority, and closing this as wontfix is a perfectly reasonable outcome.

The links table has 20,375 rows that nothing can ever read.

In March 2021 MIN_STORED_ID_LENGTH went from 6 to 9 (6fcf5d758), which made the DynamoDB prefix key 9 characters and broke lookups of existing links. It was reverted on 2021-04-09 (a194b5a59), which is where the PREFIX_LENGTH = 6 "NEVER CHANGE THIS VALUE" comment comes from. Rows written during those three weeks have a 9-character prefix.

Lookups build the key as prefix = id[0:6], so a 9-character prefix can never match. Each of the 20,375 has a 6-character twin with the same unique_subhash and full_hash (verified for all of them), and that twin is what serves the link.

Short links from that window still work. Nothing is broken.

Worth doing?

Storage is about 4 MB of a 440 MB table, so the saving is roughly a penny. The only real benefit is not having to rediscover this: the rows inflate 2021 by 6.6% in any count-by-year, which already sent one investigation down a blind alley.

If we do clean up, the 9-character rows hold 84,152 clicks from the weeks they were live, which the twins don't have. Folding those into the 6-character rows first would avoid losing the history.

Safe filter if anyone does it: delete only where length(prefix) == 9. No legitimate row can have that. PITR is on.

Dominant language
Python
Stars
439
Forks
454
Avg merge
1d 17h
Merged PRs (30d)
78

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from compiler-explorer/infra

All issues in compiler-explorer/infra

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.