Check that BigQuery `interned_strings` survives staging retention on devnet and mainnet
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 75/100
Research direction
Wait until at least 8 days after the one-time interned_strings backfill finishes on each network. Run the supplied BigQuery count query for each listed project and dataset, then compare its n, ids and max_id with the Postgres count through max_id. The check is complete when all three counts are equal on both devnet and mainnet.
Written by the indexing model from the issue text.
Description
Part of #7553. Confirms #8219's append keeps every interned_strings row in production after staging purges it.
When: 8 or more days after the one-time interned_strings backfill finishes on a network. Staging retention is 7 days there.
| network | project | prod dataset |
|---|---|---|
| devnet | da-cn-devnet |
devnet_da2_scan_prod |
| mainnet | da-cn-mainnet |
mainnet_da2_scan_prod |
For each network:
- Production, with
<project>and<dataset>from the table:bq query --project_id=<project> --use_legacy_sql=false \ 'SELECT count(*) AS n, count(DISTINCT id) AS ids, max(id) AS max_id FROM `<project>.<dataset>.scan_sv_1_interned_strings`' - Scan's Postgres, using
max_idfrom step 1. Rows newer than the last hourly append aren't in production yet, so this ignores them:cncluster psql sv-1 scan-appSELECT count(*) FROM scan_sv_1.interned_strings WHERE id <= <max_id>;
Pass: n, ids and the Postgres count are all equal.
- Dominant language
- Scala
- Stars
- 118
- Forks
- 125
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 379
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from canton-network/splice
-
BigQuery
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
canton-network/splice#8220 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
canton-network/splice#8087 ·
Maintainers usually reply within 1 day
-
Fix list of SVs in the SV UI historical proposal detailsMay be free again @loglapa claimed this 18 days ago, and no pull request is open. Open
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
canton-network/splice#7939 · 2 comments ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
canton-network/splice#6641 · 5 comments ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
canton-network/splice#4424 ·
Maintainers usually reply within 1 day
All issues in canton-network/splice
Similar issues
-
Difficulty 1/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 3 days
-
bug container
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Native MergeRows can panic or produce wrong masks - arrow 59 `and_not` mishandles sliced arraysOpenrequires-triage
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
apache/datafusion-comet#7596 ·
Maintainers usually reply within 2 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
lichess-org/lila#22466 · 1 comment ·
Maintainers usually reply within 1 day
-
30-minutes algorithm any-language codewars dart easy elixir hackerrank hacktoberfest haskell help wanted interview-prep julia leetcode lua open-source r scala students zig
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
Huerte/algo-solutions-python#226 ·
Maintainers usually reply within 1 day