Compressing the discovery cache
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 48/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- python
- Domain
- build-system, performance
Research direction
Start by locating the code that creates and reads the discovery cache's documents directory, using the v2.114.0 cache size as a baseline. Compare the available Python standard-library compression approaches and check how the cache is packaged into Docker images. Done means the cache is substantially smaller while remaining usable by the library and improving build or deploy size.
Written by the indexing model from the issue text.
Description
#2315 shaved off a bit more than 20MB of the discovery cache, by removing the json indentation.
Currently there's still 74.6MB remaining, and it appears to be growing steadily over time (from 81MB on 2023-03-31 to 94MB on 2024-01-15).
So while #2315 definitely helped, I believe that it's a good idea to consider reducing the size even further. I believe that this could significantly improve build/deploy times of the many docker images that use this library. For reference, the current latest python-slim docker image is less than 50MB.
An easy win would be to use compression.
To illustrate: if I manually zip the entire documents directory of 74.6MB in v2.114.0 using the pop-os default gnome archive manager, I end up with an archive of 11.9MB.
Creating a documents.tar.xz in the same way makes this 4.6MB.
It should be possible to achieve similar levels of compression by using Python's standard compression libraries, e.g. zlib, zipfile, or lzma.
- Dominant language
- Python
- Stars
- 9k
- Forks
- 2.6k
- Avg merge
- 28m
- Merged PRs (30d)
- 4
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from googleapis/google-api-python-client
-
fix: @_helpers.positional raises TypeError when decorating functions without default argumentsPossibly taken @codeXsidd claimed this 22 days ago. Open
Difficulty 1/5 Under an hour Newbie friendliness 88/100
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
-
Disable prettyPrint by default for Google Sheets API responsesPossibly taken @pawciobiel claimed this 91 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
test: connection leak test test_discovery_http_is_closed is shadowed and syntactically invalidPossibly taken @Capstan claimed this 167 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
googleapis/google-api-python-client#2757 · 1 comment ·
-
test: strict positional parameter enforcement is ignored under pytestPossibly taken @Capstan claimed this 167 days ago. Open
Difficulty 1/5 Under an hour Newbie friendliness 92/100
All issues in googleapis/google-api-python-client
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
cpp-best-practices/project_validator#267 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Project-N-E-K-O/N.E.K.O#3725 ·
Maintainers usually reply within 1 day
-
[Bug] Sending a Telegram message chain twice duplicates mentionsPossibly taken @123XieC claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
AstrBotDevs/AstrBot#11125 ·
Maintainers usually reply within 1 day
-
backend bug priority: high
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
nooriegarg/ContribIQ#73 ·
Maintainers usually reply within 1 day
-
bug p3 security
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Maintainers usually reply within 1 day