Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Importing certain databases from zip are extremely slow

Open
#776 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
typescript, vscode
Domain
tooling

Research direction

Start by tracing the database import path and the current streaming unzipper, then compare file-header reading with central-directory lookup for archives containing many small files. Done means large database archives import substantially faster while still handling archives with incorrect file headers.

Written by the indexing model from the issue text.

Description

bug VSCode

I can't share the database because it is private, but this database is less than 1GB zipped and about 3.4GB unzipped. Importing this database took over 1.5 hours to completion. I think the problem has to do with the fact that there are over 100,000 source files. First, these files need to be unzipped and placed in a directory. Then we need to re-zip them into a src.zip file. Our current unzipper library uses streaming, but it does read each file into memory separately.

It's possible that the slowness is exacerbated by the fix here: https://github.com/github/vscode-codeql/issues/622. Rather than read the zip in a single pass, we read the zips central directory and then grab each file based on what we find there.

I implemented this fix because some archives do not have correct file headers. The central directory is the source of truth and it exists at the end of the file. This happens when a zip file is updated after it is created.

Most of the time reading the file headers will be correct (except when they aren't). And that is likely faster than the central directory approach, especially when there are lots of small files. So, one possible solution would be to try reading via the file headers first and if that fails, fall back to the central directory.

Dominant language
TypeScript
Stars
539
Forks
240
Avg merge
2d 6h
Merged PRs (30d)
29

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from github/vscode-codeql

All issues in github/vscode-codeql

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.