Description
On a Unison codebase I have locally, which is 200MB on disk, I'm seeing unison use 557MB of memory. I suspect there's something really silly going on here. A good clue is if I start Unison on an empty codebase, it uses 4MB of memory for me. So presumably there's something about having a full namespace tree that's inefficient... the history of the namespace shouldn't be relevant as only the current point in history is loaded into memory.
I'm not sure what's going on, so just jotting some ideas down.
May become less of an issue once #1340 is implemented.
Notes
The codebase itself has just 1,147 total term and type hashes:
🌻 ls -1 ~/.unison/v1/terms | wc -l
1086
🌻 ls -1 ~/.unison/v1/types | wc -l
61
Now for some rough estimates just to establish that memory usage seems off by an order of magnitude:
- Each such hash is 512 bits = 64 bytes. So just storing these is 73K, call it an even 80K.
- The namespace tree also has hashes for all the namespaces. Rough estimate: there might be a separate namespace for every definition. So say that doubles the storage to 160K.
- There are also names for these hashes. Suppose names are on average 64 bytes. Say there's 2 names for every hash. That's another 160K, bringing us to 420K.
- Now, the
Branchalso keeps a bunch of pointers to these values. Say there's 10 pointers for each "value" (where value is a namespaces, term refs, type refs, or name). That's 5000 * 8 * 10 = 400K. Say there's 100 pointers for each value... that's 4MB. - What's missing here?
So seems like we might be off by an order of magnitude or two... probably a sign there's something silly happening.