Investigate memory footprint

Open
#153 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
25/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
python
Domain
backend, performance

Research direction

Start by reproducing the failure during the “run” step with 16 slaves on one 16 GB node and 80 total cores in the bbp viz cloud partition. Investigate whether memory is concentrated on the master node, identify which data is held in memory, and measure the footprint before and after reducing loaded data. Done means the run completes within the available memory and the relevant measurements are documented.

Written by the indexing model from the issue text.

Description

It seems BluePyMM can't run on nodes with 16 GB of RAM (for 16 slaves on same node, job with 80 cores in total, bbp viz cloud partition). Happens during the 'run' step.
This might be a problem at the master node, we probably could lower the memory footprint a bit by loading less data in memory.

Dominant language
Python
Stars
13
Forks
9
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from BlueBrain/BluePyMM

All issues in BlueBrain/BluePyMM

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.