[BUG] `aggregate_components` silently drops components with NaN in a grouping column

Open Beginner friendly
#870 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
80/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
pandas, python
Domain
data

Research direction

Read edisgo/edisgo.py around line 1885 and the named regression test, tests/properties/test_properties_edisgo.py::TestDataHandling::test_aggregate_components_keeps_loads_with_missing_sector. Check how component and time series grouping handle missing keys, then run the regression test. Done means loads with a NaN sector remain in the aggregated data and the test passes with power flow v_res and i_res unchanged.

Written by the indexing model from the issue text.

Description

bug correctness priority: medium

Summary

EDisGo.aggregate_components groups components with groupby(...).sum(), which drops groups with NaN keys (dropna=True). A load with sector = NaN and aggregate_loads_by_cols=["bus", "sector"] disappears from loads_df: p_set at the bus drops from 0.945 to 0.635 MW in test grid 1. The time series aggregation then iterates over groupby.groups, which still contains the NaN group, and raises TypeError.

Where

edisgo/edisgo.py:1885 aggregate_components.

How to reproduce

Test (strict xfail, added in #856): tests/properties/test_properties_edisgo.py::TestDataHandling::test_aggregate_components_keeps_loads_with_missing_sector

Passing: aggregate_components keeps v_res and i_res of the power flow unchanged.

Proposed fix

Use groupby(..., dropna=False) (and the same for the time series), or fill NaN in the grouping columns with a placeholder before grouping.

Related

#728 (load_aggregation_mode="bus" erases type).

Dominant language
Python
Stars
48
Forks
22
Avg merge
1d 7h
Merged PRs (30d)
38

Getting set up

  • No Dockerfile or Docker Compose file
  • Has a pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from openego/eDisGo

All issues in openego/eDisGo

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.