make_cat: objects ngmix never fit get NGMIX sentinels but MCAL_FLAGS = 0
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 84/100
Research direction
Start in make_cat.py around lines 474, 494, and 501, then trace how _save_ngmix_data fills rows missing from the ngmix output catalogue. Verify the not-fitted default for NGMIX_MCAL_FLAGS is nonzero while real ngmix rows can clear it. Check the resulting flag behavior alongside NGMIX_N_EPOCH > 0 using the final_cat_smk-g7.hdf5 acceptance output.
Written by the indexing model from the issue text.
Description
_save_ngmix_data pre-fills every NGMIX_* column with sentinels and overwrites them only for objects found in the ngmix output catalogue. An object ngmix never fit is absent from that catalogue (it raised "0 epoch to process", ngmix.py:2034, or any other exception caught at ngmix.py:1058-1063, and was skipped with continue), so it keeps every default:
NGMIX_G1/G2 = -10 make_cat.py:494
NGMIX_T, NGMIX_FLUX = 0 make_cat.py:474
NGMIX_MCAL_FLAGS = 0 make_cat.py:501
NGMIX_N_EPOCH = 0 make_cat.py:453
MCAL_FLAGS, the column meaning "the metacal fit failed", therefore reads clean for objects that were never fit at all. In final_cat_smk-g7.hdf5 (the 64-tile acceptance run of #879/#886/#887/#880) this is 18,983 of 1,851,100 objects (1.03%): cutting on MCAL_FLAGS == 0 alone gives mean e1 = −0.107 (std 1.03), against −0.004 (std 0.22) once NGMIX_N_EPOCH > 0 is also required.
Minimal fix: initialise NGMIX_MCAL_FLAGS at make_cat.py:501 to a nonzero "not fitted" value instead of 0, so the flag column is honest by default and only a real ngmix row can clear it. A dedicated bit would be preferable to a bare nonzero if the column gains a documented bit table. sp_validation's classification_galaxy_ngmix happens to reject these rows today through its NGMIX_G1_PSF_ORIG_NOSHEAR != -10 guard — an exact float comparison against a sentinel — and CosmoStat/sp_validation#343 now carries an explicit NGMIX_N_EPOCH > 0 guard instead.
— Claude (Fable) on behalf of Cail
- Dominant language
- Python
- Stars
- 18
- Forks
- 14
- Avg merge
- 8h 40m
- Merged PRs (30d)
- 10
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from CosmoStat/shapepipe
-
Difficulty 3/5 1-2 days Newbie friendliness 78/100
-
Difficulty 4/5 3-5 days Newbie friendliness 55/100
All issues in CosmoStat/shapepipe
Similar issues
-
area: harness bug status: needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Human-Agent-Society/reef#625 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
-
Difficulty 1/5 Under an hour Newbie friendliness 80/100
learningequality/kolibri#15351 · 2 comments ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Name consistency Open
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
eellak/triplestore#65 · 1 comment ·