Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Global shear response is IV-weighted while shear statistics are w_des-weighted

Open
  • Is something wrong?
#395 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
python
Domain
data

Research direction

Start with src/sp_validation/calibration.py, especially _total_response, R_selection, get_variance_ivweights, and get_calibrate_e_from_cat; compare their weighting with cosmo_val/core.py and cosmo_val/cat_config.yaml. Run the three named regression tests to reproduce the reported mismatches. Done means the response and calibration match the estimator’s weights, including selection effects and weight or cell-migration derivatives, with the variance and recalibration weighting issues addressed.

Written by the indexing model from the issue text.

Description

The global metacal shear response that calibrates the v1.4.6.3 catalogue is averaged with the inverse-variance weight w (global_R_weight: w, recorded in the released header as MC_PAR_6). Every cosmo_val statistic is weighted by w_des, with shear.R: 1.0. For a fixed selection, the response of a weighted mean sum(w e)/sum(w) is the mean of R_g under that same weight, so the applied response does not match the estimator. On a 6M-row sample of the release, the mean diagonal R_g is 0.7785 unweighted, 0.8037 with w_iv and 0.8273 with w_des. Holding the selection response fixed, a consistently weighted shear response would raise the total response by ~2.9%. Current shear amplitudes are therefore ~2.9% above that counterfactual, and xi ~5.9% above it. This is a conditional normalization shift, not a measured bias against true shear after the published m correction. Whether the image-simulation m = -0.057 absorbs it depends on the estimator that produced that m, whose provenance has not been recovered. The current image-sim workflow (unweighted response and numerator) does not contain this term, but it postdates the published chain. No S8 or B-mode PTE shift has been measured. A uniform rescaling need not move a B-mode PTE, because an empirical covariance rescales with the data.

  • metacal._total_response averages R_g with ns["w"] (src/sp_validation/calibration.py:1087-1096), while R_selection uses unweighted means (:1063-1079). The production configs set global_R_weight: w (e.g. config/calibration/mask_v1.X.6.yaml:110). cosmo_val weights by w_des with R: 1.0 (cosmo_val/cat_config.yaml:521-530, cosmo_val/core.py:597-603).
  • get_variance_ivweights adds the shape-error columns to 2*sigma_eps**2 without squaring them (calibration.py:861-869). They are standard deviations in both ShapePipe generations: pars_err at v1.4.0 (ngmix.py:260-266, 45384c91) and sqrt(g_cov) on develop (ngmix.py:957-960, 6ed93f88). On 791,667 released objects, squaring them lowers the IV-weighted mean R_g by 2.39% (subset, not full catalogue). This interacts with the first item: it does not add to it independently, and fixing it alone widens the w_des/IV gap.
  • get_calibrate_e_from_cat weights the additive bias by w_<weight_type> but averages R_g unweighted (calibration.py:635, :640-643), as does the recipe in docs/source/using_the_catalogues.md:199-212. No pipeline calls it.

Reproduction: src/sp_validation/tests/regression/test_response_weight_mismatch.py: gamma_hat=[0.022 -0.011], gamma=[0.02 -0.01]; R applied diag=[0.75 0.75], <R>_w_des=0.825 (toy weights are assigned; a fixture using the real get_w_des recovers 1.0515x the input shear)
src/sp_validation/tests/regression/test_iv_weight_adds_sigma_to_variance.py: applied global response: got 0.73707, expected 0.76115 from inverse measurement variances
src/sp_validation/tests/regression/test_recalibrator_ignores_weight_for_response.py: weight_type='des': recalibrated patch shear 0.025000 != truth 0.02

Where to fix: develop. calibration.py is identical on develop and merge/develop-into-tomo, and both branches weight v1.4.6.3 statistics by w_des.

Fix: derive the response of the estimator that is actually used. That means the w_des-weighted shear response together with a weighted selection response, including weight and cell-migration derivatives; averaging R_selection with w_des alone is not enough. Square the error columns in get_variance_ivweights, and weight R_g in get_calibrate_e_from_cat. Settling the science impact needs the historical m provenance, a consistently weighted m on the same simulations, and a likelihood rerun.

Claude (Opus 5.5) on behalf of Cail

Dominant language
Python
Stars
2
Forks
5
Avg merge
1d 3h
Merged PRs (30d)
40

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from CosmoStat/sp_validation

All issues in CosmoStat/sp_validation

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.