FluxML/Flux.jl

More robust testing for AlphaDropout

Open

#1851 opened on Jan 27, 2022

View on GitHub
 (6 comments) (0 reactions) (0 assignees)Julia (4,725 stars) (619 forks)batch import
good first issuehelp wantedtesting

Description

We currently borrow from https://github.com/pytorch/pytorch/blob/v1.10.0/test/cpp/api/modules.cpp#L1337-L1338, but some sort of goodness-of-fit test would be more robust than a simple range. Sampling the layer outputs should have a negligible impact on test runtime.

Contributor guide

Tech stack
None
Domain
testingmachine learning
Issue type
test
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
Julia basicsFlux libraryStatistical testing
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
70
Research direction
Examine the existing AlphaDropout test in Flux.jl's test suite, likely in test/layers.jl or similar. The issue suggests replacing the simple range check with a goodness of fit test, such as a chi squared test, to verify the dropout distribution. Review the PyTorch reference (test/cpp/api/modules.cpp lines 1337-1338) for inspiration, but adapt to Julia's testing framework. Consider implementing a statistical test that checks whether the output distribution matches the expected AlphaDropout distribution.