FluxML/Zygote.jl

Unable to differentiate a broadcasted constructor with CUDA array inputs

Open

#1,528 opened on Oct 3, 2024

View on GitHub
 (2 comments) (0 reactions) (0 assignees)Julia (1,568 stars) (220 forks)batch import
CUDAhelp wanted

Description

Due to a CUDA bug: https://github.com/JuliaGPU/CUDA.jl/issues/2514, crossposted for reference.

Encountered when differentiating over:

using Distributions
using CUDA
a = cu(ones(5)); b = cu(zeros(5));

Normal.(a, b)

Contributor guide

Tech stack
None
Domain
machine learning
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
4
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
3-5 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
blocked
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Julia basicsZygote.jl basicsCUDA.jl basicsBroadcasting in Julia
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
20
Research direction
The issue is blocked by a CUDA.jl bug (reference: JuliaGPU/CUDA.jl#2514). To proceed, first understand that bug and its implications on Zygote's differentiation of broadcasted constructors. Then, examine Zygote's handling of broadcasted calls and see if a workaround is possible. The provided example uses Distributions.Normal with CUDA arrays; review the interaction between Distributions and Zygote. No linked PRs or assignees exist yet.