Option for filtering by CUDNN_DETERMINISTIC in `cudnnConvolutionAlgoPerfChoose`

Is your feature request related to a problem? Please describe.

Following from https://discourse.julialang.org/t/flux-reproducibility-of-gpu-experiments/62092, there is no way to guarantee for users of e.g. NNlibCUDA to ensure convolution operations only use deterministic algorithms

Describe the solution you'd like

Something along the lines of https://github.com/pytorch/pytorch/blob/6c70cbedb6102da08fe91186d40a41b50991681d/aten/src/ATen/native/cudnn/Conv_v7.cpp#L219-L252. Whether this would need to be plumbed through higher-level functions, set as a global option or exposed through a context manager is left for debate.

Describe alternatives you've considered

The only solution now is seems to be pirating cudnnConvolutionForwardAD such that it doesn't use cudnnConvolutionFwdAlgoPerf?

Additional context

https://docs.nvidia.com/deeplearning/cudnn/developer-guide/index.html#reproducibility states:

...the following routines do not guarantee reproducibility because they use atomic operations:

• cudnnConvolutionBackwardFilter when CUDNN_CONVOLUTION_BWD_FILTER_ALGO_0 or CUDNN_CONVOLUTION_BWD_FILTER_ALGO_3 is used
• cudnnConvolutionBackwardData when CUDNN_CONVOLUTION_BWD_DATA_ALGO_0 is used

So I have no clue if/how often these show up in practice in cudnnConvolution*AlgoPerf, but I assume they must if users are seeing non-deterministic results?