[FEA] Native HPO support and implementation on conda in Windows.

Hi, I'm currently working on machine learning applications in exoplanet detection, so I work with massive datasets that put the sklearn library to its limits, which made me discover cuML. I write this not as a professional software developer, but as a physicist playing around with tools, so I may be wrong in some of the things I say, or missing a few technicalities, so I do apologise in advance for my lack of knowledge in the field. For reference, I primarily use Spyder on the Windows anaconda environment, so I had to go through a WSL2 conda environment to be able to use cuML.

I'm a big fan of cuML, as it practically supersedes sklearn in its implemention, making migrating from sklearn to cuML as simple as changing the imports. However, cuML currently does not have native HPO, functions such as GridSearchCV, that are present in sklearn. I would love to see a native implementation of something like GridSearchCV inside of cuML.

Currently, the alternative is to use both sklearn and cuML and juggle data between your CPU and your GPU, which from what I've read online, can sometimes come out to be slower than just using sklearn. (source: https://blog.dask.org/2019/03/27/dask-cuml). A native implementation of HPO inside of cuML would:

• Remove the need to import both libraries
• For people like me, with a GPU with CUDA cores better than their CPU (RTX 3080 vs i7 9700K), off load all of the computation to the GPU
• Massively accelerate GridSearchCV which is O(n^2) at best, if it is possible to use the architecture of a GPU on this problem
• Remove the need for juggling data between devices thus preventing potential bottlenecks (correct me if I am wrong on this)
• Allow for multitasking while waiting for the optimisation to complete: This is a more personal one, but off loading work to my GPU just prevents me from launching games while waiting for the training, but running optimisation on my CPU, if this process requires a lot of computation and thus uses up a lot of resources, would majorly slow down my work flow as I use my CPU for other programs.

As for the other request, it may already be work in progess, but bringing cuML to the anaconda environment in Windows would allow me to natively use the library in Spyder which is my IDE of preference for data analysis.

I was also able to get cuML working on a Windows 10 machine with WSL2, while the doc mentions that cuML is only available for Windows 11 machines. WSL2 isn't exclusive to Windows 11 (despite running a lot better on it), so it might be judicious to update the doc.