[TASK][MEDIUM] Integrate NVIDIA RAPIDS with Spark engine

Code of Conduct

• I agree to follow this project's Code of Conduct

Search before creating

• I have searched in the task list and found no similar tasks.

Mentor

• I have sufficient knowledge and experience of this task, and I volunteer to be the mentor of this task to guide contributors to complete the task.

Skill requirements

• Basic knowledge of Scala programming language, Kyuubi architecture, Apache Spark, NVIDIA RAPIDS
• Basic English writing skills

Background and Goals

RAPIDS is a Spark plugin developed by NVIDIA to accelerate Apache Spark with GPUs.

As Kyuubi currently supports using Spark as the computing engine, we can add tests to integrate with RAPIDS in the Spark engine, and write docs to guide users in setting up Kyuubi with Spark plus RAPIDS to leverage GPUs.

Implementation steps

• Add RAPIDS UTs in spark engine module, which could be conditionally enabled via maven profile and scalatest annotation
• Setup CI for RAPIDS testing (I'm not sure if GitHub Actions or ASF infra could provide such CUDA environments for running CI)
• Write docs to guide users in setting up Kyuubi with Spark plus RAPIDS to leverage GPUs.

Additional context

Introduction of https://github.com/apache/kyuubi/issues/6232