TF Java 0.3.1 shows a performance degradation on GPU compared to v 0.2.0 when loading Hugging Face models

Start with the Spark NLP tests under src/test/scala/com/johnsnowlabs and establish a minimal GPU benchmark that compares TF Java 0.3.1 with 0.2.0 while loading Hugging Face models. Confirm the reported threefold regression, then document the reproducible case and verify that the newer version matches the expected performance.

Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Mint 20.1 (Ubuntu 20.04 LTS)
• Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
• TensorFlow installed from (source or binary): binary
• TensorFlow version (use command below): TF Java 0.3.1 (TF 2.4.1)
• Python version:
• Bazel version (if compiling from source):
• GCC/Compiler version (if compiling from source):
• CUDA/cuDNN version: 11.0 / 8.0.4
• GPU model and memory:
  GeForce GTX 1060 computeCapability: 6.1
  coreClock: 1.6705GHz coreCount: 10 deviceMemorySize: 5,93GiB deviceMemoryBandwidth: 178,99GiB/s

You can collect some of this information using our environment capture script
You can also obtain the TensorFlow version with
python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"

Describe the current behavior
The usage of version TF Java bindings 0.3.1 degradates performances of a 3x factor on GPU compared to version 0.2.0 .

Describe the expected behavior
Equal, hopefully better performances while migrating to newer versions.

Code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate the problem.
Performance tests are currently on going to validate the issue. We'll update with more info asap.
https://github.com/JohnSnowLabs/spark-nlp/tree/master/src/test/scala/com/johnsnowlabs

Other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.