Private AI service container: documentation missing on huggingface downloads

No documentation file or test is named in the issue, so first locate the documentation for deploying the Private AI container on Kubernetes. Add the Hugging Face model ConfigMap, huggingface-token Secret, startup log example, and recommended logging environment variables. Done means a reader can follow the documented steps to download and verify an LLM.

Its is working, but it is undocumented yet. Please add information about how to download LLMs from huggingface into PAI container using HF_TOKEN protection.

To achieve that:

define a config.json that points to a huggingface location and store it as usual in a configmap. Example for gpt-oss LLM that fits in a small A10 GPU:

apiVersion: v1
kind: ConfigMap
metadata:
  name: private-ai-config
  namespace: private-ai
data:
  config.json: |
    {
      "models":[
        {
          "name":"gpt-oss-20b",
          "path":"openai/gpt-oss-20b",
          "runtime":"vllm",
          "capabilities":["TEXT_GENERATION"],
          "runtime_arguments": [
             "--max-num-seqs", "8",
             "--max-model-len", "16384",
             "--gpu-memory-utilization", "0.95"
          ]
        }
      ]
    }

2. get a token from huggingface and put it in the same secret as the PAI API key or the SSL password. Use the key "huggingface-token" to store the actual token. When PAI container starts, it picks up the HF_TOKEN and uses it to download the LLM.

apiVersion: v1
kind: Secret
type: Opaque
metadata:
  name: paisecret
  namespace: private-ai
data:
  api-key: yadayadayada=
  huggingface-token: myverysecretanonymizedtoken==
  privateai-ssl-pwd: notofyourinterest

Heres the log output, saying that PAI picked up the HF_TOKEN:

kubectl logs pai-quickstart-789f4fcd66-djlhg -n private-ai
INFO: Config file set to /privateai/config/config.json
INFO: Keystore path set to file:/privateai/ssl/keystore
INFO: API Key path set to /privateai/ssl/api-key
INFO: Hugging Face token loaded from secret /privateai/ssl/huggingface-token
12:09:53.415 [main] INFO  i.m.s.ObjectMappers$ObjectMapperContext$1 - Established active environments: [k8s, cloud]
  ____       _            _            _    ___
 |  _ \ _ __(_)_   ____ _| |_ ___     / \  |_ _|
 | |_) | '__| \ \ / / _` | __/ _ \   / _ \  | |
 |  __/| |  | |\ V / (_| | ||  __/  / ___ \ | |
 |_|   |_|  |_| \_/ \__,_|\__\___| /_/   \_\___|
  Private-AI (version small-cpu-infer-26.2.1.0.0, build 2.0.42)

12:09:53.650 [main] INFO  i.m.c.DefaultApplicationContext$RuntimeConfiguredEnvironment - Established active environments: [k8s, cloud]
12:13:43.130 [main] INFO  i.m.l.PropertiesLoggingLevelsConfigurer - Setting log level 'ERROR' for logger: 'io.micrometer.core.instrument.binder.cache.CaffeineCacheMetrics'
12:13:43.131 [main] INFO  i.m.l.PropertiesLoggingLevelsConfigurer - Setting log level 'ERROR' for logger: 'io.micrometer.core.instrument.binder.cache.caffeine-cache-metrics'
12:13:43.434 [main] INFO  io.micronaut.runtime.Micronaut - Startup completed in 229791ms. Server Running: https://0.0.0.0:8443

To check if the download actually worked, and especially loading the LLM into memory and to compile it, You need to set the following recommended environment variable(s). maybe add this to the doc too:

          - name: PRIVATE_AI_LOG_STDOUT_ENABLED
            value: "true"
          - name: PRIVATE_AI_LOG_LEVEL
            value: ALL