Update setting up Prometheus docs

psschwei: originally opened in serving and moved here. updates needed for docs can be found in this comment: https://github.com/knative/docs/issues/5217#issuecomment-1247306521

/area monitoring

What version of Knative?

1.7.1 (installed with https://github.com/knative/serving/releases/download/knative-v1.7.1/serving-core.yaml)

Expected Behavior

• install knative-serving
• port-fowarded one of them to debug on prometheus metrics
• pods report metrics described in https://knative.dev/docs/serving/observability/metrics/serving-metrics/

Actual Behavior

kubectl port-forward service/activator-service -n knative-serving 9090:9090
Forwarding from 127.0.0.1:9090 -> 9090
Forwarding from [::1]:9090 -> 9090
Handling connection for 9090
E0914 16:42:51.809345 3007207 portforward.go:400] an error occurred forwarding 9090 -> 9090: error forwarding port 9090 to pod 4f48d8d6708f2f6cc464957b6718d2ec89088951c6430acfb0fb04857e595a94, uid : exit status 1: 2022/09/14 07:42:51 socat[13441] E connect(5, AF=2 127.0.0.1:9090, 16): Connection refused

I had to specify the metrics.backend-destination and metrics.request-metrics-backend-destination to get this working.

Steps to Reproduce the Problem

• follow instructions on https://knative.dev/docs/install/yaml-install/serving/install-serving-with-yaml/#prerequisites
• port-forward one of the components with 9090 port exposed
• curl localhost:9090 (same error with curl localhost:9090/metrics)

apiVersion: v1
kind: ConfigMap
metadata:
  name: config-observability
  namespace: knative-serving
  labels:
    app.kubernetes.io/name: knative-serving
    app.kubernetes.io/component: observability
    app.kubernetes.io/version: "1.7.1"
  annotations:
    knative.dev/example-checksum: "fed4756e"
data: {}