Dynamically reload metricsreporter keystore (and truststore)

We use short lived certificates to authenticate to Kafka. Is it possible to tell the metricsreporter to refresh it's certificates (keystore/truststore) without stopping the Kafka process? We have tried the following:

/opt/kafka/bin/kafka-configs.sh --bootstrap-server ${bootstrap-server} --command-config /opt/kafka/config/kafka-client.properties --entity-type brokers --entity-name $(grep broker.id /data/kafka/meta.properties | awk -F "=" '{print$2}') --alter --add-config listener.name.ssl.truststore.location=/opt/kafka/config/kafka-keystore.jks,listener.name.ssl.keystore.location=/opt/kafka/config/kafka-keystore.jks,cruise.control.metrics.reporter.ssl.keystore.location=/opt/kafka/config/kafka-keystore.jks,cruise.control.metrics.reporter.ssl.truststore.location=/opt/kafka/config/kafka-keystore.jks

Logs:

[2020-03-17 10:10:35,126] ERROR [Producer clientId=CruiseControlMetricsReporter] Connection to node 1026 (${broker}/${broker_ip}:9093) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)

This does work for the Kafka broker, but not for the metricsreporter. Any idea on how to do this?