HttpClientFactory doesn't clean up after itself

Description

Recently I ran into an issue running tests using dotnet test that it would crash without any obvious reason. Running my tests also showed memory would steadily increase over time whilst they were being run which indicated the presence of a memory leak. Analysing the memory dump there were 1000s of timers holding on to objects that I would've expected to be disposed of all centered around ActiveHandlerTrackingEntry.

Reproduction Steps

Judging by the code, I'd say you could just get away with a for loop that creates a service collection, adds the http client factory, creates a service provider and then create a dozen or so http clients with it.

Expected behavior

When the service provider it's associated with is disposed of, all resources the http client factory is associated with should be disposed of and freed up, including any timers that it creates.

Actual behavior

Timers still hang around in the background, long after that instance of the http client factory is no longer in use. Which eventually consumes all available resources.

Regression?

No response

Known Workarounds

When running the tests ensuring the handler lifetime is set to infinite effectively bypasses the handler lifetime logic that creates the timers. Which stops the crashes that I was experiencing.

Configuration

Running .net 8. MacOS and Linux, ARM64 and x64. I don't think the configuration matters.

Other information

No response