The configuration for number of query errors in the `show queries` does not work anymore
#9533 aperta il 7 set 2022
Metriche repository
- Star
- (5739 star)
- Metriche merge PR
- (Merge medio 4g 20h) (6 PR mergiate in 30 g)
Descrizione
Describe the bug
When a query has errors, a list of errors collected is displayed by the SHOW QUERIES EXTENDED or EXPLAIN <query> commands. This list has a config to limit the number of errors displayed. The config is ksql.query.error.max.queue.size and is default to 10. However, the SHOW QUERIES command is displaying more errors than what it is configured and it keeps growing.
Looking at the code, I found the issue was caused by this fix https://github.com/confluentinc/ksql/pull/8875
The queue used to use an EvictionQueue class which evicts queue elements when a size/capacity has been reached. After the previous fix, the capacity is not used anymore, so the queue becomes unbounded on size. It is still time-bounded because the evict() method is called after adding an element. This evict method removes elements that expired in 1 hour window (configured).
Fix:
We should keep the queue bounded to capacity and time (as configured). It is important to keep the queue thread-safe, so maybe the fix should be in the evict() method to remove elements if the queue has reach the configured capacity.