alibaba/Sentinel

[DISCUSSION] Circuit breaker will remain half-open state forever when the request is blocked by upcoming rules

Open

#1,638 建立於 2020年7月30日

在 GitHub 查看
 (7 留言) (0 反應) (0 負責人)Java (8,150 fork)batch import
area/circuit-breakinghelp wantedkind/bug

倉庫指標

Star
 (23,109 star)
PR 合併指標
 (30 天內沒有已合併 PR)

描述

Issue Description

Type: bug report

Describe what happened (or what feature you want)

The circuit breaker won't recover from half-open state when the request is actually blocked by upcoming rules. For example, there are two degrade rules of the same resource: R1(circuit breaker state=OPEN, recoveryTimeout=10s) and R2(circuit breaker state=OPEN, recoveryTimeout=20s)

There may be circumstances when R1 has reached the recovery timepoint but R2 has not. If a request comes, there will be transformation: R1(OPEN → HALF-OPEN), R2(OPEN)

This request will be allowed by R1 but rejected by R2, thus finally blocked. The invocation won't actually occur, so it will NEVER complete. For state transformation from HALF-OPEN to OPEN/CLOSED, it should happen only when invocation completes, so for R1 the associated circuit breaker state will be HALF-OPEN forever. Actually this could happen when there are any blocked rules after R1 (not only degrade rules).

This is a fatal bug and should be carefully resolved. We may need a temporary workaround for the half-open case in 1.8.0, then improve the overall design later. Discussions are welcomed.

Original discussions can be found in https://github.com/alibaba/Sentinel/pull/1490#discussion_r462079959

貢獻者指南