alibaba/Sentinel

[DISCUSSION] Circuit breaker will remain half-open state forever when the request is blocked by upcoming rules

Open

#1,638 创建于 2020年7月30日

在 GitHub 查看
 (7 评论) (0 反应) (0 负责人)Java (8,150 fork)batch import
area/circuit-breakinghelp wantedkind/bug

仓库指标

Star
 (23,109 star)
PR 合并指标
 (30 天内没有已合并 PR)

描述

Issue Description

Type: bug report

Describe what happened (or what feature you want)

The circuit breaker won't recover from half-open state when the request is actually blocked by upcoming rules. For example, there are two degrade rules of the same resource: R1(circuit breaker state=OPEN, recoveryTimeout=10s) and R2(circuit breaker state=OPEN, recoveryTimeout=20s)

There may be circumstances when R1 has reached the recovery timepoint but R2 has not. If a request comes, there will be transformation: R1(OPEN → HALF-OPEN), R2(OPEN)

This request will be allowed by R1 but rejected by R2, thus finally blocked. The invocation won't actually occur, so it will NEVER complete. For state transformation from HALF-OPEN to OPEN/CLOSED, it should happen only when invocation completes, so for R1 the associated circuit breaker state will be HALF-OPEN forever. Actually this could happen when there are any blocked rules after R1 (not only degrade rules).

This is a fatal bug and should be carefully resolved. We may need a temporary workaround for the half-open case in 1.8.0, then improve the overall design later. Discussions are welcomed.

Original discussions can be found in https://github.com/alibaba/Sentinel/pull/1490#discussion_r462079959

贡献者指南