Regression after upgrading Microsoft.Azure.DurableTask.Core 3.2.0 → 3.6.0 (TaskHub initialization errors + unexpected 404s against Azure Table Storage)
@sophiatev is already working on this.
Since Dec 11, 2025.
Assessment
This issue has not been assessed yet.
Description
Hi team,
After upgrading Microsoft.Azure.DurableTask.Core and DurableTask.AzureStorage from 3.2.0 to 3.6.0, we started seeing a large spike in storage-related failures coming from the Azure Storage provider. These errors begin immediately after deployment and disappear completely when rolling back to 3.2.0.
Right after the upgrade, orchestration workers start throwing repeated errors while trying to initialize or process work for the TaskHub.
Here is an example of one of the failures:
An unexpected failure occurred while processing instance 'mgmt-cme2e1204233657cluster': DurableTask.AzureStorage.Storage.DurableTaskStorageException: An error occurred while communicating with Azure Storage ---> Azure.RequestFailedException: The specified resource does not exist.
RequestId:670a9fd6-a002-0052-4bac-65cb63000000
Time:2025-12-05T05:58:29.0299754Z
Status: 404 (Not Found)
ErrorCode: ResourceNotFound
Content:
{"odata.error":{"code":"ResourceNotFound","message":{"lang":"en-US","value":"The specified resource does not exist.\nRequestId:670a9fd6-a002-0052-4bac-65cb63000000\nTime:2025-12-05T05:58:29.0299754Z"}}}
Headers:
Transfer-Encoding: chunked
x-ms-request-id: 670a9fd6-a002-0052-4bac-65cb63000000
x-ms-client-request-id: 87682197-128c-412b-91d5-045f83cfc17d
x-ms-version: REDACTED
X-Content-Type-Options: REDACTED
Cache-Control: no-cache
Content-Type: application/json;odata=minimalmetadata;streaming=true;charset=utf-8
Date: Fri, 05 Dec 2025 05:58:28 GMT
at Azure.Data.Tables.TableRestClient.d__33.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Azure.Data.Tables.TableClient.d__511.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at DurableTask.AzureStorage.Storage.ClientResponseExtensions.<DecorateFailure>d__1.MoveNext() in /_/src/DurableTask.AzureStorage/Storage/ClientResponseExtensions.cs:line 30 --- End of inner exception stack trace --- at DurableTask.AzureStorage.Storage.ClientResponseExtensions.<DecorateFailure>d__1.MoveNext() in /_/src/DurableTask.AzureStorage/Storage/ClientResponseExtensions.cs:line 34 --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at DurableTask.AzureStorage.Storage.Table.<MergeEntityAsync>d__161.MoveNext() in //src/DurableTask.AzureStorage/Storage/Table.cs:line 97
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at DurableTask.AzureStorage.Tracking.AzureTableTrackingStore.d__52.MoveNext() in //src/DurableTask.AzureStorage/Tracking/AzureTableTrackingStore.cs:line 815
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at DurableTask.AzureStorage.AzureStorageOrchestrationService.d__89.MoveNext() in //src/DurableTask.AzureStorage/AzureStorageOrchestrationService.cs:line 1083
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at DurableTask.AzureStorage.AzureStorageOrchestrationService.d__83.MoveNext() in //src/DurableTask.AzureStorage/AzureStorageOrchestrationService.cs:line 796
Impact-
TaskHub workers get stuck retrying.
Queue length (work items) grows quickly.
Latency spikes.
Some orchestrations never get picked up.
Rolling back to 3.2.0 immediately resolves the issue.
Were there any breaking changes introduced in 3.6.0 that affect the Azure Storage provider?
Did the table schema or required entity format change?
Are there new configuration flags or capabilities we need to enable?
Is this a known issue or regression?
- Dominant language
- C#
- Stars
- 1.7k
- Forks
- 335
- Avg merge
- 4d 2h
- Merged PRs (30d)
- 6
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from Azure/durabletask
-
Difficulty 4/5 3-5 days Newbie friendliness 55/100
Azure/durabletask#1398 · 2 comments ·
-
Azure/durabletask#1389 · 1 comment · 1 assignee ·
-
Difficulty 3/5 1-2 days Newbie friendliness 74/100
Azure/durabletask#1376 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 72/100
Azure/durabletask#1378 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
Azure/durabletask#1332 ·
All issues in Azure/durabletask
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
nightscout/nocturne#1379 ·
-
priority-0
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Difficulty 1/5 Under an hour Newbie friendliness 78/100
StackExchange/StackExchange.Redis#3249 ·
-
[Feat] 조합 영역 구분선 개선 Open
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
documentation
Difficulty 2/5 1-3 hours Newbie friendliness 72/100