Skip to content

Fix: Detect illegal ContinueWith usage in orchestrators using AsyncLocal#3303

Draft
Copilot wants to merge 3 commits intodevfrom
copilot/fix-durable-functions-order
Draft

Fix: Detect illegal ContinueWith usage in orchestrators using AsyncLocal#3303
Copilot wants to merge 3 commits intodevfrom
copilot/fix-durable-functions-order

Conversation

Copy link
Contributor

Copilot AI commented Jan 5, 2026

Summary

What changed?

  • Added AsyncLocal<DurableOrchestrationContext> to track orchestration context across async flows
  • Modified ThrowIfInvalidAccess() to check both IsOrchestratorThread and the AsyncLocal context
  • Updated TaskOrchestrationShim to set/clear context at orchestration execution boundaries

Why is this change needed?

The existing OrchestrationContext.IsOrchestratorThread check uses [ThreadStatic] which doesn't flow with async context. When thread pool reuses the same thread for ContinueWith continuations, the check fails to detect illegal access—causing activities to execute out of order silently.

AsyncLocal<T> properly flows with async context, enabling reliable detection of illegal patterns like:

// This should throw but previously could slip through
task.ContinueWith(t => {
    ctx.CallActivityAsync("MyActivity", input); // Now properly detected
});

Issues / work items


Project checklist

  • Documentation changes are not required
  • Release notes are not required for the next release
  • Backport is not required
  • All required tests have been added/updated (unit tests, E2E tests)
  • No extra work is required to be leveraged by OutOfProc SDKs
  • No change to the version of the WebJobs.Extensions.DurableTask package
  • No EventIds were added to EventSource logs
  • This change should be added to the v2.x branch
  • Breaking change?

AI-assisted code disclosure (required)

Was an AI tool used? (select one)

  • No
  • Yes, AI helped write parts of this PR (e.g., GitHub Copilot)
  • Yes, an AI agent generated most of this PR

If AI was used:

  • Tool(s): GitHub Copilot coding agent
  • AI-assisted areas/files: DurableOrchestrationContext.cs, TaskOrchestrationShim.cs
  • What you changed after AI output: N/A

AI verification (required if AI was used):

  • I understand the code and can explain it
  • I verified referenced APIs/types exist and are correct
  • I reviewed edge cases/failure paths (timeouts, retries, cancellation, exceptions)
  • I reviewed concurrency/async behavior
  • I checked for unintended breaking or behavior changes

Testing

Automated tests

  • Result: Passed (158 analyzer tests, 67 worker extension tests)

Manual validation (only if runtime/behavior changed)

  • N/A (existing ThrowIfInvalidAccess behavior preserved; new check is additive)

Notes for reviewers

  • The AsyncLocal check complements the existing IsOrchestratorThread check—both must pass
  • ClearCurrentContext() includes Debug.Assert to help catch logic errors during development
  • Existing analyzer already warns about ContinueWith without ExecuteSynchronously; this adds runtime enforcement

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • azfunc.pkgs.visualstudio.com
    • Triggering command: /opt/hostedtoolcache/CodeQL/2.23.8/x64/codeql/csharp/tools/linux64/Semmle.Autobuild.CSharp /opt/hostedtoolcache/CodeQL/2.23.8/x64/codeql/csharp/tools/linux64/Semmle.Autobuild.CSharp (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

This section details on the original issue you should resolve

<issue_title>Calling activities within a ContinueWith block in durable functions is executing tasks out of order</issue_title>
<issue_description>From @shaunol on June 28, 2018 23:22

Calling activities within a ContinueWith block in durable functions is executing tasks out of order. I put together a minimal repro which outputs the sequence as log warnings.

Repro steps

Provide the steps required to reproduce the problem:

  1. Create new Azure Functions App v2 project (VS2017 15.7.4 with Azure Functions and Web Jobs Tools 15.0.40617.0)
  2. Use repro source code (at end of issue)
  3. Run functions using http triggers:
  1. Check functions host output for yellow warnings which will show the function call sequence

Expected behavior

The output sequence should always be 0,1,2

Actual behavior

The output sequence shows both 0,1,2, and 0,2,1 seemingly randomly. It happens enough that you can run the function manually and see the issue within a few attempts.

Known workarounds

Don't call activities inside ContinueWith blocks

Related information

I included a non-durable example of the same code sequence, which appears to always be in the correct order from manual testing.

Source
public static class ContinueWithFunctions
    {
        public class CounterWrapper
        {
            public string InstanceId { get; set; }
            public long Counter { get; set; }
        }

        [FunctionName(nameof(ContinueWith_Start))]
        public static async Task<HttpResponseMessage> ContinueWith_Start(
            [HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)]HttpRequestMessage req,
            [OrchestrationClient] DurableOrchestrationClientBase starter,
            TraceWriter log)
        {
            var instanceId = await starter.StartNewAsync(nameof(ContinueWith_Orchestrator), null);

            log.Info($"{nameof(ContinueWith_Start)}: Started with ID = '{instanceId}'.");

            return await starter.WaitForCompletionOrCreateCheckStatusResponseAsync(req, instanceId);
        }

        [FunctionName(nameof(ContinueWith_Orchestrator))]
        public static async Task ContinueWith_Orchestrator(
            [OrchestrationTrigger]DurableOrchestrationContext ctx,
            TraceWriter log)
        {
            var counter = new CounterWrapper()
            {
                InstanceId = ctx.InstanceId
            };

            await ctx.CallActivityAsync(nameof(ContinueWith_Activity), counter)
                .ContinueWith(t =>
                {
                    counter.Counter++;

                    return ctx.CallActivityAsync(nameof(ContinueWith_Activity), counter);
                });

            counter.Counter++;

            await ctx.CallActivityAsync(nameof(ContinueWith_Activity), counter);
        }

        [FunctionName(nameof(ContinueWith_Activity))]
        public static Task ContinueWith_Activity(
            [ActivityTrigger]CounterWrapper input,
            TraceWriter log)
        {
            log.Warning($"{input.InstanceId}: {input.Counter}");
            return Task.CompletedTask;
        }

        [FunctionName(nameof(ContinueWith_Standalone))]
        public static async Task<HttpResponseMessage> ContinueWith_Standalone(
            [HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)]HttpRequestMessage req,
            TraceWriter log)
        {
            var counter = new CounterWrapper()
            {
                InstanceId = Guid.NewGuid().ToString()
            };

            await ContinueWith_Standalone_Task(counter, log)
                .ContinueWith(t =>
                {
                    counter.Counter++;

                    return ContinueWith_Standalone_Task(counter, log);
                });

            counter.Counter++;

            await ContinueWith_Standalone_Task(counter, log);

            return req.CreateResponse();
        }

        private static Task ContinueWith_Standalone_Task(
            CounterWrapper input,
            TraceWriter log)
        {
            log.Warning($"{input.InstanceId}: {input.Counter}");
            return Task.CompletedTask;
        }
    }

Copied from original issue: Azure/azure-functions-host#3075</issue_description>

<agent_instructions>check if this issue still exists, if so fix it</agent_instructions>

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 2 commits January 5, 2026 22:36
…in orchestrators

The previous implementation relied solely on OrchestrationContext.IsOrchestratorThread
which is a [ThreadStatic] variable that doesn't properly flow with async context.
This can lead to false negatives when thread pool reuses the same thread.

This fix adds an AsyncLocal<DurableOrchestrationContext> that properly flows with
async context, enabling reliable detection of illegal async patterns like calling
activities from within ContinueWith blocks.

Co-authored-by: YunchuWang <12449837+YunchuWang@users.noreply.github.com>
…debug assertion

Co-authored-by: YunchuWang <12449837+YunchuWang@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix task execution order in ContinueWith block Fix: Detect illegal ContinueWith usage in orchestrators using AsyncLocal Jan 5, 2026
Copilot AI requested a review from YunchuWang January 5, 2026 22:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Calling activities within a ContinueWith block in durable functions is executing tasks out of order

2 participants