Allow `NodeProviderOutOfProcTaskHost` to manage multiple nodes instead of one per arch #12521

surayya-MS · 2025-09-12T18:03:05Z

Context

Changes Made

Testing

Notes

previously _nodeContext was dict with key of HandshakeOptions, now it is nodeId, previously NodeProviderOutOfProcTaskHost allowed max 4 nodes depending on the arch and etc., now it is unlimited

Copilot

Pull Request Overview

This PR refactors the NodeProviderOutOfProcTaskHost class to support managing multiple nodes dynamically rather than being limited to exactly 4 nodes. The change removes the hardcoded MaxNodeCount = 4 limitation and introduces proper node ID management.

Key Changes

Replaced HandshakeOptions enum-based node identification with integer-based node IDs
Removed the hardcoded maximum node count limitation of 4 nodes
Updated the node management system to use ConcurrentDictionary with proper locking mechanisms

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
TaskHostTask.cs	Updated to use integer node IDs instead of HandshakeOptions for communication with task host provider
NodeProviderOutOfProcTaskHost.cs	Refactored node management system to support unlimited nodes with proper concurrent access and ID generation

src/Build/BackEnd/Components/Communications/NodeProviderOutOfProcTaskHost.cs

surayya-MS · 2025-09-12T18:05:29Z

Changed Message task to append Process id for manual validation (did't commit this change)
cc @AR-May

C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\MSBuild.exe .\MainProject.proj -mt -m
MSBuild version 17.15.0-dev-25462-01+ce57e7f1e for .NET Framework
Build started 9/12/2025 7:56:38 PM.
     1>Project "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\MainProject\MainProject.proj
       " on node 2 (default targets).
     1>Build:
         Task assembly was loaded from 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Build.dll' w
         hile the desired location was 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Build.Tasks.
         Core.dll'.
         MainProject building (PID 4264)
     1>Project "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\MainProject\MainProject.proj
       " (1) is building "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\ProjectD\ProjectD.
       proj" (4) on node 5 (Build target(s)).
     4>Build:
         Task assembly was loaded from 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Build.dll' w
         hile the desired location was 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Build.Tasks.
         Core.dll'.
     1>Project "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\MainProject\MainProject.proj
       " (1) is building "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\ProjectB\ProjectB.
       proj" (3) on node 3 (Build target(s)).
     3>Build:
         Task assembly was loaded from 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Build.dll' w
         hile the desired location was 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Build.Tasks.
         Core.dll'.
     1>Project "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\MainProject\MainProject.proj
       " (1) is building "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\ProjectA\ProjectA.
       proj" (2) on node 2 (Build target(s)).
     2>Build:
         Task assembly was loaded from 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Build.dll' w
         hile the desired location was 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Build.Tasks.
         Core.dll'.
     1>Project "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\MainProject\MainProject.proj
       " (1) is building "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\ProjectC\ProjectC.
       proj" (5) on node 4 (Build target(s)).
     5>Build:
         Task assembly was loaded from 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Build.dll' w
         hile the desired location was 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Build.Tasks.
         Core.dll'.
     4>Build:
         Building ProjectD (PID 19180)
     5>Build:
         Building ProjectC (PID 27048)
     4>Done Building Project "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\ProjectD\Proje
       ctD.proj" (Build target(s)).
     5>Done Building Project "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\ProjectC\Proje
       ctC.proj" (Build target(s)).
     3>Build:
         Building ProjectB (PID 35836)
     3>Done Building Project "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\ProjectB\Proje
       ctB.proj" (Build target(s)).
     2>Build:
         Building ProjectA (PID 36408)
     2>Done Building Project "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\ProjectA\Proje
       ctA.proj" (Build target(s)).
     1>Build:
         Task assembly was loaded from 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Build.dll' w
         hile the desired location was 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Build.Tasks.
         Core.dll'.
         MainProject is built (PID 38028)
     1>Done Building Project "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\MainProject\Ma
       inProject.proj" (default targets).

Build succeeded.
    0 Warning(s)
    0 Error(s)

src/Build/BackEnd/Components/Communications/NodeProviderOutOfProcTaskHost.cs

AR-May · 2025-09-15T07:49:08Z

Let's also check that taskhosts are reused properly during the build - project which has multiple message tasks, during the build with /mt /m should print have same process ID.

…mment

surayya-MS · 2025-09-15T08:47:40Z

Let's also check that taskhosts are reused properly during the build - project which has multiple message tasks, during the build with /mt /m should print have same process ID.

I added second message to each of the projects.

The result is different process ids for messages from the same project. I then checked same without -mt, and the result is the same.

But in main without -mt, the result in is same process id for tasks from the same project.

>C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\MSBuild.exe .\MainProject.proj -m -mt
MSBuild version 17.15.0-dev-25465-01+34e536570 for .NET Framework
Build started 9/15/2025 10:44:49 AM.
     1>Project "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\MainProject\MainProject
       .proj" on node 2 (default targets).
     1>Build:
         Task assembly was loaded from 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Build.d
         ll' while the desired location was 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Bu
         ild.Tasks.Core.dll'.
         MainProject building (PID 2360)
     1>Project "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\MainProject\MainProject
       .proj" (1) is building "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\ProjectD
       \ProjectD.proj" (3) on node 5 (Build target(s)).
     3>Build:
         Task assembly was loaded from 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Build.d
         ll' while the desired location was 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Bu
         ild.Tasks.Core.dll'.
     1>Project "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\MainProject\MainProject
       .proj" (1) is building "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\ProjectB
       \ProjectB.proj" (2) on node 3 (Build target(s)).
     2>Build:
         Task assembly was loaded from 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Build.d
         ll' while the desired location was 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Bu
         ild.Tasks.Core.dll'.
     1>Project "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\MainProject\MainProject
       .proj" (1) is building "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\ProjectA
       \ProjectA.proj" (4) on node 2 (Build target(s)).
     4>Build:
         Task assembly was loaded from 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Build.d
         ll' while the desired location was 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Bu
         ild.Tasks.Core.dll'.
     1>Project "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\MainProject\MainProject
       .proj" (1) is building "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\ProjectC
       \ProjectC.proj" (5) on node 4 (Build target(s)).
     5>Build:
         Task assembly was loaded from 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Build.d
         ll' while the desired location was 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Bu
         ild.Tasks.Core.dll'.
         Building ProjectC 1 (PID 12572)
         Task assembly was loaded from 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Build.d
         ll' while the desired location was 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Bu
         ild.Tasks.Core.dll'.
     4>Build:
         Building ProjectA 1 (PID 13948)
         Task assembly was loaded from 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Build.d
         ll' while the desired location was 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Bu
         ild.Tasks.Core.dll'.
         Building ProjectA 2 (PID 25240)
     2>Build:
         Building ProjectB 1 (PID 30800)
     4>Done Building Project "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\ProjectA\
       ProjectA.proj" (Build target(s)).
     2>Build:
         Task assembly was loaded from 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Build.d
         ll' while the desired location was 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Bu
         ild.Tasks.Core.dll'.
     3>Build:
         Building ProjectD 1 (PID 39648)
         Task assembly was loaded from 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Build.d
         ll' while the desired location was 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Bu
         ild.Tasks.Core.dll'.
     5>Build:
         Building ProjectC 2 (PID 13036)
     2>Build:
         Building ProjectB 2 (PID 28188)
     5>Done Building Project "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\ProjectC\
       ProjectC.proj" (Build target(s)).
     2>Done Building Project "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\ProjectB\
       ProjectB.proj" (Build target(s)).
     3>Build:
         Building ProjectD 2 (PID 32000)
     3>Done Building Project "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\ProjectD\
       ProjectD.proj" (Build target(s)).
     1>Build:
         Task assembly was loaded from 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Build.d
         ll' while the desired location was 'C:\msbuild\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\Microsoft.Bu
         ild.Tasks.Core.dll'.
         MainProject is built (PID 27148)
     1>Done Building Project "C:\Users\shuseynzada\source\repos\many-projects-with-msg\many-projects-with-msg\MainProje
       ct\MainProject.proj" (default targets).

Build succeeded.
    0 Warning(s)
    0 Error(s)

src/Build/BackEnd/Components/Communications/NodeProviderOutOfProcTaskHost.cs

surayya-MS · 2025-09-15T11:02:43Z

From the previous results:
The problem of not being able to reuse nodes is because I create a new node very time in NodeProviderOutOfProcTaskHost.AcquireAndSetUpHost.
There is no nodeId (the in-proc one) to be able to map it for task in order to reuse it.

Before the multi-threaded mode was introduced, there was one node per process , and the node to reuse in NodeProviderOutOfProcTaskHost was selected by HandshakeOptions.

We need to somehow pass on nodeId (in-proc) to TaskBuilder, TaskHostTask, NodeProviderOutOfProcTaskHost to be able to reuse same node for the tasks from the same project.

JanProvaznik · 2025-09-15T11:48:04Z

src/Build/BackEnd/Components/Communications/NodeProviderOutOfProcTaskHost.cs

            {
-                // node already exists, so "creation" automatically succeeded
-                nodeCreationSucceeded = true;
+                nodeId = _nextNodeId++;


consider https://learn.microsoft.com/en-us/dotnet/api/system.threading.interlocked.increment?view=net-9.0 instead of the lock

JanProvaznik · 2025-09-15T11:52:05Z

src/Build/Instance/TaskFactories/TaskHostTask.cs

-        private HandshakeOptions _requiredContext = HandshakeOptions.None;
+        private HandshakeOptions _requiredHandshakeOptions = HandshakeOptions.None;
+
+        private int _requiredNodeId = -1;


this naming is confusing, if you want to express that a specific taskhosttask has a mapping to a nodeId, then I'd just say something like _taskHostNodeId

surayya-MS · 2025-09-29T14:45:04Z

Closing this in favor of #12577

surayya-MS added 2 commits September 12, 2025 20:00

change NodeProviderOutOfProcTaskHost to manage pools of nodes:

3321023

previously _nodeContext was dict with key of HandshakeOptions, now it is nodeId, previously NodeProviderOutOfProcTaskHost allowed max 4 nodes depending on the arch and etc., now it is unlimited

remove unused prop

9bae89a

Copilot AI review requested due to automatic review settings September 12, 2025 18:03

Copilot AI reviewed Sep 12, 2025

View reviewed changes

surayya-MS marked this pull request as draft September 12, 2025 18:06

surayya-MS self-assigned this Sep 12, 2025

Merge branch 'main' into out-of-proc-task-host-pools

46cfde5

surayya-MS changed the title ~~Allow NodeProviderOutOfProcTaskHost to manage multiple nodes not just 4~~ Allow NodeProviderOutOfProcTaskHost to manage multiple nodes instead of one per arch Sep 13, 2025

surayya-MS changed the title ~~Allow NodeProviderOutOfProcTaskHost to manage multiple nodes instead of one per arch~~ Allow NodeProviderOutOfProcTaskHost to manage multiple nodes instead of one per arch Sep 13, 2025

AR-May reviewed Sep 15, 2025

View reviewed changes

src/Build/BackEnd/Components/Communications/NodeProviderOutOfProcTaskHost.cs Outdated Show resolved Hide resolved

surayya-MS added 2 commits September 15, 2025 09:52

return int.MaxValue for AvailableNodes, fix error message, cleanup co…

d3b705b

…mment

remove lock since _odeContexts is now ConcurrentDictionary

34e5365

surayya-MS marked this pull request as ready for review September 15, 2025 08:07

JanProvaznik reviewed Sep 15, 2025

View reviewed changes

src/Build/BackEnd/Components/Communications/NodeProviderOutOfProcTaskHost.cs Outdated Show resolved Hide resolved

surayya-MS added 2 commits September 15, 2025 11:50

remove lock on _nodeContexts

4912cf7

use LockType

8a43dcc

JanProvaznik reviewed Sep 15, 2025

View reviewed changes

JanProvaznik mentioned this pull request Sep 24, 2025

[WIP] Route nonthreadsafe Tasks to taskhosts in mt mode #12459

Closed

Merge branch 'main' into out-of-proc-task-host-pools

2a2140f

surayya-MS closed this Sep 29, 2025

surayya-MS removed their assignment Sep 29, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Allow `NodeProviderOutOfProcTaskHost` to manage multiple nodes instead of one per arch #12521

Allow `NodeProviderOutOfProcTaskHost` to manage multiple nodes instead of one per arch #12521

Uh oh!

surayya-MS commented Sep 12, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

surayya-MS commented Sep 12, 2025

Uh oh!

Uh oh!

AR-May commented Sep 15, 2025

Uh oh!

surayya-MS commented Sep 15, 2025 •

edited

Loading

Uh oh!

Uh oh!

surayya-MS commented Sep 15, 2025 •

edited

Loading

Uh oh!

JanProvaznik Sep 15, 2025

Uh oh!

JanProvaznik Sep 15, 2025

Uh oh!

surayya-MS commented Sep 29, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Allow NodeProviderOutOfProcTaskHost to manage multiple nodes instead of one per arch #12521

Allow NodeProviderOutOfProcTaskHost to manage multiple nodes instead of one per arch #12521

Uh oh!

Conversation

surayya-MS commented Sep 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Context

Changes Made

Testing

Notes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Key Changes

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

surayya-MS commented Sep 12, 2025

Uh oh!

Uh oh!

AR-May commented Sep 15, 2025

Uh oh!

surayya-MS commented Sep 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

surayya-MS commented Sep 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JanProvaznik Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

JanProvaznik Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

surayya-MS commented Sep 29, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Allow `NodeProviderOutOfProcTaskHost` to manage multiple nodes instead of one per arch #12521

Allow `NodeProviderOutOfProcTaskHost` to manage multiple nodes instead of one per arch #12521

surayya-MS commented Sep 12, 2025 •

edited

Loading

surayya-MS commented Sep 15, 2025 •

edited

Loading

surayya-MS commented Sep 15, 2025 •

edited

Loading