Skip to content

Conversation

@JanProvaznik
Copy link
Member

@JanProvaznik JanProvaznik commented Aug 29, 2025

Fixes #12196

Context

before we implement the mechanism for detection how to route tasks via interfaces we can have this intermediate solution for tasks in our repo that we convert to thread safe

Changes Made

  1. NodeProviderOutOfProcTaskHost manages pools of TaskHosts of given architecture instead of one per each
  2. allowlist for thread safe tasks

Testing

TBD

testing project
<Project>
    <ItemGroup>
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=1" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=2" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=3" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=4" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=5" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=6" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=7" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=8" />
       <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=9" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=10" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=11" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=12" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=13" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=14" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=15" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=16" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=17" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=18" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=19" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=20" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=21" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=22" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=23" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=24" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=25" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=26" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=27" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=28" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=29" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=30" />
        <P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=31" />
	<P Include="$(MSBuildThisFileFullPath)" AdditionalProperties="Disambiguator=32" />
	
    </ItemGroup>

    <!-- Define source files to copy (example) -->
    <ItemGroup>
        <SourceFiles Include="$(MSBuildProjectDirectory)\*.txt" />
        <SourceFiles Include="$(MSBuildProjectDirectory)\*.xml" />
    </ItemGroup>

    <PropertyGroup>
        <!-- Base output directory -->
        <OutputBaseDir>$(MSBuildProjectDirectory)\Output</OutputBaseDir>
    </PropertyGroup>

    <Target Name="Go">
        <MSBuild Projects="@(P)"
                 Targets="Inner"
                 BuildInParallel="true" />
    </Target>

    <Target Name="Inner">
        <Message Text="Running Inner Target on $(Disambiguator)" Importance="High" />
        
        <!-- Create a unique output directory for each parallel execution -->
        <PropertyGroup>
            <OutputDir>$(OutputBaseDir)\Instance_$(Disambiguator)</OutputDir>
        </PropertyGroup>

        <!-- Ensure the output directory exists -->
        <MakeDir Directories="$(OutputDir)" />

        <!-- Copy files to the instance-specific directory -->
        <Copy SourceFiles="@(SourceFiles)"
              DestinationFolder="$(OutputDir)"
              SkipUnchangedFiles="true"
              Retries="3"
              RetryDelayMilliseconds="300">
            <Output TaskParameter="CopiedFiles" ItemName="CopiedFilesList"/>
        </Copy>

        <Message Text="Copied @(CopiedFilesList->Count()) files to $(OutputDir)" Importance="High" />
    </Target>
</Project>

-> in binlog we see that message (on allowlist) is inproc, and that copy is outofproc
but this needs unit tests coverage too

Notes

@JanProvaznik
Copy link
Member Author

closing in favor of #12521

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement task routing for multi-threaded execution

2 participants