Skip to content

Commit f87a18c

Browse files
AR-MayAR-May
authored
Thread-Safe Tasks spec (#12111)
Related to #11828. This spec defines the API contract for the new kind of Task that would be safe to use in multithreaded mode of execution.
1 parent ccf3981 commit f87a18c

File tree

3 files changed

+303
-1
lines changed

3 files changed

+303
-1
lines changed

‎documentation/specs/multithreading/multithreaded-msbuild.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -217,7 +217,7 @@ With a sidecar TaskHost per node, tasks will get the same constraints and freedo
217217

218218
## Thread-safe tasks
219219

220-
To mark that a task is multithreaded-MSBuild-aware, we will introduce a new interface that tasks can implement. We will provide a new `TaskExecutionContext` object with information about the task invocation, including the current environment and working directory, so that tasks can access the same information they would have in a single-threaded or out-of-process execution.
220+
To mark that a task is multithreaded-MSBuild-aware, we will introduce a new interface that tasks can implement. We will provide a new `TaskEnvironment` object with information about the task invocation, including the current environment and working directory, so that tasks can access the same information they would have in a single-threaded or out-of-process execution.
221221

222222
To ease task authoring, we will provide a Roslyn analyzer that will check for known-bad API usage, like `System.Environment.GetEnvironmentVariable` or `System.IO.Directory.SetCurrentDirectory`, and suggest alternatives that use the object provided by the engine (such as `context.GetEnvironmentVariable`).
223223

‎documentation/specs/multithreading/thread-safe-tasks-api-analysis.md‎

Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
# Thread-Safe Tasks: API Analysis Reference (DRAFT)
2+
3+
This document provides a list of .NET APIs that should not be used or should be used with caution in thread-safe tasks. These APIs are problematic because they either rely on or modify process-level state, which can cause race conditions in multithreaded execution.
4+
5+
The APIs listed in this document will be detected by Roslyn analyzers and/or MSBuild BuildCheck to help identify potential threading issues in tasks that implement `IMultiThreadableTask`.
6+
7+
**Note**: The analyzers rely on **static code analysis** and may not catch all dynamic scenarios (such as reflection-based API calls).
8+
9+
## API Issues Categories
10+
11+
Categories of threading issues with .NET API usage in thread-safe tasks to be aware of:
12+
13+
1. **Working Directory Modifications and Usage**, such as file system operations with relative paths.
14+
1. **Environment Variables Modification and Usage**
15+
1. **Process Culture Modification and Usage**, which can affect data formatting.
16+
1. **Assembly Loading**
17+
1. **Static Fields**
18+
19+
### Best Practices
20+
21+
Instead of the problematic APIs listed below, thread-safe tasks should:
22+
23+
1. **Use `TaskEnvironment`** for all file system operations, environment variable changes, and working directory changes.
24+
1. **Always use absolute paths** when still using some standard .NET file system APIs.
25+
1. **Explicitly configure external processes** using `TaskEnvironment`.
26+
1. **Never modify process culture**: Avoid modifying culture defaults.
27+
28+
## Detailed API Reference
29+
30+
The following tables list specific .NET APIs and their threading safety classification:
31+
32+
### System.IO.Path Class
33+
34+
| API | Level | Short Reason | Recommendation |
35+
|-----|-------|--------------|-------|
36+
| `Path.GetFullPath(string path)` | ERROR | Uses current working directory | Use MSBuild API |
37+
38+
### System.IO.File Class
39+
40+
| API | Level | Short Reason | Recommendation |
41+
|-----|-------|--------------|-------|
42+
| All methods | ERROR | Uses current working directory | Use absolute paths |
43+
44+
### System.IO.Directory Class
45+
46+
| API | Level | Short Reason | Recommendation |
47+
|-----|-------|--------------|-------|
48+
| All methods | ERROR | Uses current working directory | Use absolute paths |
49+
50+
### System.Environment Class
51+
52+
| API | Level | Short Reason | Recommendation |
53+
|-----|-------|--------------|-------|
54+
| All properties setters | ERROR | Modifies process-level state | Use MSBuild API |
55+
| `Environment.CurrentDirectory` (getter, setter) | ERROR | Accesses process-level state | Use MSBuild API |
56+
| `Environment.Exit(int exitCode)` | ERROR | Terminates entire process | Return false from task or throw exception |
57+
| `Environment.FailFast` all overloads | ERROR | Terminates entire process | Return false from task or throw exception |
58+
| All other methods | ERROR | Modifies process-level state | Use MSBuild API |
59+
60+
### System.IO.FileInfo Class
61+
62+
| API | Level | Short Reason | Recommendation |
63+
|-----|-------|--------------|-------|
64+
| Constructor `FileInfo(string fileName)` | WARNING | Uses current working directory | Use absolute paths |
65+
| `CopyTo` all overloads | WARNING | Destination path relative to current directory | Use absolute paths |
66+
| `MoveTo` all overloads | WARNING | Destination path relative to current directory | Use absolute paths |
67+
| `Replace` all overloads | WARNING | Paths relative to current directory | Use absolute paths |
68+
69+
### System.IO.DirectoryInfo Class
70+
71+
| API | Level | Short Reason | Recommendation |
72+
|-----|-------|--------------|-------|
73+
| Constructor `DirectoryInfo(string path)` | WARNING | Uses current working directory | Use absolute paths |
74+
| `MoveTo(string destDirName)` | WARNING | Destination path relative to current directory | Use absolute paths |
75+
76+
### System.IO.FileStream Class
77+
78+
| API | Level | Short Reason | Recommendation |
79+
|-----|-------|--------------|-------|
80+
| Constructor `FileStream` all overloads | WARNING | Uses current working directory | Use absolute paths |
81+
82+
### System.IO Stream Classes
83+
84+
| API | Level | Short Reason | Recommendation |
85+
|-----|-------|--------------|-------|
86+
| Constructor `StreamReader` all overloads | WARNING | Uses current working directory | Use absolute paths |
87+
88+
### System.Diagnostics.Process Class
89+
90+
| API | Level | Short Reason | Recommendation |
91+
|-----|-------|--------------|-------|
92+
| All properties setters | ERROR | Modifies process-level state | Avoid |
93+
| `Process.GetCurrentProcess().Kill()` | ERROR | Terminates entire process | Avoid |
94+
| `Process.GetCurrentProcess().Kill(bool entireProcessTree)` | ERROR | Terminates entire process | Avoid |
95+
| `Process.Start` all overloads | ERROR | May inherit process state | Use MSBuild API |
96+
97+
### System.Diagnostics.ProcessStartInfo Class
98+
99+
| API | Level | Short Reason | Recommendation |
100+
|-----|-------|--------------|-------|
101+
| Constructor `ProcessStartInfo()` all overloads | ERROR | May inherit process state | Use MSBuild API |
102+
103+
### System.Threading.ThreadPool Class
104+
105+
| API | Level | Short Reason | Recommendation |
106+
|-----|-------|--------------|-------|
107+
| `ThreadPool.SetMinThreads(int workerThreads, int completionPortThreads)` | ERROR | Modifies process-wide settings | Avoid |
108+
| `ThreadPool.SetMaxThreads(int workerThreads, int completionPortThreads)` | ERROR | Modifies process-wide settings | Avoid |
109+
110+
### System.Globalization.CultureInfo Class
111+
112+
| API | Level | Short Reason | Recommendation |
113+
|-----|-------|--------------|-------|
114+
| `CultureInfo.DefaultThreadCurrentCulture` (setter) | ERROR | Affects new threads | Modify the thread culture instead |
115+
| `CultureInfo.DefaultThreadCurrentUICulture` (setter) | ERROR | Affects new threads | Modify the thread culture instead |
116+
117+
### Static
118+
119+
| API | Level | Short Reason | Recommendation |
120+
|-----|-------|--------------|-------|
121+
| Static fields | WARNING | Shared across threads, can cause race conditions | Avoid |
122+
123+
### Assembly Loading (System.Reflection.Assembly class, System.Activator class)
124+
Tasks that load assemblies dynamically in the task host may cause version conflicts. Version conflicts in task assemblies will cause build failures (previously these might have been sporadic). Both dynamically loaded dependencies and static dependencies can cause issues.
125+
126+
| API | Level | Short Reason | Recommendation |
127+
|-----|-------|--------------|-------|
128+
| `Assembly.LoadFrom(string assemblyFile)` | WARNING | May cause version conflicts | Be aware of potential conflicts, use absolute paths |
129+
| `Assembly.LoadFile(string path)` | WARNING | May cause version conflicts | Be aware of potential conflicts |
130+
| `Assembly.Load` all overloads | WARNING | May cause version conflicts | Be aware of potential conflicts |
131+
| `Assembly.LoadWithPartialName(string partialName)` | WARNING | May cause version conflicts | Be aware of potential conflicts |
132+
| `Activator.CreateInstanceFrom(string assemblyFile, string typeName)` | WARNING | May cause version conflicts | Be aware of potential conflicts |
133+
| `Activator.CreateInstance(string assemblyName, string typeName)` | WARNING | May cause version conflicts | Be aware of potential conflicts |
134+
| `AppDomain.Load` all overloads | WARNING | May cause version conflicts | Be aware of potential conflicts |
135+
| `AppDomain.CreateInstanceFrom(string assemblyFile, string typeName)` | WARNING | May cause version conflicts | Be aware of potential conflicts |
136+
| `AppDomain.CreateInstance(string assemblyName, string typeName)` | WARNING | May cause version conflicts | Be aware of potential conflicts |
137+
138+
### P/Invoke
139+
140+
**Concerns**:
141+
- P/Invoke calls may use process-level state like current working directory
142+
- Native code may not be thread-safe
143+
- Native APIs may modify global process state
144+
145+
| API | Level | Short Reason | Recommendation |
146+
|-----|-------|--------------|-------|
147+
| `[DllImport]` attribute | WARNING | Not covered by analyzers | Review for thread safety, use absolute paths |

‎documentation/specs/multithreading/thread-safe-tasks.md‎

Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
# Thread-Safe Tasks
2+
3+
## Overview
4+
5+
MSBuild's current execution model assumes that tasks have exclusive control over the entire process during execution. This allows tasks to freely modify global process state such as environment variables, the current working directory, and other process-level resources. This design works well for MSBuild's approach of executing builds in separate processes for parallelization. With the introduction of multithreaded execution within a single MSBuild process, multiple tasks can now run concurrently. This requires a new task design to ensure that multiple tasks do not access/modify shared process state, and the relative paths are resolved correctly.
6+
7+
To enable this multithreaded execution model, tasks will declare their capability to run in multiple threads within one process. These capabilities are referred to as **thread-safety** capabilities and the corresponding tasks are called **thread-safe tasks**. Thread-safe tasks must avoid using APIs that modify or depend on global process state, as this could cause conflicts when multiple tasks execute concurrently. See [Thread-Safe Tasks API Analysis Reference](thread-safe-tasks-api-analysis.md) for detailed guidelines. Task authors will also get access to a `TaskEnvironment` that provides safe alternatives to global process state APIs. For example, task authors should use `TaskEnvironment.GetAbsolutePath()` instead of `Path.GetFullPath()` to ensure correct path resolution in multithreaded scenarios.
8+
9+
Tasks that are not thread-safe can still participate in multithreaded builds. MSBuild will execute these tasks in separate TaskHost processes to provide process-level isolation.
10+
11+
## Thread-Safe Capability Indicators
12+
13+
Task authors can declare thread-safe capabilities in two different ways:
14+
1. **Interface-Based Thread-Safe Capability Declaration** - Provides access to thread-safe APIs through `TaskEnvironment` to be used in the task code.
15+
2. **Attribute-Based Thread-Safe Capability Declaration** - Allows existing tasks to declare its ability run in multithreaded mode without code changes. It is a **compatibility bridge option**.
16+
17+
Tasks that use `TaskEnvironment` cannot load in older MSBuild versions that do not support multithreading features, requiring authors to drop support for older MSBuild versions. To address this challenge, MSBuild provides a compatibility bridge that allows certain tasks targeting older MSBuild versions to participate in multithreaded builds. While correct absolute path resolution can be and should be achieved without accessing `TaskEnvironment` in tasks that use compatibility bridge options, tasks must avoid relying on environment variables or modifying global process state.
18+
19+
So, task authors who need to support older MSBuild versions will have three choices:
20+
1. **Maintain separate implementations** - Create and support both thread-safe and legacy versions of the same task.
21+
2. **Use compatibility bridge approaches** - Rely on MSBuild's ability to run legacy tasks in multithreaded mode without access to `TaskEnvironment`.
22+
3. **Accept reduced performance** - Tasks will execute more slowly than their thread-safe versions because they must run in a separate TaskHost process
23+
24+
### Interface-Based Thread-Safe Capability Declaration
25+
26+
Tasks indicate thread-safety capabilities by implementing the `IMultiThreadableTask` interface.
27+
28+
```csharp
29+
namespace Microsoft.Build.Framework;
30+
public interface IMultiThreadableTask : ITask
31+
{
32+
TaskEnvironment TaskEnvironment { get; set; }
33+
}
34+
```
35+
36+
Similar to how MSBuild provides the abstract `Task` class with default implementations for the `ITask` interface, MSBuild will offer a `MultiThreadableTask` abstract class with default implementations for the `IMultiThreadableTask` interface. Task authors will only need to implement the `Execute` method for the `ITask` interface and use `TaskEnvironment` within it to create their thread-safe tasks.
37+
38+
```csharp
39+
namespace Microsoft.Build.Utilities;
40+
public abstract class MultiThreadableTask : Task, IMultiThreadableTask
41+
{
42+
public TaskEnvironment TaskEnvironment{ get; set; }
43+
}
44+
```
45+
46+
Task authors who want to support older MSBuild versions need to:
47+
- Maintain both thread-safe and legacy implementations.
48+
- Use conditional task declarations based on MSBuild version to select which assembly to load the task from.
49+
50+
**Note:** Consider backporting `IMultiThreadableTask` to MSBuild 17.14 for graceful failure when the interface is used.
51+
52+
### Attribute-Based Thread-Safe Capability Declaration
53+
54+
Task authors can indicate thread-safety capabilities by marking their task classes with a specific attribute. Tasks marked with this attribute can run in multithreaded builds but do not have access to `TaskEnvironment` APIs.
55+
56+
```csharp
57+
namespace Microsoft.Build.Framework;
58+
[AttributeUsage(AttributeTargets.Class, AllowMultiple = false)]
59+
internal class MSBuildMultiThreadableTaskAttribute : Attribute
60+
{
61+
public MSBuildMultiThreadableTaskAttribute() { }
62+
}
63+
```
64+
65+
MSBuild detects `MSBuildMultiThreadableTaskAttribute` by its namespace and name only, ignoring the defining assembly, which allows customers to define the attribute in their own assemblies alongside their tasks. Since MSBuild does not ship the attribute, customers using newer MSBuild versions should prefer the Interface-Based Thread-Safe Capability Declaration.
66+
67+
For tasks to be eligible for multithreaded execution using this approach, they must satisfy the following conditions:
68+
- The task must not modify global process state (environment variables, working directory)
69+
- The task must not depend on global process state, including relative path resolution
70+
71+
#### API Usage Example
72+
73+
```csharp
74+
[MSBuildMultiThreadableTask]
75+
public class MyTask : Task {...}
76+
```
77+
78+
## TaskEnvironment API
79+
80+
The `TaskEnvironment` provides thread-safe alternatives to APIs that use global process state, enabling tasks to execute safely in a multithreaded environment.
81+
82+
```csharp
83+
namespace Microsoft.Build.Framework;
84+
public interface IMultiThreadableTask : ITask
85+
{
86+
TaskEnvironment TaskEnvironment { get; set; }
87+
}
88+
89+
public class TaskEnvironment
90+
{
91+
public virtual AbsolutePath ProjectDirectory { get; internal set; }
92+
93+
// This function resolves paths relative to ProjectDirectory.
94+
public virtual AbsolutePath GetAbsolutePath(string path);
95+
96+
public virtual string? GetEnvironmentVariable(string name);
97+
public virtual IReadOnlyDictionary<string, string> GetEnvironmentVariables();
98+
public virtual void SetEnvironmentVariable(string name, string? value);
99+
100+
public virtual ProcessStartInfo GetProcessStartInfo();
101+
}
102+
```
103+
104+
The `TaskEnvironment` class that MSBuild provides is not thread-safe. Task authors who spawn multiple threads within their task implementation must provide their own synchronization when accessing the task environment from multiple threads. However, each task receives its own isolated environment object, so synchronization with other concurrent tasks is not required.
105+
106+
### Path Handling
107+
108+
To prevent common thread-safety issues related to path handling, we introduce path type that is implicitly convertible to string:
109+
110+
```csharp
111+
namespace Microsoft.Build.Framework;
112+
public readonly struct AbsolutePath
113+
{
114+
// Default value returns string.Empty for Path property
115+
public string Path { get; }
116+
internal AbsolutePath(string path, bool ignoreRootedCheck) { }
117+
public AbsolutePath(string path); // Checks Path.IsPathRooted
118+
public AbsolutePath(string path, AbsolutePath basePath) { }
119+
public static implicit operator string(AbsolutePath path) { }
120+
public override string ToString() => Path;
121+
}
122+
```
123+
124+
`AbsolutePath` converts implicitly to string for seamless integration with existing File/Directory APIs.
125+
126+
### API Usage Example
127+
128+
```csharp
129+
public bool Execute(...)
130+
{
131+
// Use APIs provided by TaskEnvironment
132+
string envVar = TaskEnvironment.GetEnvironmentVariable("EnvVar");
133+
134+
// Convert string properties to strongly-typed paths and use them in standard File/Directory APIs
135+
AbsolutePath path = TaskEnvironment.GetAbsolutePath("SomePath");
136+
string content = File.ReadAllText(path);
137+
string content2 = File.ReadAllText(path.ToString());
138+
string content3 = File.ReadAllText(path.Path);
139+
...
140+
}
141+
```
142+
143+
## Appendix: Alternatives
144+
145+
This appendix collects alternative approaches considered during design.
146+
147+
### Alternative Approach: API Hooking
148+
149+
An alternative approach to the `TaskEnvironment` API could be to use API hooking (such as Microsoft Detours) to automatically virtualize global process state without requiring any changes from task authors.
150+
151+
The main advantages of API hooking include requiring no action from task authors since existing tasks would work without modification or recompilation, and having no compatibility concerns with older MSBuild versions. However, it would be a Windows-only solution, making it unsuitable for cross-platform scenarios.
152+
153+
### Alternative to Attribute-Based Thread-Safe Capability Declaration
154+
155+
We considered making the thread-safety signal using the task declaration (for example, a `ThreadSafe="true"` attribute on `UsingTask`) so that project authors could declare compatibility without changing task assemblies. However, because older MSBuild versions treat unknown attributes in task declarations as errors, this approach would require updating older MSBuild versions or servicing them to ignore the attribute.

0 commit comments

Comments
 (0)