Skip to content

Commit cb2031d

Browse files
rkargMsftIEvangelist
rkargMsft
and
IEvangelist
authored
[Orleans] Silo Metadata and Placement Filtering (dotnet#44187)
* [Orleans] Silo Metadata and Placement Filtering * Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine <[email protected]> * Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine <[email protected]> * Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine <[email protected]> * Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine <[email protected]> * Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine <[email protected]> * Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine <[email protected]> * Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine <[email protected]> * Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine <[email protected]> * Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine <[email protected]> * Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine <[email protected]> * Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine <[email protected]> * Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine <[email protected]> * Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine <[email protected]> * Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine <[email protected]> * Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine <[email protected]> * Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine <[email protected]> * Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine <[email protected]> * Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine <[email protected]> * Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine <[email protected]> * reworded explaination * fixing markdownlint erros for multiple newlines * Update silo-metadata.md Aligning with changes to implementation. * update orleans toc moved Silo metadata to host configuration section * Apply suggestions from code review Co-authored-by: David Pine <[email protected]> * Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine <[email protected]> * Correcting spacing warnings * Updating example with required parameterless constructor * corrected pathing * Updating configuration example * Nesting Grain placement documents based on feedback * yaml indenting --------- Co-authored-by: David Pine <[email protected]>
1 parent a55249e commit cb2031d

File tree

3 files changed

+241
-2
lines changed

3 files changed

+241
-2
lines changed

‎docs/orleans/grains/grain-placement-filtering.md‎

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
1+
---
2+
title: Grain placement filtering
3+
description: Learn about grain placement filtering in Microsoft Orleans.
4+
5+
ms.date: 01/08/2025
6+
---
7+
8+
# Grain placement filtering
9+
10+
Glain placement filtering in Orleans allows developers additional control over the placement of grains within a cluster. It works in conjunction with placement strategies, adding an additional layer of filtering to determine candidate silos for grain activation.
11+
12+
This filterting takes place before candidate silos are passed on to the configured placement method allowing for more flexibility and reuse of the filters.
13+
14+
For example, the existing `PreferLocal` placement strategy is hard coded to fall back to `Random` placement if the local silo is unable to host the grain type. But by using filters, a `PreferLocalPlacementFilter` could be implemented to filter down to either the local silo or all compatible silos. Then any placement strategy (`Random`, `ResourceOptimizedPlacement`, `ActivationCount`, etc.) could be configured for that grain type. This allows for any set of filters and any placement strategy to be configured for a grain type.
15+
16+
## How placement filtering works
17+
18+
Placement filtering operates as an additional step in the grain placement process. After all compatible silos for the grain type are identified, all placement filters configured for that grain type, if any, are applied to allow further refinement of the selection by eliminating silos that do not meet the defined criteria.
19+
20+
### Ordering
21+
22+
Filters running in different orders may result in different behavior so explicit ordering is required when two or more filters are defined on a type. This needs to be configured with the `order:` parameter, as the type metadata pulled at runtime may return the attributes on a type in a different order from how they appear in the source code. Ordering must have unique values so an explicit ordering can be determined.
23+
24+
## Built-in filters
25+
26+
Orleans provides various built-in filters for you to choose from. However, if you are unable to find one that suits your needs you can always [implement your own](#implement-placement-filters).
27+
28+
### Silo metadata
29+
30+
These filters work with [*Silo Metadata*](../host/configuration-guide/silo-metadata.md) to filter candidate silos.
31+
32+
#### `RequiredMatchSiloMetadata`
33+
34+
Silo Metadata is used to filter candidate silos to only ones that matches all of the specified metadata keys with the calling silo. If there are no compatible silos that match all of the keys then an empty set of silos will be returned and placement will ultimately fail for the grain.
35+
36+
#### `PreferredMatchSiloMetadata`
37+
38+
This filtering will attempt to select only silos that match all of the configured metadata keys with the values of the calling silo. However, instead of returning an empty set if there are not matches as the above Required filtering this will then fall back to partial matches. The first configured metadata key is dropped and a match is made against the remaining keys. This will continue, dropping the initial keys, until a sufficient number of matches are made. If there are not any compatible silos that match *any* of the metadata keys, then all of the candidate silos are returned.
39+
40+
The `minCandidates` value configures how many candidates must be found to stop the filtering process. This value is used to prevent a single silo or small number of silos from getting quickly overloaded if the match were that small.
41+
42+
If `minCandidates` were not considered, then there could be a scenario where there are a large number of silos but only one silo that best matches the configured metadata keys. All placements would be concentrated on that one silo despite having many more available that could host activations. The purpose of `minCandidates` is to allow for a balance between preferring only best matches and avoiding hot silos. It is often desirable to not focus activation (or do scheduling in general) on one target. Set it to a value larger than 1 to ensure a minimum candidate set size so that future placement decisions are able to avoid concentrating activations on one or a few hot silos. Note that this config is a minimum value; more candidates could be returned. If you would prefer most specific matching only then set `minCandidates: 1` to always prefer best match. This might be preferable in specific use cases where there is low activation throughput and where there is a great penalty when moving to a less specific match from a more specific one. In general use, the default value of 2 should be used (and not need to be specified in the attribute).
43+
44+
For example, if filtering on `[PreferredMatchSiloMetadata(["cloud.availability-zone", "cloud.region"], minCandidates:2)]` and there is only one matching silo on both `cloud.availability-zone` and `cloud.region`, then this scenario with `minCandidates: 2` would fail to match on both keys because only one silo matches both metadata keys and that's below the minimum configured size of 2. It would then then fall back to matching only on `cloud.region`. If there were 2 or more silos that match only `cloud.region` then those would get returned. Otherwise, it would fall back to returning all of the candidates.
45+
46+
## Implement placement filters
47+
48+
To implement a custom placement filter in Orleans, follow these steps:
49+
50+
1. **Implementation**
51+
- Create marker Attribute derived from `PlacementFilterAttribute`
52+
- Create Strategy derived from `PlacementFilterStrategy` to manage any configuration values
53+
- Create Director derived from `IPlacementFilterDirector` which contains the filtering logic
54+
- Define the filtering logic in the `Filter` method, which takes a list of candidate silos and returns a filtered list.
55+
56+
2. **Register the Filter**
57+
- Call `AddPlacementFilter` to register the Strategy and corresponding Director
58+
59+
3. **Apply the Filter**
60+
- Add the Attribute to a grain class to apply the filter
61+
62+
Here is an example of a simple custom Placement Filter. It is similar in behavior to using `[PreferLocalPlacement]` without any filter, but this has the advantage of being able to specify any placement method. Whereas `PreferLocalPlacement` falls back to Random placement if the local silo is unable to host a grain, this example has configured `ActivationCountBasedPlacement`. Any other placement could similarly be used with this filter
63+
64+
```csharp
65+
[AttributeUsage(
66+
AttributeTargets.Class, AllowMultiple = false)]
67+
public class ExamplePreferLocalPlacementFilterAttribute(int order)
68+
: PlacementFilterAttribute(
69+
new ExamplePreferLocalPlacementFilterStrategy(order));
70+
```
71+
72+
```csharp
73+
public class ExamplePreferLocalPlacementFilterStrategy(int order)
74+
: PlacementFilterStrategy(order)
75+
{
76+
public ExamplePreferLocalPlacementFilterStrategy() : this(0) { }
77+
}
78+
```
79+
80+
```csharp
81+
internal class ExamplePreferLocalPlacementFilterDirector(
82+
ILocalSiloDetails localSiloDetails)
83+
: IPlacementFilterDirector
84+
{
85+
public IEnumerable<SiloAddress> Filter(PlacementFilterStrategy filterStrategy, PlacementTarget target, IEnumerable<SiloAddress> silos)
86+
{
87+
var siloList = silos.ToList();
88+
var localSilo = siloList.FirstOrDefault(s => s == localSiloDetails.SiloAddress);
89+
if (localSilo is not null)
90+
{
91+
return [localSilo];
92+
}
93+
return siloList;
94+
}
95+
}
96+
```
97+
98+
After implementing this filter, it can be registered and applied to grains.
99+
100+
```csharp
101+
builder.ConfigureServices(services =>
102+
{
103+
services.AddPlacementFilter<
104+
ExamplePreferLocalPlacementFilterStrategy,
105+
ExamplePreferLocalPlacementFilterDirector>();
106+
107+
});
108+
```
109+
110+
```csharp
111+
[ExamplePreferLocalPlacementFilter]
112+
[ActivationCountBasedPlacement]
113+
public class MyGrain() : Grain, IMyGrain
114+
{
115+
// ...
116+
}
117+
```

‎docs/orleans/host/configuration-guide/silo-metadata.md‎

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
---
2+
title: Silo metadata
3+
description: Learn about silo metadata in .NET Orleans.
4+
ms.date: 01/08/2025
5+
---
6+
7+
# Silo metadata
8+
9+
Silo metadata is a feature in Orleans that allows developers to assign custom metadata to silos within a cluster. This metadata provides a flexible mechanism for annotating silos with descriptive information or specific capabilities.
10+
11+
This feature is particularly useful in scenarios where different silos have distinct roles, hardware configurations, or other unique characteristics. For example, silos can be tagged based on their region, compute power, or specialized responsibilities within the system.
12+
13+
Silo metadata lays the groundwork for additional Orleans features, such as [Grain placement filtering](../../grains/grain-placement-filtering.md).
14+
15+
## Key concepts
16+
17+
Silo metadata introduces a way to attach key-value pairs to silos within an Orleans cluster. This feature allows developers to configure silo-specific characteristics that can be leveraged by Orleans components.
18+
19+
Silo metadata is represented as an **immutable** dictionary of key-value pairs:
20+
21+
- **Keys**: Strings that identify the metadata (for example, `"cloud.region"`, `"compute.reservation.type"`).
22+
- **Values**: Strings that describe the corresponding property (for example, `"us-east1"`, `"spot"`).
23+
24+
## Configuration
25+
26+
Silo metadata in Orleans is configured using two methods, either .NET configuration or directly in code.
27+
28+
### Configure Silo metadata with configuration
29+
30+
Silo metadata can be defined in the app's configuration, such as _appsettings.json_, environment variables, or any other available configuration source.
31+
32+
#### Example: _appsettings.json_ configuration
33+
34+
```json
35+
{
36+
"Orleans": {
37+
"Metadata": {
38+
"cloud.region": "us-east1",
39+
"compute.reservation.type": "spot",
40+
"role": "worker"
41+
}
42+
}
43+
}
44+
```
45+
46+
The preceding configuration defines metadata for a Silo, tagging it with:
47+
48+
- `cloud.region`: `"us-east1"`
49+
- `compute.reservation.type`: `"spot"`
50+
- `role`: `"worker"`
51+
52+
To apply this configuration, use the following setup in your silo host builder:
53+
54+
```csharp
55+
Host.CreateApplicationBuilder(args).UseOrleans(siloBuilder =>
56+
{
57+
// Configuration section Orleans:Metadata is used by default
58+
siloBuilder.UseSiloMetadata();
59+
});
60+
```
61+
62+
Alternatively, an explicit `IConfiguration` or `IConfigurationSection` can be passed in to control where in configuration the metadata is pulled from.
63+
64+
### Configuring silo metadata in code
65+
66+
For scenarios requiring programmatic metadata configuration, developers can add metadata directly in the Silo host builder.
67+
68+
#### Example: Direct Code Configuration
69+
70+
```csharp
71+
var siloBuilder = new SiloHostBuilder()
72+
.UseSiloMetadata(new Dictionary<string, string>
73+
{
74+
["cloud.region"] = "us-east1",
75+
["compute.reservation.type"] = "spot",
76+
["role"] = "worker"
77+
});
78+
79+
```
80+
81+
The preceding example achieves the same result as the JSON configuration but allows metadata values to be computed or loaded dynamically during Silo initialization.
82+
83+
### Merge configurations
84+
85+
If both .NET configuration and direct code configuration are used, the direct configuration overrides any conflicting metadata values from the .NET configuration. This allows developers to set defaults with configuration files and dynamically adjust specific metadata during runtime.
86+
87+
## Usage
88+
89+
Developers can retrieve metadata through the `ISiloMetadataCache` interface. This interface allows for querying metadata for individual silos across the cluster. Metadata will always be returned from a local cache of metadata that gets updated in the background as cluster membership changes.
90+
91+
### Access metadata for a specific silo
92+
93+
The `ISiloMetadataCache` provides a method to retrieve the metadata for a specific silo by its unique identifier (`SiloAddress`). The `ISoloMetadataCache` implementation is registered in the `UseSiloMetadata` method and can be injected as a dependency.
94+
95+
#### Example: Access metadata for a Silo
96+
97+
```csharp
98+
var siloMetadata = siloMetadataCache.GetSiloMetadata(siloAddress);
99+
100+
if (siloMetadata.Metadata.TryGetValue("role", out var role))
101+
{
102+
Console.WriteLine($"Silo Role for {siloAddress}: {role}");
103+
// Execute role-specific logic
104+
}
105+
```
106+
107+
In the preceding example:
108+
109+
- `GetSiloMetadata(siloAddress)` retrieves the metadata for the specified silo.
110+
- Metadata keys like `"role"` can be used to influence application logic.
111+
112+
## Internal implementation
113+
114+
Internally, the `SiloMetadataCache` monitors changes in cluster membership on `MembershipTableManager` and keeps a local cache of metadata in sync with membership changes. Metadata is immutable for a given Silo, so it's retrieved once and cached until that Silo leaves the cluster. Cached metadata for clusters that are `Dead` or have left the membership table will be cleared out of the local cache.
115+
116+
Each silo hosts an `ISystemTarget` that provides that silo's metadata. Calls to `SiloMetadataCache : ISiloMetadataCache` return a result from the local cache.

‎docs/orleans/toc.yml‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,8 +24,12 @@ items:
2424
href: grains/grain-references.md
2525
- name: Grain identity
2626
href: grains/grain-identity.md
27-
- name: Grain placement
28-
href: grains/grain-placement.md
27+
- name: Grain placement
28+
items:
29+
- name: Overview
30+
href: grains/grain-placement.md
31+
- name: Grain placement filtering
32+
href: grains/grain-placement-filtering.md
2933
- name: Grain extensions
3034
href: grains/grain-extensions.md
3135
- name: Timers and reminders
@@ -138,6 +142,8 @@ items:
138142
href: host/configuration-guide/typical-configurations.md
139143
- name: Options classes
140144
href: host/configuration-guide/list-of-options-classes.md
145+
- name: Silo metadata
146+
href: host/configuration-guide/silo-metadata.md
141147
- name: Activation collection
142148
href: host/configuration-guide/activation-collection.md
143149
- name: Configure .NET garbage collection

0 commit comments

Comments
 (0)