Add a new configuration to convert things to scope (#3037)

* add the configuration but it is untested yet

* remove a temporary configuration since it is no longer needed or used in our SDKs

* various changes and fixes for assigning a path as scope

* fix some issues around usual scope

* a format

* introduce a patch especially for resourcemanager

* generalized the previous fix

* another fix for recent changes

* fix test cases

* add a new case in test project for the new scope

* update documents

* add test case to the testgen as well

* fix the base test on scope resources

* add some more clarification for generate arm resource extensions configuration

* add some description

* fix a using issue in generated samples

Author: ArcturusZhang

docs/mgmt/readme.md

@@ -650,12 +650,34 @@ and since it is a scope resource without any configuration, its parent is anythi
 ```csharp
 public static partial class ResourcesExtension
 {
-    public static DeploymentCollection GetDeployments(this ArmResource armResource)
+    public static DeploymentCollection GetDeployments(this ArmClient client, ResourceIdentifier scope)
     {
         /* ... */
     }
 }
 ```
+If you would like to generate an extension method of the `ArmResource` class for this scope resource, you need to add this configuration `generate-arm-resource-extensions` by adding the request path of this scope resource:
+```yaml
+generate-arm-resource-extensions:
+- /{scope}/providers/Microsoft.Resources/deployments/{deploymentName}
+```
+And you will see this change in the generated code:
+```diff
+public static partial class ResourcesExtension
+{
+    public static DeploymentCollection GetDeployments(this ArmClient client, ResourceIdentifier scope)
+    {
+        /* ... */
+    }
++
++   public static DeploymentCollection GetDeployments(this ArmResource armResource)
++   {
++       /* ... */
++   }
+}
+```
+Please note this extension methods have a huge side effect: because all `Resource` class would inherit from `ArmResource`, once the user import the namespace of this SDK, they would see this `GetDeployments` method on any `Resource` instance, while in the real life, this scope resource might not be that general to be applied onto any resource coming from any RP. Please only use this configuration when it could be confirmed to support plenty of resources and has the plan to support more.
+
 To assign specific resource types to this scope, you can use the following configuration:
 
 ```yaml
@@ -689,6 +711,65 @@ public static partial class ResourcesExtension
 }
 ```
 
+In some cases, we might have a resource that extends another resource from another RP. For instance this resource in `guestconfiguration` RP: `/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Compute/virtualMachines/{vmName}/providers/Microsoft.GuestConfiguration/guestConfigurationAssignments/{guestConfigurationAssignmentName}` extends another resource virtual machine in the `compute` RP.
+
+By default, because the generator will never find the `VirtualMachineResource` when generating this SDK, the parent resource of this `VmGuestConfigurationResource` will be `ResourceGroupResource`.
+```csharp
+public static partial class GuestConfigurationExtensions
+{
+    public static VmGuestConfigurationCollection GetVmGuestConfigurations(this ResourceGroupResource resourceGroup, string vmName)
+    {
+        /* ... */
+    }
+}
+```
+
+To show the relationship between resources across different RPs, we could convert it into a scope resource by using the following configuration:
+```yaml
+parameterized-scopes:
+- /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Compute/virtualMachines/{vmName}
+```
+This configuration registers the listed request paths as resources the generator could recognize even if they might not exist in the current context. After applying this configuration, the generated code will have the following changes:
+```diff
+public static partial class GuestConfigurationExtensions
+{
+-   public static VmGuestConfigurationCollection GetVmGuestConfigurations(this ResourceGroupResource resourceGroup, string vmName)
+-   {
+-       /* ... */
+-   }
++   public static VmGuestConfigurationCollection GetVmGuestConfigurations(this ArmClient client, ResourceIdentifier scope)
++   {
++       if (!scope.ResourceType.Equals("Microsoft.Compute/virtualMachines"))
++           throw new InvalidOperationException(string.Format("Invalid resource type {0} expected Microsoft.Compute/virtualMachines", scope.ResourceType));
++       /* ... */
++   }
+}
+```
+
+This configuration works fine with the configuration introduced above:
+```yaml
+generate-arm-resource-extensions:
+- /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Compute/virtualMachines/{vmName}/providers/Microsoft.GuestConfiguration/guestConfigurationAssignments/{guestConfigurationAssignmentName}
+```
+and this is how the generated code would change:
+```diff
+public static partial class GuestConfigurationExtensions
+{
+    public static VmGuestConfigurationCollection GetVmGuestConfigurations(this ArmClient client, ResourceIdentifier scope)
+    {
+        if (!scope.ResourceType.Equals("Microsoft.Compute/virtualMachines"))
+            throw new InvalidOperationException(string.Format("Invalid resource type {0} expected Microsoft.Compute/virtualMachines", scope.ResourceType));
+        /* ... */
+    }
++
++   public static VmGuestConfigurationCollection GetVmGuestConfigurations(this ArmResource armResource)
++   {
++       /* ... */
++   }
+}
+```
+Please note this `ArmResource` extension method has the same side effect as explained above: this extension is only meant to be got or created under the virtual machine resource, but the extension method will let you see this method on any resource instance because they all inherit from `ArmResource`.
+
 ### SDK polishing configurations
 
 During the SDK review, we would like to make some polish to our generated SDK according to the review comments, for instance, changing names of types, properties, making type of properties more specific, etc. To achieve these, you will need the SDK polishing configurations.

src/AutoRest.CSharp/Mgmt/AutoRest/MgmtConfiguration.cs

@@ -3,9 +3,11 @@
 
 using System;
 using System.Collections.Generic;
+using System.Collections.Immutable;
 using System.Linq;
 using System.Text.Json;
 using AutoRest.CSharp.AutoRest.Communication;
+using AutoRest.CSharp.Mgmt.Models;
 
 namespace AutoRest.CSharp.Input
 {
@@ -95,7 +97,7 @@ public MgmtConfiguration(
             IReadOnlyList<string> noResourceSuffix,
             IReadOnlyList<string> schemasToPrependRPPrefix,
             IReadOnlyList<string> generateArmResourceExtensions,
-            IReadOnlyList<string> preventWrappingReturnType,
+            IReadOnlyList<string> parameterizedScopes,
             MgmtDebugConfiguration mgmtDebug,
             JsonElement? requestPathToParent = default,
             JsonElement? requestPathToResourceName = default,
@@ -164,11 +166,7 @@ public MgmtConfiguration(
             NoResourceSuffix = noResourceSuffix;
             PrependRPPrefix = schemasToPrependRPPrefix;
             GenerateArmResourceExtensions = generateArmResourceExtensions;
-            if (preventWrappingReturnType.Any())
-            {
-                Console.Error.WriteLine($"WARNING: The configuration 'prevent-wrapping-return-type' is a workaround and will be removed in the future.");
-            }
-            PreventWrappingReturnType = preventWrappingReturnType;
+            RawParameterizedScopes = parameterizedScopes;
             IsArmCore = Configuration.DeserializeBoolean(armCore, false);
             DoesResourceModelRequireType = Configuration.DeserializeBoolean(resourceModelRequiresType, true);
             DoesResourceModelRequireName = Configuration.DeserializeBoolean(resourceModelRequiresName, true);
@@ -211,6 +209,10 @@ private static Dictionary<TKey, TValue> DeserializeDictionary<TKey, TValue>(Json
         public IReadOnlyDictionary<string, string[]> RequestPathToScopeResourceTypes { get; }
         public IReadOnlyDictionary<string, string[]> OperationPositions { get; }
         public IReadOnlyDictionary<string, string[]> MergeOperations { get; }
+        public IReadOnlyList<string> RawParameterizedScopes { get; }
+        private ImmutableHashSet<RequestPath>? _parameterizedScopes;
+        internal ImmutableHashSet<RequestPath> ParameterizedScopes
+            => _parameterizedScopes ??= RawParameterizedScopes.Select(scope => RequestPath.FromString(scope)).ToImmutableHashSet();
         public IReadOnlyList<string> OperationGroupsToOmit { get; }
         public IReadOnlyList<string> RequestPathIsNonResource { get; }
         public IReadOnlyList<string> NoPropertyTypeReplacement { get; }
@@ -221,12 +223,11 @@ private static Dictionary<TKey, TValue> DeserializeDictionary<TKey, TValue>(Json
         public IReadOnlyList<string> KeepPluralResourceData { get; }
         public IReadOnlyList<string> PrependRPPrefix { get; }
         public IReadOnlyDictionary<string, IReadOnlyDictionary<string, string>> OperationIdMappings { get; }
-        public IReadOnlyDictionary<string, string> UpdateRequiredCopy {get;}
+        public IReadOnlyDictionary<string, string> UpdateRequiredCopy { get; }
         public IReadOnlyDictionary<string, IReadOnlyDictionary<string, string>> PatchInitializerCustomization { get; }
 
         public IReadOnlyList<string> NoResourceSuffix { get; }
         public IReadOnlyList<string> GenerateArmResourceExtensions { get; }
-        public IReadOnlyList<string> PreventWrappingReturnType { get; }
 
         public bool IsArmCore { get; }
 
@@ -244,7 +245,7 @@ internal static MgmtConfiguration GetConfiguration(IPluginCommunication autoRest
                 noResourceSuffix: autoRest.GetValue<string[]?>("no-resource-suffix").GetAwaiter().GetResult() ?? Array.Empty<string>(),
                 schemasToPrependRPPrefix: autoRest.GetValue<string[]?>("prepend-rp-prefix").GetAwaiter().GetResult() ?? Array.Empty<string>(),
                 generateArmResourceExtensions: autoRest.GetValue<string[]?>("generate-arm-resource-extensions").GetAwaiter().GetResult() ?? Array.Empty<string>(),
-                preventWrappingReturnType: autoRest.GetValue<string[]?>("prevent-wrapping-return-type").GetAwaiter().GetResult() ?? Array.Empty<string>(),
+                parameterizedScopes: autoRest.GetValue<string[]?>("parameterized-scopes").GetAwaiter().GetResult() ?? Array.Empty<string>(),
                 mgmtDebug: MgmtDebugConfiguration.GetConfiguration(autoRest),
                 requestPathToParent: autoRest.GetValue<JsonElement?>("request-path-to-parent").GetAwaiter().GetResult(),
                 requestPathToResourceName: autoRest.GetValue<JsonElement?>("request-path-to-resource-name").GetAwaiter().GetResult(),
@@ -281,7 +282,6 @@ internal void SaveConfiguration(Utf8JsonWriter writer)
             WriteNonEmptySettings(writer, nameof(NoResourceSuffix), NoResourceSuffix);
             WriteNonEmptySettings(writer, nameof(PrependRPPrefix), PrependRPPrefix);
             WriteNonEmptySettings(writer, nameof(GenerateArmResourceExtensions), GenerateArmResourceExtensions);
-            WriteNonEmptySettings(writer, nameof(PreventWrappingReturnType), PreventWrappingReturnType);
             WriteNonEmptySettings(writer, nameof(OperationGroupsToOmit), OperationGroupsToOmit);
             WriteNonEmptySettings(writer, nameof(RequestPathToParent), RequestPathToParent);
             WriteNonEmptySettings(writer, nameof(OperationPositions), OperationPositions);
@@ -324,7 +324,6 @@ internal static MgmtConfiguration LoadConfiguration(JsonElement root)
             root.TryGetProperty(nameof(NoResourceSuffix), out var noResourceSuffixElement);
             root.TryGetProperty(nameof(PrependRPPrefix), out var prependRPPrefixElement);
             root.TryGetProperty(nameof(GenerateArmResourceExtensions), out var generateArmResourceExtensionsElement);
-            root.TryGetProperty(nameof(PreventWrappingReturnType), out var preventWrappingReturnTypeElement);
             root.TryGetProperty(nameof(RequestPathToParent), out var requestPathToParent);
             root.TryGetProperty(nameof(RequestPathToResourceName), out var requestPathToResourceName);
             root.TryGetProperty(nameof(RequestPathToResourceData), out var requestPathToResourceData);
@@ -341,6 +340,7 @@ internal static MgmtConfiguration LoadConfiguration(JsonElement root)
             root.TryGetProperty(nameof(OverrideOperationName), out var operationIdToName);
             root.TryGetProperty(nameof(MergeOperations), out var mergeOperations);
             root.TryGetProperty(nameof(PromptedEnumValues), out var promptedEnumValuesElement);
+            root.TryGetProperty(nameof(RawParameterizedScopes), out var parameterizedScopesElement);
 
             var operationGroupToOmit = Configuration.DeserializeArray(operationGroupsToOmitElement);
             var requestPathIsNonResource = Configuration.DeserializeArray(requestPathIsNonResourceElement);
@@ -353,7 +353,7 @@ internal static MgmtConfiguration LoadConfiguration(JsonElement root)
             var noResourceSuffix = Configuration.DeserializeArray(noResourceSuffixElement);
             var prependRPPrefix = Configuration.DeserializeArray(prependRPPrefixElement);
             var generateArmResourceExtensions = Configuration.DeserializeArray(generateArmResourceExtensionsElement);
-            var preventWrappingReturnType = Configuration.DeserializeArray(preventWrappingReturnTypeElement);
+            var parameterizedScopes = Configuration.DeserializeArray(parameterizedScopesElement);
 
             root.TryGetProperty("ArmCore", out var isArmCore);
             root.TryGetProperty(nameof(MgmtDebug), out var mgmtDebugRoot);
@@ -376,7 +376,7 @@ internal static MgmtConfiguration LoadConfiguration(JsonElement root)
                 noResourceSuffix: noResourceSuffix,
                 schemasToPrependRPPrefix: prependRPPrefix,
                 generateArmResourceExtensions: generateArmResourceExtensions,
-                preventWrappingReturnType: preventWrappingReturnType,
+                parameterizedScopes: parameterizedScopes,
                 mgmtDebug: MgmtDebugConfiguration.LoadConfiguration(mgmtDebugRoot),
                 requestPathToParent: requestPathToParent,
                 requestPathToResourceName: requestPathToResourceName,

src/AutoRest.CSharp/Mgmt/Decorator/ParameterMappingBuilder.cs

@@ -42,7 +42,7 @@ public static IEnumerable<ContextualParameterMapping> BuildContextualParameters(
         private static void BuildContextualParameterMappingHierarchy(RequestPath current, Stack<ContextualParameterMapping> parameterMappingStack, string idVariableName = "Id", string invocationSuffix = "")
         {
             // Check if the current path is a scope parameter
-            if (current.IsParameterizedScope())
+            if (current.IsRawParameterizedScope())
             {
                 // in this case, we should only have one segment in this current path
                 parameterMappingStack.Push(new ContextualParameterMapping(string.Empty, current.Last(), $"{idVariableName}{invocationSuffix}"));

src/AutoRest.CSharp/Mgmt/Decorator/ParentDetection.cs

@@ -38,6 +38,7 @@ public static IEnumerable<MgmtTypeProvider> Parent(this Resource resource)
 
         private static IEnumerable<MgmtTypeProvider> GetParent(this Resource resource)
         {
+            var scope = resource.RequestPath.GetScopePath();
             var resourceOperationSet = resource.OperationSet;
             var parentRequestPath = resourceOperationSet.ParentRequestPath(resource.ResourceType);
 
@@ -46,6 +47,15 @@ private static IEnumerable<MgmtTypeProvider> GetParent(this Resource resource)
                 // my parent is myself? Only tenant has this attribute, return empty
                 return Enumerable.Empty<MgmtTypeProvider>();
             }
+            // if the scope of this request path is parameterized, and the direct parent path we get from the resource list is parent of the scope, we return the scope as its parent since the scope here is a child
+            // if the request path is a "by id" path, its scope is the same as itself, therefore this condition here is nullified and should be skipped
+            if (!resource.RequestPath.IsById && scope.IsParameterizedScope() && (parentRequestPath.IsAncestorOf(scope) || parentRequestPath == scope))
+            {
+                // we already verified that the scope is parameterized, therefore we assert the type can never be null
+                var types = resource.RequestPath.GetParameterizedScopeResourceTypes()!;
+                return FindScopeParents(types).Distinct();
+            }
+
             if (MgmtContext.Library.TryGetArmResource(parentRequestPath, out var parent))
             {
                 return parent.AsIEnumerable();
@@ -58,8 +68,7 @@ private static IEnumerable<MgmtTypeProvider> GetParent(this Resource resource)
             if (parentRequestPath.Equals(RequestPath.Subscription))
                 return MgmtContext.Library.SubscriptionExtensions.AsIEnumerable();
             // the only option left is the tenant. But we have our last chance that its parent could be the scope of this
-            var scope = parentRequestPath.GetScopePath();
-            // if the scope of this request path is parameterized, we return the scope as its parent
+            scope = parentRequestPath.GetScopePath(); // we do this because some request path its scope is the same as itself
             if (scope.IsParameterizedScope())
             {
                 // we already verified that the scope is parameterized, therefore we assert the type can never be null
@@ -171,6 +180,7 @@ private static RequestPath GetParent(this RequestPath requestPath)
             var scope = requestPath.GetScopePath();
             var candidates = MgmtContext.Library.ResourceOperationSets.Select(operationSet => operationSet.GetRequestPath())
                 .Concat(new List<RequestPath> { RequestPath.ResourceGroup, RequestPath.Subscription, RequestPath.ManagementGroup }) // When generating management group in management.json, the path is /providers/Microsoft.Management/managementGroups/{groupId} while RequestPath.ManagementGroup is /providers/Microsoft.Management/managementGroups/{managementGroupId}. We pick the first one.
+                .Concat(Configuration.MgmtConfiguration.ParameterizedScopes)
                 .Where(r => r.IsAncestorOf(requestPath)).OrderByDescending(r => r.Count);
             if (candidates.Any())
             {