[Suggestion] Add the ability to compile all test apps in one job and run the tests in a following job (microsoft#2066)

### ❔What, Why & How

<!-- Include description of the changes that will help reviewers in
their task -->
We're trying to see if we can adjust our build graph such that we can
compile everything in the first build matrix and then run all test tests
in the following build matrix. However, currently it seems that AL-Go
won't run tests for test apps downloaded as dependencies (from another
job). This PR would add the ability to change that (as a preview
feature). This may be something we'd change later on though

Related to
[AB#613617](https://dynamicssmb2.visualstudio.com/1fcb79e7-ab07-432a-a3c6-6cf5a88ba4a5/_workitems/edit/613617)

### ✅ Checklist

- [ ] Add tests (E2E, unit tests)
- [x] Update RELEASENOTES.md
- [x] Update documentation (e.g. for new settings or scenarios)
- [ ] Add telemetry

<!-- Include more checklist entries, if needed -->

Author: aholstrup1

Actions/.Modules/ReadSettings.psm1

@@ -148,6 +148,7 @@ function GetDefaultSettings
         "installApps"                                   = @()
         "installTestApps"                               = @()
         "installOnlyReferencedApps"                     = $true
+        "runTestsInAllInstalledTestApps"                = $false
         "generateDependencyArtifact"                    = $false
         "skipUpgrade"                                   = $false
         "applicationDependency"                         = "18.0.0.0"

Actions/.Modules/settings.schema.json

@@ -241,6 +241,10 @@
             "type": "boolean",
             "description": "Install only referenced apps. See https://aka.ms/ALGoSettings#installonlyreferencedapps"
         },
+        "runTestsInAllInstalledTestApps": {
+            "type": "boolean",
+            "description": "PREVIEW: Run tests in all installed test apps. See https://aka.ms/ALGoSettings#runTestsInAllInstalledTestApps"
+        },
         "generateDependencyArtifact": {
             "type": "boolean",
             "description": "See https://aka.ms/ALGoSettings#generatedependencyartifact"

Actions/RunPipeline/RunPipeline.ps1

@@ -209,6 +209,11 @@ try {
         }
     }
 
+    if ($settings.runTestsInAllInstalledTestApps) {
+        # Trim parentheses from test apps. Run-ALPipeline will skip running tests in test apps wrapped in ()
+        $install.TestApps = $install.TestApps | ForEach-Object { $_.TrimStart("(").TrimEnd(")") }
+    }
+
     # Replace secret names in install.apps and install.testApps
     foreach($list in @('Apps','TestApps')) {
         $install."$list" = @($install."$list" | ForEach-Object {
@@ -237,6 +242,7 @@ try {
 
     # Analyze InstallApps and InstallTestApps before launching pipeline
 
+
     # Check if codeSignCertificateUrl+Password is used (and defined)
     if (!$settings.doNotSignApps -and $codeSignCertificateUrl -and $codeSignCertificatePassword -and !$settings.keyVaultCodesignCertificateName) {
         OutputWarning -message "Using the legacy CodeSignCertificateUrl and CodeSignCertificatePassword parameters. Consider using the new Azure Keyvault signing instead. Go to https://aka.ms/ALGoSettings#keyVaultCodesignCertificateName to find out more"

Scenarios/settings.md

@@ -103,6 +103,7 @@ The repository settings are only read from the repository settings file (.github
 | <a id="adminCenterApiCredentialsSecretName"></a>adminCenterApiCredentialsSecretName | Specifies the name (**NOT the secret**) of the adminCenterApiCredentials secret. Default is adminCenterApiCredentials. AL-Go for GitHub will look for a secret with this name in GitHub Secrets or Azure KeyVault to use when connecting to the Admin Center API when creating Online Development Environments. Read [this](CreateOnlineDevEnv2.md) for more information. | AdminCenterApiCredentials |
 | <a id="installApps"></a>installApps | An array of 3rd party dependency apps, which you do not have access to through the appDependencyProbingPaths. The setting should be an array of either secure URLs or paths to folders or files relative to the project, where the CI/CD workflow can find and download the apps. The apps in installApps are downloaded and installed before compiling and installing the apps.<br/>**Note:** If you specify ${{SECRETNAME}} as part of a URL, it will be replaced by the value of the secret SECRETNAME. | [ ] |
 | <a id="installTestApps"></a>installTestApps | An array of 3rd party dependency apps, which you do not have access to through the appDependencyProbingPaths. The setting should be an array of either secure URLs or paths to folders or files relative to the project, where the CI/CD workflow can find and download the apps. The apps in installTestApps are downloaded and installed before compiling and installing the test apps. Adding a parentheses around the setting indicates that the test in this app will NOT be run, only installed.<br/>**Note:** If you specify ${{SECRETNAME}} as part of a URL, it will be replaced by the value of the secret SECRETNAME. | [ ] |
+| <a id="runTestsInAllInstalledTestApps"></a>runTestsInAllInstalledTestApps | PREVIEW: By default AL-Go will publish test apps built in previous jobs within the workflow but will not execute the tests. By setting this to true, AL-Go will start running tests in all the test apps produced by previous jobs | false |
 | <a id="configPackages"></a>configPackages | An array of configuration packages to be applied to the build container before running tests. Configuration packages can be the relative path within the project or it can be STANDARD, EXTENDED or EVALUATION for the rapidstart packages, which comes with Business Central. | [ ] |
 | <a id="configPackages.country"></a>configPackages.country | An array of configuration packages to be applied to the build container for country **country** before running tests. Configuration packages can be the relative path within the project or it can be STANDARD, EXTENDED or EVALUATION for the rapidstart packages, which comes with Business Central. | [ ] |
 | <a id="installOnlyReferencedApps"></a>installOnlyReferencedApps | By default, only the apps referenced in the dependency chain of your apps will be installed when inspecting the settings: InstallApps, InstallTestApps and appDependencyProbingPath. If you change this setting to false, all apps found will be installed. | true |