Skip to content

Conversation

@Numpsy
Copy link
Contributor

@Numpsy Numpsy commented Jan 9, 2025

Description

As #2818, but updated on top of the latest changes and with a couple of extra commits to try to get more tests passing - see further comments for details

let scenarioPath = resolvePath scenario ""
// dotnet tool install --version 5.19.0-alpha.local.1 fake-cli --add-source /e/Projects/FAKE/release/dotnetcore/

// Work around https://github.com/dotnet/sdk/issues/40655 by specifying the tool manifest explicitly
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of tests in previous runs seemed to be failing with dotnet/sdk#40655, whereby installing the dotnet tool in a subdirectory that contains a tools config file will ignore that file and use the one in the repo root, but then trying to use that tool will use the local config file and then fail.
That issue is marked as resolved, but it still seemed to be occuring in the SDK version used by the tests

|> Async.RunSynchronously
|> List.ofSeq
|> List.find (fun product -> product.ProductVersion.Equals("6.0"))
|> List.find (fun product -> product.ProductVersion.Equals("8.0"))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what the logic for this (or cachedDotnetSdkReleases.json) should be when updating .NET versions.
The current cachedDotnetSdkReleases.json only contains the .NET 6 releases and that seemed to cause test failures when built as .NET 8 and looking for the v8 SDK versions.
Changing this to 8.0 fixed that, but that might then not be right if fake-cli is multi-targetted at .NET 6 and .NET 8 as the .NET 6 version might still need the list of v6 releases?

@github-actions
Copy link
Contributor

github-actions bot commented Jan 9, 2025

Test Results

  4 files   -   8    4 suites   - 8   4m 43s ⏱️ - 28m 11s
419 tests  -  32  414 ✅  -  36  1 💤 ±0  4 ❌ +4 
419 runs   - 868  414 ✅  - 870  1 💤  - 2  4 ❌ +4 

For more details on these failures, see this check.

Results for commit ba8e1ed. ± Comparison against base commit bac1982.

This pull request removes 35 and adds 3 tests. Note that renamed tests count towards both.
[Fake.DotNet.FxCop.Tests; Analysis failure is handled]
[Fake.DotNet.FxCop.Tests; Test errors are read as expected]
[Fake.DotNet.FxCop.Tests; Test full command line is created]
[Fake.DotNet.FxCop.Tests; Test process is created with working directory]
[Fake.DotNet.FxCop.Tests; Test process is created]
[Fake.DotNet.FxCop.Tests; Test project file update can be enabled]
[Fake.DotNet.FxCop.Tests; Test that Xsl is defaulted]
[Fake.DotNet.FxCop.Tests; Test that all arguments are processed as expected]
[Fake.DotNet.FxCop.Tests; Test that console output can be switched off]
[Fake.DotNet.FxCop.Tests; Test that default arguments are processed as expected (containing spaces)]
…
[Fake.DotNet.sdkAssemblyResolverTests; Runner run script with 8.0.400 SDK version assemblies and resolve runtime version from cached file]
[Fake.DotNet.sdkAssemblyResolverTests; Runner run script with 8.0.400 SDK version assemblies]
[Fake.DotNet.sdkAssemblyResolverTests; Runner run script with 8.0.401 SDK version assemblies]

♻️ This comment has been updated with latest results.

@Numpsy
Copy link
Contributor Author

Numpsy commented Jan 9, 2025

The remaining failures are in the TemplateIntegrationTests, with errors like

25h[11:48:10 ERR] Fake.DotNet.Cli.IntegrationTests.TemplateTests.can install and run the template.fails to build a target that doesn't exist failed in 00:00:09.6230000. 
Expected the message 'Target Nonexistent is not defined' but got: No exact match of product releases 8.0.401 found.
No product release found for 8.0.401. Maybe a pre-release? Returning all the versions.
No exact match of product releases 8.0.404 found.
No product release found for 8.0.404. Maybe a pre-release? Returning all the versions.
There was a problem while setting up the environment:
-> Could not find referenced assemblies in path: '[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.36\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.35\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.33\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.32\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.31\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.30\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.29\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.28\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.27\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.26\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.25\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.24\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.23\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.22\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.21\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.20\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.19\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.18\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.16\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.15\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.14\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.13\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.12\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.11\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.10\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.9\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.8\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.7\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.6\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.5\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.4\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.3\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.2\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.1\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-rc.2.21480.5\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-rc.1.21451.13\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-preview.7.21377.19\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-preview.6.21352.12\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-preview.5.21301.5\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-preview.4.21253.7\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-preview.3.21201.4\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-preview.2.21154.6\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-preview.1.21102.12\ref\net6.0]', please check installed SDK and runtime versions

and I'm not sure if this is really a problem with the new stuff, or if it's because it's trying to use the 6.1.3 release to test the templates, and failing when running as .NET 8? (I suppose it shou;dn't be looking at the 6.0 assemblies when running as 8.0)

@Thorium
Copy link
Member

Thorium commented Jul 17, 2025

I expect that there is a chicken-and-egg problem, and the integration tests are actually restoring the .NET6 fake to try compiling .NET8, instead of referring to the just built .NET 8 version.

I tried this branch locally (Numpsy/new8-build-bump-roll-tests-2) on my machine, and my observations are here:

src\app\fake-cli\bin\Release\ has now net6.0 and net8.0 paths. Can a dotnet tool even have multiple target frameworks? Or should we just accept that FAKE 6.1.3 is working well for .NET6 projects and from now on everything else is .NET 8 only?

I run on my Windows machine dotnet build, and then from the actions:
dotnet tool install fake-cli --add-source "./src/app/fake-cli/bin/Debug" --add-source "./src/app/fake-cli/bin/Release" --version 1.0.0-1

It turned out that this didn't work properly. So I removed the Clean-task and called fake manually:

C:\git\FAKE>src\app\fake-cli\bin\Debug\net8.0\fake-cli -v build -e FAKE_DETAILED_ERRORS=true

And behold, all the integration tests went through, after painfully long half-an-hour later:

image

Where it did finally fail is:

dotnet.exe  deb --runtime linux-x64 --framework net8.0 --configuration Release --output "C:\git\FAKE\release\dotnetcore" --no-restore`

error NETSDK1045: The current .NET SDK does not support targeting .NET 8.0.  Either target .NET 6.0 or lower, or use a version of the .NET SDK that supports .NET 8.0. [C:\git\FAKE\src\app\fake-cl

So basically, dotnet deb tool has to be updated from 0.1.220 to version 0.1.232

"version": "0.1.220",

...and that's it, I think we could consider merging this and releasing a beta or new major version.

@xperiandri
Copy link
Collaborator

Can a dotnet tool even have multiple target frameworks?

@Thorium yes it can.
But there are some challenges with that.
You can see my changes in this PR to test that the older framework works fsprojects/FSharpLint#748

@Numpsy
Copy link
Contributor Author

Numpsy commented Jul 17, 2025

well for .NET6 projects and from now on everything else is .NET 8 only?

Whether there's any benefit to supporting .NET 6 in fake-cli any more is another question, and given the amount of pain in testing it all, just doing 8 seems simpler for a new major release?

.NET 10 is on the horizon, so making sure it works for that going forward seems more important than keeping 6 working in new versions. (My personal opinion anyway).

I'll see about rebasing it and fixing the conflicts anyway

@Thorium
Copy link
Member

Thorium commented Jul 17, 2025

Many projects use a separate non-FAKE fsproj as a build-project and reference the FAKE libraries, and there I see value in having .NET6 support kept.

But the fake-cli tool, I think, could target only the NET8 (or even 10 later).

@xperiandri
Copy link
Collaborator

xperiandri commented Jul 17, 2025

Then it will be impossible to run it unless you have the latest SDK installed

@Numpsy
Copy link
Contributor Author

Numpsy commented Jul 17, 2025

Then it will be impossible to run it unless you have the latest SDK installed

The 'DNX' stuff that MS are talking about with .NET 10 (https://github.com/dotnet/core/blob/main/release-notes/10.0/preview/preview6/sdk.md#platform-specific-net-tools) could maybe help if the tool could be self contained, but that's a work in progress so it remains to be seen

@Numpsy
Copy link
Contributor Author

Numpsy commented Jul 17, 2025

On the subject of targetting multiple .NET versions in fake-cli, re-reading the old comments reminded me of the situation with cachedDotnetSdkReleases.json - i.e., if the tool i smulti-targetted, you might need multiple versions of that file for different .NET versions?

@Numpsy Numpsy force-pushed the new8-build-bump-roll-tests-2 branch from 0364d81 to 233081b Compare July 17, 2025 17:31
@Numpsy
Copy link
Contributor Author

Numpsy commented Jul 17, 2025

@Thorium I didn't see your other PR until after i'd rebased the changes to fix the merge conflicts, which caused conflicts in that, so I've cherry picked your dotnet-deb update straight into this PR

@Numpsy Numpsy marked this pull request as ready for review July 17, 2025 19:07
@Thorium
Copy link
Member

Thorium commented Jul 18, 2025

I think this file https://github.com/fsprojects/FAKE/blob/master/src/template/fake-template/Content/.config/dotnet-tools.json
says "use the nuget version of FAKE in the integration test templates", which causes any major changes to FAKE (like .NET update) to break the integration tests, because the NuGet version doesn't have these changes yet.

But I'm not sure if there is any workaround possible.

@Thorium
Copy link
Member

Thorium commented Jul 18, 2025

One more thing: Because the system uses FSharp.Compiler.Service v43.8, the fsdocs-tool has to be upgraded from 16 to 20, to make GenerateDocs task pass.

Edit: Updated my PR, but you can also just do dotnet tool update fsdocs-tool instead.

After this, my local build does pass:
image

@xperiandri
Copy link
Collaborator

@arialdomartini

@Numpsy Numpsy force-pushed the new8-build-bump-roll-tests-2 branch from 62084b1 to 7b0784d Compare October 21, 2025 10:09
@Numpsy Numpsy force-pushed the new8-build-bump-roll-tests-2 branch from 7b0784d to 41a04b5 Compare October 25, 2025 21:42
nuget Microsoft.Build.Framework 17.3.2
nuget Microsoft.Build.Tasks.Core 17.3.2
nuget Microsoft.Build.Utilities.Core 17.3.2
nuget Microsoft.Build 17.11.4
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

17.11.4 is listed as suffering from GHSA-w3q9-fxm7-j8fq, so I think this should be updated to 17.11.48

@Numpsy
Copy link
Contributor Author

Numpsy commented Oct 26, 2025

General question about all the FAKE modules - should they all be both .NET 6 and .NET 8, or just NET 8? (or maybe only 6 if they don't have any dependencies where it matters?)

<OutputType>Exe</OutputType>
<ToolCommandName>fake</ToolCommandName>
<TargetFramework>net6.0</TargetFramework>
<TargetFrameworks>net8.0;net6.0</TargetFrameworks>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe fake-cli itself should only be 8 regardless of what the modules are? (the integration tests are 8 only as it stands, so if it is still built as 6 as well it moght not get any testing)

@Numpsy
Copy link
Contributor Author

Numpsy commented Oct 26, 2025

Otherwise the basic question is whether this should be turned into 8.0.0 alpha 1 and then the templates and tests fixed on top of that, or if it's worth trying something more minimal with just a .NET 8 / v8 build of fake-cli and then doing the modules afterwards.

Change global.json .NET 6 to 8
Search and replace targetframeworks from fsproj files to add net8.0
Add net8.0 to paket.dependencies
dotnet paket install to find .NET8 compatible dependencies
Expecto had to be hardcoded for now, because some tests are running on netstandard2.0 library (hopefully we can update this separately later)
MSBuild.StructuredLogger problem: DisableInternalBinLog = true had to be added to build.fsx NuGet commands (hopefully we can update this separately later)
A few places of code had new overrides so had to explicitly type to strings
SdkAssemblyResolver to default .NET 8 as well
Readme update
GitHub pipeline configs: Add .NET 8 install.
@Numpsy Numpsy force-pushed the new8-build-bump-roll-tests-2 branch from 41a04b5 to ba8e1ed Compare October 26, 2025 13:47
@Numpsy
Copy link
Contributor Author

Numpsy commented Oct 26, 2025

I expect that there is a chicken-and-egg problem, and the integration tests are actually restoring the .NET6 fake to try compiling .NET8, instead of referring to the just built .NET 8 version.

I tried this branch locally (Numpsy/new8-build-bump-roll-tests-2) on my machine, and my observations are here:

src\app\fake-cli\bin\Release\ has now net6.0 and net8.0 paths. Can a dotnet tool even have multiple target frameworks? Or should we just accept that FAKE 6.1.3 is working well for .NET6 projects and from now on everything else is .NET 8 only?

I run on my Windows machine dotnet build, and then from the actions: dotnet tool install fake-cli --add-source "./src/app/fake-cli/bin/Debug" --add-source "./src/app/fake-cli/bin/Release" --version 1.0.0-1

It turned out that this didn't work properly. So I removed the Clean-task and called fake manually:

C:\git\FAKE>src\app\fake-cli\bin\Debug\net8.0\fake-cli -v build -e FAKE_DETAILED_ERRORS=true

And behold, all the integration tests went through, after painfully long half-an-hour later:
image

Where it did finally fail is:

dotnet.exe  deb --runtime linux-x64 --framework net8.0 --configuration Release --output "C:\git\FAKE\release\dotnetcore" --no-restore`

error NETSDK1045: The current .NET SDK does not support targeting .NET 8.0.  Either target .NET 6.0 or lower, or use a version of the .NET SDK that supports .NET 8.0. [C:\git\FAKE\src\app\fake-cl

So basically, dotnet deb tool has to be updated from 0.1.220 to version 0.1.232

"version": "0.1.220",

...and that's it, I think we could consider merging this and releasing a beta or new major version.

It appears to work (all integration tests pass) if the .NET 6 runtime files are present in the local .NET install created by the tests, but not otherwise.

So

If i run the tests from this branch on a clean setup, if fails as the CI does.

If I run the build from the master branch, it installs the .NET 6 libs into %AppData%\Local\Microsoft\dotnet to run the .NET 6 tests, and leaves the files there.

If I then run the tests from this branch again, it installs the .NET 8 libs into %AppData%\Local\Microsoft\dotnet alongside the .NET 6 libs rather than replacing them, and the template integration tests find those and the tests pass.

Having .NET 6 installed in the default system location doesn't effect this as the tests are looking in the custom install location.

@SteveGilham
Copy link
Contributor

As net6.0 is past end of support, as is net7.0, after any necessary bridging release to get to net8.0, it would make sense to drop all mention of net6.0 - and work to prepare for net10.0 so we don't have an end of support crisis this time next year as both net8.0 and net9.0 are superseded on 10-Nov-26.

Let us hope that that process that time can be made as easy as "for a bridging release add mention of net10.0 at these points in the build system, the project targets and the code, then enable this bridging flag; for real net10.0 release, remove net8.0 and clear the flag. Lather, rinse repeat for net12.0, net14.0 ..."

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants