C#: Detect `TargetFramework`s and install them if there is no `global.json` #14821

mbg · 2023-11-16T16:24:08Z

Summary

For .NET projects, our builds may fail if a .NET version that is specified as a TargetFramework is not available on the host. We have seen this recently in cases where e.g. a .csproj specifies .NET 8 but .NET 8 was not available on a GHA runner yet by default.

This PR:

Retrieves TargetFramework values from the project files, where available.
If there's no global.json, we look at the TargetFramework values and install those SDKs. (Should we do this even if there is a global.json?)

tamasvajk · 2023-11-21T14:05:25Z

csharp/autobuilder/Semmle.Autobuild.CSharp/DotNetRule.cs

+                // TargetFramework values in the project files and install the latest versions
+                // from the corresponding channels


Why do we only install the latest? Why not all? If there are multiple projects with different target framework settings in the repo, some of them will fail the build, won't it?

We do install all! Maybe you had a look at the initial draft version of this PR, which only installed the latest, but now the review-ready version does install all the ones we find (in theory at least - pending changes we may have to make to DownloadDotNetVersion)

If there are multiple projects with different target framework settings in the repo, some of them will fail the build, won't it?

This is an interesting question. I would assume that to be the case yes, but if so that's been a pretty big oversight on our part with respect to C# projects generally.

Also relates to my question whether we should do this even if there is a global.json.

We could add an integration test to check this.

I think it's not possible to pin multiple versions of dotnet in global.json. So maybe whoever needs this would add a build script which takes care of the installation.

According to the docs for global.json, the version specified there just controls which version of the CLI is used and is independent of the SDK versions that the projects target.

tamasvajk · 2023-11-21T14:05:58Z

csharp/autobuilder/Semmle.Autobuild.Shared/Project.cs

@@ -51,7 +53,8 @@ public Project(Autobuilder<TAutobuildOptions> builder, string path) : base(build
            {
                if (root.HasAttribute("Sdk"))
                {
-                    DotNetProject = true;
+                    this.DotNetProject = true;
+                    this.TargetFramework = root.SelectSingleNode("//Project/PropertyGroup/TargetFramework/text()")?.Value;


We should also include the values from <TargetFrameworks> (plural).

tamasvajk · 2023-11-21T14:08:06Z

csharp/autobuilder/Semmle.Autobuild.CSharp/DotNetRule.cs

+                    .OfType<Project<CSharpAutobuildOptions>>()
+                    .Select(p => p.TargetFramework)
+                    .OfType<string>()
+                    .Where(v => v.StartsWith("net"));


I think we could be more specific about the versions. There's no point in issuing a web request for things that will obviously fail, such as netstandard.*, net48 and below, and maybe netcoreapp3.1 and below also doesn't work.

netstandard and netcoreapp should already get filtered out by the code further down (see the comment there). There was a unit test which initially failed for me (because it specified netcoreapp2.1 or something), but now passes since we no longer attempt to install those SDKs.

I can also filter out older net versions.

Maybe it's as simple as filtering out anything that has a major version number below 5.

tamasvajk · 2023-11-21T14:40:33Z

csharp/autobuilder/Semmle.Autobuild.CSharp/DotNetRule.cs

+                    Version? version = null;
+                    if (Version.TryParse(targetFrameworkVersion[3..], out version))
+                    {
+                        installScript &= DownloadDotNetVersion(builder, installDir, "latest", channel: version.ToString());


Does the download logic work for both net7.0 and net70?

I think the Version parsing here would interpret the latter as 70.0 essentially, so I doubt that the dotnet-install.sh script would be happy with that.

michaelnebel · 2023-11-21T15:28:15Z

csharp/autobuilder/Semmle.Autobuild.CSharp/DotNetRule.cs

-            // Download versions mentioned in `global.json` files
-            // See https://docs.microsoft.com/en-us/dotnet/core/tools/global-json
-            var installScript = BuildScript.Success;
-            var validGlobalJson = false;
            foreach (var path in builder.Paths.Select(p => p.Item1).Where(p => p.EndsWith("global.json", StringComparison.Ordinal)))
            {
                string version;


Need to declare version explicitly. You can just return directly within the try block.

try { var o = JObject.Parse(File.ReadAllText(path)); return (string)(o?["sdk"]?["version"]!); } catch // lgtm[cs/catch-of-all-exceptions] { // not a valid global.json file continue; }

michaelnebel · 2023-11-21T15:36:09Z

csharp/autobuilder/Semmle.Autobuild.CSharp/DotNetRule.cs

+                {
+                    // For simplicity, we only accept target frameworks of the form "netX.Y"
+                    Version? version = null;
+                    if (Version.TryParse(targetFrameworkVersion[3..], out version))


Consider using out var version and avoid declaring version above.
What will happen in case the target framework is net70 and not net7.0 (I think it will be parsed as version 70, which probably will cause problems downstream).

mbg added the C# label Nov 16, 2023

mbg self-assigned this Nov 16, 2023

mbg added 4 commits November 21, 2023 13:32

C#: Add channel to DownloadDotNetVersion

098812b

C#: Tolerate missing solution files in tests

adde95d

C#: Refactor code to read global.json version

373eb0a

C#: Load TargetFramework from project files

ebc4aaa

mbg force-pushed the mbg/csharp/detect-target-framework branch from 09fc91e to 9ab7c96 Compare November 21, 2023 13:32

mbg added 2 commits November 21, 2023 13:35

C#: Fall back to TargetFramework to install .NET

a126e1d

C#: Add test for TargetFramework install

2bce387

mbg changed the title ~~C#: Detect highest TargetFramework and install it if there is no global.json~~ C#: Detect TargetFrameworks and install them if there is no global.json Nov 21, 2023

mbg force-pushed the mbg/csharp/detect-target-framework branch from 9ab7c96 to 2bce387 Compare November 21, 2023 13:51

mbg marked this pull request as ready for review November 21, 2023 13:51

mbg requested a review from a team as a code owner November 21, 2023 13:51

tamasvajk reviewed Nov 21, 2023

View reviewed changes

michaelnebel reviewed Nov 21, 2023

View reviewed changes

C#: Detect `TargetFramework`s and install them if there is no `global.json` #14821

C#: Detect `TargetFramework`s and install them if there is no `global.json` #14821

mbg commented Nov 16, 2023 •

edited

tamasvajk Nov 21, 2023

mbg Nov 21, 2023

mbg Nov 21, 2023 •

edited

tamasvajk Nov 21, 2023

tamasvajk Nov 21, 2023 •

edited

mbg Nov 21, 2023

tamasvajk Nov 21, 2023

tamasvajk Nov 21, 2023

mbg Nov 21, 2023

tamasvajk Nov 21, 2023

tamasvajk Nov 21, 2023

mbg Nov 21, 2023 •

edited

michaelnebel Nov 21, 2023 •

edited

michaelnebel Nov 21, 2023

		// TargetFramework values in the project files and install the latest versions
		// from the corresponding channels

C#: Detect TargetFrameworks and install them if there is no global.json #14821

Are you sure you want to change the base?

C#: Detect TargetFrameworks and install them if there is no global.json #14821

Conversation

mbg commented Nov 16, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mbg Nov 21, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tamasvajk Nov 21, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mbg Nov 21, 2023 • edited

Choose a reason for hiding this comment

michaelnebel Nov 21, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

C#: Detect `TargetFramework`s and install them if there is no `global.json` #14821

C#: Detect `TargetFramework`s and install them if there is no `global.json` #14821

mbg commented Nov 16, 2023 •

edited

mbg Nov 21, 2023 •

edited

tamasvajk Nov 21, 2023 •

edited

mbg Nov 21, 2023 •

edited

michaelnebel Nov 21, 2023 •

edited