Skip to content

C#: Narrow collection like types in model generation.#10158

Merged
michaelnebel merged 8 commits into
github:mainfrom
michaelnebel:csharp/narrowcollectiontypes
Aug 30, 2022
Merged

C#: Narrow collection like types in model generation.#10158
michaelnebel merged 8 commits into
github:mainfrom
michaelnebel:csharp/narrowcollectiontypes

Conversation

@michaelnebel

@michaelnebel michaelnebel commented Aug 24, 2022

Copy link
Copy Markdown
Contributor

The purpose of this PR is to

  • Align the relevant types part of the model generation, such that we no longer accept collection like types in read/store steps (except for bulk datatypes that can be used for string representation).

To see the impact of the change above, models were regenerated based on main and committed as a part of this PR, such that we can pinpoint the exact diff in model generation that this PR produces.

This revealed some interesting things

  • Only very few summaries will no longer be produced for .NET Runtime when excluding collection like types in model generation.
  • There appears to be an issue with the source model generation (the same has been observed for java). That is, source model generation based on "file" kind has been disabled for C#. This needs to be investigated (will add this as an item to the ongoing issue with model generation work)
  • Some of the tuple related summaries that has previously disappeared has re-appeared in the generated models. Doing a case analysis is way to time-consuming, but this hints that we need much better tooling around the model generation to observe which changes impacts the generated models instead of having to figure it out later.

Will try to run DCA to check if the exclusion of collection like types narrows the search space in way that is measurable in terms of performance.

@github-actions github-actions Bot added the C# label Aug 24, 2022
@michaelnebel michaelnebel force-pushed the csharp/narrowcollectiontypes branch 2 times, most recently from 0ef47b4 to b17d9e3 Compare August 25, 2022 09:42
@github-actions

Copy link
Copy Markdown
Contributor

⚠️ The head of this PR and the base branch were compared for differences in the framework coverage reports. The generated reports are available in the artifacts of this workflow run. The differences will be picked up by the nightly job after the PR gets merged.

Click to show differences in coverage

csharp

Generated file changes for csharp

  • Changes to framework-coverage-csharp.rst:
-    System,"``System.*``, ``System``",4,11809,43,7
+    System,"``System.*``, ``System``",695,12081,65,7
-    Others,"``Dapper``, ``JsonToItemsTaskFactory``, ``Microsoft.ApplicationBlocks.Data``, ``Microsoft.CSharp``, ``Microsoft.EntityFrameworkCore``, ``Microsoft.Extensions.Caching.Distributed``, ``Microsoft.Extensions.Caching.Memory``, ``Microsoft.Extensions.Configuration``, ``Microsoft.Extensions.DependencyInjection``, ``Microsoft.Extensions.DependencyModel``, ``Microsoft.Extensions.FileProviders``, ``Microsoft.Extensions.FileSystemGlobbing``, ``Microsoft.Extensions.Hosting``, ``Microsoft.Extensions.Http``, ``Microsoft.Extensions.Logging``, ``Microsoft.Extensions.Options``, ``Microsoft.Extensions.Primitives``, ``Microsoft.Interop``, ``Microsoft.NET.Build.Tasks``, ``Microsoft.NETCore.Platforms.BuildTasks``, ``Microsoft.VisualBasic``, ``Microsoft.Win32``, ``MySql.Data.MySqlClient``, ``Newtonsoft.Json``, ``Windows.Security.Cryptography.Core``",,554,138,
+    Others,"``Dapper``, ``JsonToItemsTaskFactory``, ``Microsoft.ApplicationBlocks.Data``, ``Microsoft.CSharp``, ``Microsoft.EntityFrameworkCore``, ``Microsoft.Extensions.Caching.Distributed``, ``Microsoft.Extensions.Caching.Memory``, ``Microsoft.Extensions.Configuration``, ``Microsoft.Extensions.DependencyInjection``, ``Microsoft.Extensions.DependencyModel``, ``Microsoft.Extensions.FileProviders``, ``Microsoft.Extensions.FileSystemGlobbing``, ``Microsoft.Extensions.Hosting``, ``Microsoft.Extensions.Http``, ``Microsoft.Extensions.Logging``, ``Microsoft.Extensions.Options``, ``Microsoft.Extensions.Primitives``, ``Microsoft.Interop``, ``Microsoft.NET.Build.Tasks``, ``Microsoft.NETCore.Platforms.BuildTasks``, ``Microsoft.VisualBasic``, ``Microsoft.Win32``, ``MySql.Data.MySqlClient``, ``Newtonsoft.Json``, ``Windows.Security.Cryptography.Core``",14,555,138,
-    Totals,,4,12370,375,7
+    Totals,,709,12643,397,7
  • Changes to framework-coverage-csharp.csv:
- Microsoft.Extensions.Configuration,,,83,,,,,,,,,,,,80,3
+ Microsoft.Extensions.Configuration,,4,83,,,,,,,,,,4,,80,3
- Microsoft.Extensions.FileProviders,,,15,,,,,,,,,,,,15,
+ Microsoft.Extensions.FileProviders,,1,16,,,,,,,,,,1,,16,
- Microsoft.Extensions.Primitives,,,63,,,,,,,,,,,,63,
+ Microsoft.Extensions.Primitives,,2,63,,,,,,,,,,2,,63,
- Microsoft.NET.Build.Tasks,,,1,,,,,,,,,,,,1,
+ Microsoft.NET.Build.Tasks,,4,1,,,,,,,,,,4,,1,
- Microsoft.NETCore.Platforms.BuildTasks,,,4,,,,,,,,,,,,4,
+ Microsoft.NETCore.Platforms.BuildTasks,,3,4,,,,,,,,,,3,,4,
- System,43,4,11809,,1,1,1,,4,,33,3,1,3,9867,1942
+ System,65,695,12081,,8,8,9,,4,,33,3,692,3,10139,1942

@github-actions github-actions Bot added the Java label Aug 25, 2022
@github-actions

Copy link
Copy Markdown
Contributor

⚠️ The head of this PR and the base branch were compared for differences in the framework coverage reports. The generated reports are available in the artifacts of this workflow run. The differences will be picked up by the nightly job after the PR gets merged.

Click to show differences in coverage

csharp

Generated file changes for csharp

  • Changes to framework-coverage-csharp.rst:
-    System,"``System.*``, ``System``",4,11809,43,7
+    System,"``System.*``, ``System``",4,12081,65,7
-    Others,"``Dapper``, ``JsonToItemsTaskFactory``, ``Microsoft.ApplicationBlocks.Data``, ``Microsoft.CSharp``, ``Microsoft.EntityFrameworkCore``, ``Microsoft.Extensions.Caching.Distributed``, ``Microsoft.Extensions.Caching.Memory``, ``Microsoft.Extensions.Configuration``, ``Microsoft.Extensions.DependencyInjection``, ``Microsoft.Extensions.DependencyModel``, ``Microsoft.Extensions.FileProviders``, ``Microsoft.Extensions.FileSystemGlobbing``, ``Microsoft.Extensions.Hosting``, ``Microsoft.Extensions.Http``, ``Microsoft.Extensions.Logging``, ``Microsoft.Extensions.Options``, ``Microsoft.Extensions.Primitives``, ``Microsoft.Interop``, ``Microsoft.NET.Build.Tasks``, ``Microsoft.NETCore.Platforms.BuildTasks``, ``Microsoft.VisualBasic``, ``Microsoft.Win32``, ``MySql.Data.MySqlClient``, ``Newtonsoft.Json``, ``Windows.Security.Cryptography.Core``",,554,138,
+    Others,"``Dapper``, ``JsonToItemsTaskFactory``, ``Microsoft.ApplicationBlocks.Data``, ``Microsoft.CSharp``, ``Microsoft.EntityFrameworkCore``, ``Microsoft.Extensions.Caching.Distributed``, ``Microsoft.Extensions.Caching.Memory``, ``Microsoft.Extensions.Configuration``, ``Microsoft.Extensions.DependencyInjection``, ``Microsoft.Extensions.DependencyModel``, ``Microsoft.Extensions.FileProviders``, ``Microsoft.Extensions.FileSystemGlobbing``, ``Microsoft.Extensions.Hosting``, ``Microsoft.Extensions.Http``, ``Microsoft.Extensions.Logging``, ``Microsoft.Extensions.Options``, ``Microsoft.Extensions.Primitives``, ``Microsoft.Interop``, ``Microsoft.NET.Build.Tasks``, ``Microsoft.NETCore.Platforms.BuildTasks``, ``Microsoft.VisualBasic``, ``Microsoft.Win32``, ``MySql.Data.MySqlClient``, ``Newtonsoft.Json``, ``Windows.Security.Cryptography.Core``",,555,138,
-    Totals,,4,12370,375,7
+    Totals,,4,12643,397,7
  • Changes to framework-coverage-csharp.csv:
- Microsoft.Extensions.FileProviders,,,15,,,,,,,,,,,,15,
+ Microsoft.Extensions.FileProviders,,,16,,,,,,,,,,,,16,
- System,43,4,11809,,1,1,1,,4,,33,3,1,3,9867,1942
+ System,65,4,12081,,8,8,9,,4,,33,3,1,3,10139,1942

@michaelnebel michaelnebel force-pushed the csharp/narrowcollectiontypes branch from 50ed1ea to f4835e3 Compare August 26, 2022 07:30
@github-actions

Copy link
Copy Markdown
Contributor

⚠️ The head of this PR and the base branch were compared for differences in the framework coverage reports. The generated reports are available in the artifacts of this workflow run. The differences will be picked up by the nightly job after the PR gets merged.

Click to show differences in coverage

csharp

Generated file changes for csharp

  • Changes to framework-coverage-csharp.rst:
-    System,"``System.*``, ``System``",4,11809,43,7
+    System,"``System.*``, ``System``",4,12081,65,7
-    Others,"``Dapper``, ``JsonToItemsTaskFactory``, ``Microsoft.ApplicationBlocks.Data``, ``Microsoft.CSharp``, ``Microsoft.EntityFrameworkCore``, ``Microsoft.Extensions.Caching.Distributed``, ``Microsoft.Extensions.Caching.Memory``, ``Microsoft.Extensions.Configuration``, ``Microsoft.Extensions.DependencyInjection``, ``Microsoft.Extensions.DependencyModel``, ``Microsoft.Extensions.FileProviders``, ``Microsoft.Extensions.FileSystemGlobbing``, ``Microsoft.Extensions.Hosting``, ``Microsoft.Extensions.Http``, ``Microsoft.Extensions.Logging``, ``Microsoft.Extensions.Options``, ``Microsoft.Extensions.Primitives``, ``Microsoft.Interop``, ``Microsoft.NET.Build.Tasks``, ``Microsoft.NETCore.Platforms.BuildTasks``, ``Microsoft.VisualBasic``, ``Microsoft.Win32``, ``MySql.Data.MySqlClient``, ``Newtonsoft.Json``, ``Windows.Security.Cryptography.Core``",,554,138,
+    Others,"``Dapper``, ``JsonToItemsTaskFactory``, ``Microsoft.ApplicationBlocks.Data``, ``Microsoft.CSharp``, ``Microsoft.EntityFrameworkCore``, ``Microsoft.Extensions.Caching.Distributed``, ``Microsoft.Extensions.Caching.Memory``, ``Microsoft.Extensions.Configuration``, ``Microsoft.Extensions.DependencyInjection``, ``Microsoft.Extensions.DependencyModel``, ``Microsoft.Extensions.FileProviders``, ``Microsoft.Extensions.FileSystemGlobbing``, ``Microsoft.Extensions.Hosting``, ``Microsoft.Extensions.Http``, ``Microsoft.Extensions.Logging``, ``Microsoft.Extensions.Options``, ``Microsoft.Extensions.Primitives``, ``Microsoft.Interop``, ``Microsoft.NET.Build.Tasks``, ``Microsoft.NETCore.Platforms.BuildTasks``, ``Microsoft.VisualBasic``, ``Microsoft.Win32``, ``MySql.Data.MySqlClient``, ``Newtonsoft.Json``, ``Windows.Security.Cryptography.Core``",,555,138,
-    Totals,,4,12370,375,7
+    Totals,,4,12643,397,7
  • Changes to framework-coverage-csharp.csv:
- Microsoft.Extensions.FileProviders,,,15,,,,,,,,,,,,15,
+ Microsoft.Extensions.FileProviders,,,16,,,,,,,,,,,,16,
- System,43,4,11809,,1,1,1,,4,,33,3,1,3,9867,1942
+ System,65,4,12081,,8,8,9,,4,,33,3,1,3,10139,1942

@michaelnebel michaelnebel marked this pull request as ready for review August 26, 2022 11:23
@michaelnebel michaelnebel requested review from a team as code owners August 26, 2022 11:23
@michaelnebel michaelnebel added the no-change-note-required This PR does not need a change note label Aug 26, 2022
@michaelnebel

Copy link
Copy Markdown
Contributor Author

The purpose of this PR is to

  • Align the relevant types part of the model generation, such that we no longer accept collection like types in read/store steps (except for bulk datatypes that can be used for string representation).

To see the impact of the change above, models were regenerated based on main and committed as a part of this PR, such that we can pinpoint the exact diff in model generation that this PR produces.

This revealed some interesting things

  • Only very few summaries will no longer be produced for .NET Runtime when excluding collection like types in model generation.
  • There appears to be an issue with the source model generation (the same has been observed for java). That is, source model generation based on "file" kind has been disabled for C#. This needs to be investigated (will add this as an item to the ongoing issue with model generation work)
  • Some of the tuple related summaries that has previously disappeared has re-appeared in the generated models. Doing a case analysis is way to time-consuming, but this hints that we need much better tooling around the model generation to observe which changes impacts the generated models instead of having to figure it out later.

Will try to run DCA to check if the exclusion of collection like types narrows the search space in way that is measurable in terms of performance.

DCA did not show any improvements or regressions (but shows that there is a bunch of source models that will no longer be produced (as expected)).
In any case, we should consider to merge this change to be aligned with the Java implementation.

@michaelnebel michaelnebel merged commit e020ae7 into github:main Aug 30, 2022
@michaelnebel michaelnebel deleted the csharp/narrowcollectiontypes branch August 30, 2022 12:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

C# Java no-change-note-required This PR does not need a change note

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants