C#: Re-create summary models and include source and sink models as well. by michaelnebel · Pull Request #9327 · github/codeql

michaelnebel · 2022-05-25T14:48:43Z

In this PR we

Include the generation of new sources and sinks (very limited outcome as we have only very few sources and sinks in CSV).
Recreate the summaries. The most recent commit of the summaries was not created based on the newest versions of the model generator. At least some of the removed summaries can be explained by the newly introduced ranking (ie. we don't follow up code path if a handwritten summary exist). This means that eg. summary for IEnumerable.GetEnumerator will cover many existing implementations and the source code will not be used.

github-actions · 2022-05-25T14:51:13Z

⚠️ The head of this PR and the base branch were compared for differences in the framework coverage reports. The generated reports are available in the artifacts of this workflow run. The differences will be picked up by the nightly job after the PR gets merged.

Click to show differences in coverage

csharp

Generated file changes for csharp

Changes to framework-coverage-csharp.rst:

-    System,"``System.*``, ``System``",3,12038,28,5
+    System,"``System.*``, ``System``",3,11796,32,7
-    Totals,,3,12599,359,5
+    Totals,,3,12357,363,7

Changes to framework-coverage-csharp.csv:

- System,28,3,12038,,4,,23,1,3,10096,1942
+ System,32,3,11796,,4,,25,3,3,9854,1942

github-actions · 2022-05-29T08:35:59Z

⚠️ The head of this PR and the base branch were compared for differences in the framework coverage reports. The generated reports are available in the artifacts of this workflow run. The differences will be picked up by the nightly job after the PR gets merged.

Click to show differences in coverage

csharp

Generated file changes for csharp

Changes to framework-coverage-csharp.rst:

-    System,"``System.*``, ``System``",3,12038,28,5
+    System,"``System.*``, ``System``",3,11796,32,7
-    Totals,,3,12599,359,5
+    Totals,,3,12357,363,7

Changes to framework-coverage-csharp.csv:

- System,28,3,12038,,4,,23,1,3,10096,1942
+ System,32,3,11796,,4,,25,3,3,9854,1942

michaelnebel · 2022-05-30T05:10:50Z

DCA looks fine. No changes in performance or alerts.

github-actions · 2022-06-22T11:06:04Z

⚠️ The head of this PR and the base branch were compared for differences in the framework coverage reports. The generated reports are available in the artifacts of this workflow run. The differences will be picked up by the nightly job after the PR gets merged.

Click to show differences in coverage

csharp

Generated file changes for csharp

Changes to framework-coverage-csharp.rst:

-    System,"``System.*``, ``System``",3,12038,28,5
+    System,"``System.*``, ``System``",3,11796,32,7
-    Totals,,3,12599,359,5
+    Totals,,3,12357,363,7

Changes to framework-coverage-csharp.csv:

- System,28,3,12038,,4,,23,1,3,10096,1942
+ System,32,3,11796,,4,,25,3,3,9854,1942

tamasvajk

There are quite a few unexpected summary deletions in this PR. Should these be investigated? Or should we just update these models, and then investigate and possibly improve the generator in a separate PR?

tamasvajk · 2022-06-24T07:48:35Z

-| System.Collections.Generic;KeyValuePair;false;Create<,>;(TKey,TValue);;Argument[0];ReturnValue;taint;generated |
-| System.Collections.Generic;KeyValuePair;false;Create<,>;(TKey,TValue);;Argument[1];ReturnValue;taint;generated |


These also seem to be correct flows. Why are these removed? Without checking, I expect the Create<,> method to be quite simple, so we should be able to identify flow in there.

It appears that these rows have disappered since we no longer use the source code for the KeyValuePair constructor as there are CSV rows it.
The rows look like

"System.Collections.Generic;KeyValuePair<,>;false;KeyValuePair;(TKey,TValue);;Argument[0];ReturnValue.Property[System.Collections.Generic.KeyValuePair<,>.Key];value;manual", "System.Collections.Generic;KeyValuePair<,>;false;KeyValuePair;(TKey,TValue);;Argument[1];ReturnValue.Property[System.Collections.Generic.KeyValuePair<,>.Value];value;manual"

That is, based on these rows the model generator no longer yields any results for

public static KeyValuePair<TKey, TValue> Create<TKey, TValue>(TKey key, TValue value) => new KeyValuePair<TKey, TValue>(key, value);

but a summary will be derived for

public static TKey Create2<TKey, TValue>(TKey key, TValue value) => new KeyValuePair<TKey, TValue>(key, value).Key;

tamasvajk · 2022-06-24T07:48:52Z

-| System.Collections.Generic;CollectionExtensions;false;TryAdd<,>;(System.Collections.Generic.IDictionary<TKey,TValue>,TKey,TValue);;Argument[1];Argument[0].Element;taint;generated |
-| System.Collections.Generic;CollectionExtensions;false;TryAdd<,>;(System.Collections.Generic.IDictionary<TKey,TValue>,TKey,TValue);;Argument[2];Argument[0].Element;taint;generated |


These two rows seem to be correct flow summaries.

The explanation is most likely the same as above. The IDictionary.Add method has a handwritten field specific version, which seems to be incompatible with the model generator.

tamasvajk · 2022-06-24T07:56:36Z

-        "System;TupleExtensions;false;ToTuple<,,,,,,,,,,,,,,,,,,,,>;(System.ValueTuple<T1,T2,T3,T4,T5,T6,T7,System.ValueTuple<T8,T9,T10,T11,T12,T13,T14,System.ValueTuple<T15,T16,T17,T18,T19,T20,T21>>>);;Argument[0];ReturnValue;taint;generated",
-        "System;TupleExtensions;false;ToTuple<,,,,,,,,,,,,,,,,,,,>;(System.ValueTuple<T1,T2,T3,T4,T5,T6,T7,System.ValueTuple<T8,T9,T10,T11,T12,T13,T14,System.ValueTuple<T15,T16,T17,T18,T19,T20>>>);;Argument[0];ReturnValue;taint;generated",
-        "System;TupleExtensions;false;ToTuple<,,,,,,,,,,,,,,,,,,>;(System.ValueTuple<T1,T2,T3,T4,T5,T6,T7,System.ValueTuple<T8,T9,T10,T11,T12,T13,T14,System.ValueTuple<T15,T16,T17,T18,T19>>>);;Argument[0];ReturnValue;taint;generated",
-        "System;TupleExtensions;false;ToTuple<,,,,,,,,,,,,,,,,,>;(System.ValueTuple<T1,T2,T3,T4,T5,T6,T7,System.ValueTuple<T8,T9,T10,T11,T12,T13,T14,System.ValueTuple<T15,T16,T17,T18>>>);;Argument[0];ReturnValue;taint;generated",
-        "System;TupleExtensions;false;ToTuple<,,,,,,,,,,,,,,,,>;(System.ValueTuple<T1,T2,T3,T4,T5,T6,T7,System.ValueTuple<T8,T9,T10,T11,T12,T13,T14,System.ValueTuple<T15,T16,T17>>>);;Argument[0];ReturnValue;taint;generated",
-        "System;TupleExtensions;false;ToTuple<,,,,,,,,,,,,,,,>;(System.ValueTuple<T1,T2,T3,T4,T5,T6,T7,System.ValueTuple<T8,T9,T10,T11,T12,T13,T14,System.ValueTuple<T15,T16>>>);;Argument[0];ReturnValue;taint;generated",
-        "System;TupleExtensions;false;ToTuple<,,,,,,,,,,,,,,>;(System.ValueTuple<T1,T2,T3,T4,T5,T6,T7,System.ValueTuple<T8,T9,T10,T11,T12,T13,T14,System.ValueTuple<T15>>>);;Argument[0];ReturnValue;taint;generated",
-        "System;TupleExtensions;false;ToTuple<,,,,,,,,,,,,,>;(System.ValueTuple<T1,T2,T3,T4,T5,T6,T7,System.ValueTuple<T8,T9,T10,T11,T12,T13,T14>>);;Argument[0];ReturnValue;taint;generated",
-        "System;TupleExtensions;false;ToTuple<,,,,,,,,,,,,>;(System.ValueTuple<T1,T2,T3,T4,T5,T6,T7,System.ValueTuple<T8,T9,T10,T11,T12,T13>>);;Argument[0];ReturnValue;taint;generated",
-        "System;TupleExtensions;false;ToTuple<,,,,,,,,,,,>;(System.ValueTuple<T1,T2,T3,T4,T5,T6,T7,System.ValueTuple<T8,T9,T10,T11,T12>>);;Argument[0];ReturnValue;taint;generated",
-        "System;TupleExtensions;false;ToTuple<,,,,,,,,,,>;(System.ValueTuple<T1,T2,T3,T4,T5,T6,T7,System.ValueTuple<T8,T9,T10,T11>>);;Argument[0];ReturnValue;taint;generated",
-        "System;TupleExtensions;false;ToTuple<,,,,,,,,,>;(System.ValueTuple<T1,T2,T3,T4,T5,T6,T7,System.ValueTuple<T8,T9,T10>>);;Argument[0];ReturnValue;taint;generated",
-        "System;TupleExtensions;false;ToTuple<,,,,,,,,>;(System.ValueTuple<T1,T2,T3,T4,T5,T6,T7,System.ValueTuple<T8,T9>>);;Argument[0];ReturnValue;taint;generated",
-        "System;TupleExtensions;false;ToTuple<,,,,,,,>;(System.ValueTuple<T1,T2,T3,T4,T5,T6,T7,System.ValueTuple<T8>>);;Argument[0];ReturnValue;taint;generated",
-        "System;TupleExtensions;false;ToTuple<,,,,,,>;(System.ValueTuple<T1,T2,T3,T4,T5,T6,T7>);;Argument[0];ReturnValue;taint;generated",
-        "System;TupleExtensions;false;ToTuple<,,,,,>;(System.ValueTuple<T1,T2,T3,T4,T5,T6>);;Argument[0];ReturnValue;taint;generated",
-        "System;TupleExtensions;false;ToTuple<,,,,>;(System.ValueTuple<T1,T2,T3,T4,T5>);;Argument[0];ReturnValue;taint;generated",
-        "System;TupleExtensions;false;ToTuple<,,,>;(System.ValueTuple<T1,T2,T3,T4>);;Argument[0];ReturnValue;taint;generated",
-        "System;TupleExtensions;false;ToTuple<,,>;(System.ValueTuple<T1,T2,T3>);;Argument[0];ReturnValue;taint;generated",
-        "System;TupleExtensions;false;ToTuple<,>;(System.ValueTuple<T1,T2>);;Argument[0];ReturnValue;taint;generated",
-        "System;TupleExtensions;false;ToTuple<>;(System.ValueTuple<T1>);;Argument[0];ReturnValue;taint;generated",


I also find these removals unexpected. Is it somehow an interplay of manual summaries that use value steps, and autogenerated flows that use taint?

I suspect the issue is the same as above. We have a Field specific summary for the method that is being called and this doesn't propagate the taint. The first example has been added to the Models as Data issue as something we can consider to investigate. Maybe we should do this after making the field based implementation of the summary generator.

michaelnebel · 2022-07-19T11:37:10Z

There are quite a few unexpected summary deletions in this PR. Should these be investigated? Or should we just update these models, and then investigate and possibly improve the generator in a separate PR?

I will try and investigate. It looks like all the removals are related to the heuristic of not using source code, when a summary exist. Will try and elaborate, if I can find any good answers.

michaelnebel · 2022-07-19T13:28:44Z

@tamasvajk : Thank you much for the review. There is identified at least one interesting aspect of the model generator in its current state, which we probably should look into, when the model generator has been made field based.
Thank you for looking into this with care. I think we should merge as is, as this reflects the current state of the generator with the precision that we have.

michaelnebel requested a review from a team as a code owner May 25, 2022 14:48

github-actions Bot added the C# label May 25, 2022

michaelnebel marked this pull request as draft May 25, 2022 14:56

michaelnebel force-pushed the csharp/dotnetruntimerefresh branch from 11da75b to 5e6f480 Compare May 29, 2022 08:33

michaelnebel marked this pull request as ready for review May 30, 2022 05:11

michaelnebel added the no-change-note-required This PR does not need a change note label May 30, 2022

michaelnebel marked this pull request as draft June 22, 2022 09:01

michaelnebel added 2 commits June 22, 2022 13:03

C#: Update .NET Runtime models and add sources and sinks.

0ef97b4

C#: Update tests.

8899bf7

michaelnebel force-pushed the csharp/dotnetruntimerefresh branch from 5e6f480 to 8899bf7 Compare June 22, 2022 11:03

michaelnebel marked this pull request as ready for review June 24, 2022 07:09

tamasvajk reviewed Jun 24, 2022

View reviewed changes

michaelnebel requested a review from tamasvajk July 19, 2022 13:28

hvitved approved these changes Aug 1, 2022

View reviewed changes

michaelnebel merged commit 02165e8 into github:main Aug 1, 2022

michaelnebel deleted the csharp/dotnetruntimerefresh branch August 1, 2022 14:15

		\| System.Collections.Generic;KeyValuePair;false;Create<,>;(TKey,TValue);;Argument[0];ReturnValue;taint;generated \|
		\| System.Collections.Generic;KeyValuePair;false;Create<,>;(TKey,TValue);;Argument[1];ReturnValue;taint;generated \|

		\| System.Collections.Generic;CollectionExtensions;false;TryAdd<,>;(System.Collections.Generic.IDictionary<TKey,TValue>,TKey,TValue);;Argument[1];Argument[0].Element;taint;generated \|
		\| System.Collections.Generic;CollectionExtensions;false;TryAdd<,>;(System.Collections.Generic.IDictionary<TKey,TValue>,TKey,TValue);;Argument[2];Argument[0].Element;taint;generated \|

Conversation

michaelnebel commented May 25, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented May 25, 2022

csharp

Generated file changes for csharp

Uh oh!

github-actions Bot commented May 29, 2022

csharp

Generated file changes for csharp

Uh oh!

michaelnebel commented May 30, 2022

Uh oh!

github-actions Bot commented Jun 22, 2022

csharp

Generated file changes for csharp

Uh oh!

tamasvajk left a comment

Choose a reason for hiding this comment

Uh oh!

tamasvajk Jun 24, 2022

Choose a reason for hiding this comment

Uh oh!

michaelnebel Jul 19, 2022

Choose a reason for hiding this comment

Uh oh!

tamasvajk Jun 24, 2022

Choose a reason for hiding this comment

Uh oh!

michaelnebel Jul 19, 2022

Choose a reason for hiding this comment

Uh oh!

tamasvajk Jun 24, 2022

Choose a reason for hiding this comment

Uh oh!

michaelnebel Jul 19, 2022

Choose a reason for hiding this comment

Uh oh!

michaelnebel commented Jul 19, 2022

Uh oh!

michaelnebel commented Jul 19, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

michaelnebel commented May 25, 2022 •

edited

Loading