Java: model top JDK APIs by jcogs33 · Pull Request #11779 · github/codeql

jcogs33 · 2022-12-21T21:24:22Z

Description:

This PR adds missing MaD models for the next 100 top JDK APIs. (The first 100 were reviewed and modeled in #11572).

Consideration:

Please review the added models in detail, especially the following:

Why does java.lang.String#endsWith have an existing summary model instead of a neutral model? It returns a boolean, so I thought it should have a neutral model. Should I leave this one as a summary model or switch it to a neutral model?
Does the model for java.lang.System#getProperty(String) look correct? I couldn't seem to get a working test case for this one.
Should any of the following be summary models instead of neutrals?
- java.lang.Class#getClassLoader()
- java.text.DateFormat#format(Date)
- java.text.SimpleDateFormat#SimpleDateFormat(String)
- java.time.LocalDate#of(int,int,int)
- java.util.function.Consumer#accept(Object)
- java.util.Date#Date(long)
- java.util.stream.Collectors#joining(CharSequence)
- java.util.stream.Collectors#toMap(Function,Function)

egregius313

LGTM. I can approve once we get an answer about the endsWith row.

github-actions · 2023-01-12T02:51:35Z

⚠️ The head of this PR and the base branch were compared for differences in the framework coverage reports. The generated reports are available in the artifacts of this workflow run. The differences will be picked up by the nightly job after the PR gets merged.

Click to show differences in coverage

java

Generated file changes for java

Changes to framework-coverage-java.rst:

-    Java Standard Library,``java.*``,3,602,130,28,,,7,,,10
+    Java Standard Library,``java.*``,3,615,130,28,,,7,,,10
-    Totals,,217,8449,1563,129,6,10,107,33,1,86
+    Totals,,217,8462,1563,129,6,10,107,33,1,86

Changes to framework-coverage-java.csv:

- java.io,37,,40,,15,,,,,,,,,,,,,,,,,,,,,,,,,,,22,,,,,,,,40,
+ java.io,37,,42,,15,,,,,,,,,,,,,,,,,,,,,,,,,,,22,,,,,,,,41,1
- java.lang,13,,75,,,,,,,,,,,,8,,,,,,4,,,1,,,,,,,,,,,,,,,,56,19
+ java.lang,13,,80,,,,,,,,,,,,8,,,,,,4,,,1,,,,,,,,,,,,,,,,53,27
- java.math,,,1,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,1,
- java.sql,11,,1,,,,,,,,4,,,,,,,,,,,,,,,,,7,,,,,,,,,,,,1,
+ java.sql,11,,2,,,,,,,,4,,,,,,,,,,,,,,,,,7,,,,,,,,,,,,1,1
- java.util,44,,461,,,,,,,,,,,,34,,,,,,,5,2,,1,2,,,,,,,,,,,,,,36,425
+ java.util,44,,467,,,,,,,,,,,,34,,,,,,,5,2,,1,2,,,,,,,,,,,,,,39,428

…y models

jcogs33 · 2023-01-12T03:44:30Z

Thanks all! I've switched the numeric flow models to neutrals in 6bb865a. Let me know if you want any changes to how I stored/commented them.

We'll then probably need to do the same for numeric flow models added in the previous PR, for consistency.

I updated all of the numeric flow models from the previous PR as well (java.lang.Integer#parseInt(String), java.lang.String#valueOf(int), java.math.BigDecimal#BigDecimal(String)).

java.util.function.Consumer#accept(Object)
This is related to specialized lambda flow, so I don't think it should be handled by MaD.

@atorralba Should java.util.function.Function#apply(Object) not be handled by MaD either? I had added a model for it but am now questioning whether it should be added since it is also related to lambda flow.

atorralba · 2023-01-12T08:52:32Z

@atorralba Should java.util.function.Function#apply(Object) not be handled by MaD either? I had added a model for it but am now questioning whether it should be added since it is also related to lambda flow.

Yes, without knowledge of what the function does, assuming that flow is propagated from arg0 to return value is too "greedy" and prone to FP, I think. Doing that with precision requires analyzing the code of the function itself, and building the dataflow graph accordingly (which, if I'm not mistaken, the existing lambda flow support already does, at least in some cases).

github-actions · 2023-01-12T21:37:24Z

⚠️ The head of this PR and the base branch were compared for differences in the framework coverage reports. The generated reports are available in the artifacts of this workflow run. The differences will be picked up by the nightly job after the PR gets merged.

Click to show differences in coverage

java

Generated file changes for java

Changes to framework-coverage-java.rst:

-    Java Standard Library,``java.*``,3,602,130,28,,,7,,,10
+    Java Standard Library,``java.*``,3,614,130,28,,,7,,,10
-    Totals,,217,8449,1563,129,6,10,107,33,1,86
+    Totals,,217,8461,1563,129,6,10,107,33,1,86

Changes to framework-coverage-java.csv:

- java.io,37,,40,,15,,,,,,,,,,,,,,,,,,,,,,,,,,,22,,,,,,,,40,
+ java.io,37,,42,,15,,,,,,,,,,,,,,,,,,,,,,,,,,,22,,,,,,,,41,1
- java.lang,13,,75,,,,,,,,,,,,8,,,,,,4,,,1,,,,,,,,,,,,,,,,56,19
+ java.lang,13,,80,,,,,,,,,,,,8,,,,,,4,,,1,,,,,,,,,,,,,,,,53,27
- java.math,,,1,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,1,
- java.sql,11,,1,,,,,,,,4,,,,,,,,,,,,,,,,,7,,,,,,,,,,,,1,
+ java.sql,11,,2,,,,,,,,4,,,,,,,,,,,,,,,,,7,,,,,,,,,,,,1,1
- java.util,44,,461,,,,,,,,,,,,34,,,,,,,5,2,,1,2,,,,,,,,,,,,,,36,425
+ java.util,44,,466,,,,,,,,,,,,34,,,,,,,5,2,,1,2,,,,,,,,,,,,,,38,428

github-actions · 2023-01-12T22:17:51Z

⚠️ The head of this PR and the base branch were compared for differences in the framework coverage reports. The generated reports are available in the artifacts of this workflow run. The differences will be picked up by the nightly job after the PR gets merged.

Click to show differences in coverage

java

Generated file changes for java

Changes to framework-coverage-java.rst:

-    Java Standard Library,``java.*``,3,602,130,28,,,7,,,10
+    Java Standard Library,``java.*``,3,613,130,28,,,7,,,10
-    Totals,,217,8449,1563,129,6,10,107,33,1,86
+    Totals,,217,8460,1563,129,6,10,107,33,1,86

Changes to framework-coverage-java.csv:

- java.io,37,,40,,15,,,,,,,,,,,,,,,,,,,,,,,,,,,22,,,,,,,,40,
+ java.io,37,,42,,15,,,,,,,,,,,,,,,,,,,,,,,,,,,22,,,,,,,,41,1
- java.lang,13,,75,,,,,,,,,,,,8,,,,,,4,,,1,,,,,,,,,,,,,,,,56,19
+ java.lang,13,,80,,,,,,,,,,,,8,,,,,,4,,,1,,,,,,,,,,,,,,,,53,27
- java.math,,,1,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,1,
- java.sql,11,,1,,,,,,,,4,,,,,,,,,,,,,,,,,7,,,,,,,,,,,,1,
+ java.sql,11,,2,,,,,,,,4,,,,,,,,,,,,,,,,,7,,,,,,,,,,,,1,1
- java.util,44,,461,,,,,,,,,,,,34,,,,,,,5,2,,1,2,,,,,,,,,,,,,,36,425
+ java.util,44,,465,,,,,,,,,,,,34,,,,,,,5,2,,1,2,,,,,,,,,,,,,,38,427

michaelnebel

Looks good to me!
Minor: Added a comment about List.clear and Map.clear.

github-actions · 2023-01-13T18:26:23Z

⚠️ The head of this PR and the base branch were compared for differences in the framework coverage reports. The generated reports are available in the artifacts of this workflow run. The differences will be picked up by the nightly job after the PR gets merged.

Click to show differences in coverage

java

Generated file changes for java

Changes to framework-coverage-java.rst:

-    Java Standard Library,``java.*``,3,602,130,28,,,7,,,10
+    Java Standard Library,``java.*``,3,613,130,28,,,7,,,10
-    Totals,,217,8449,1563,129,6,10,107,33,1,86
+    Totals,,217,8460,1563,129,6,10,107,33,1,86

Changes to framework-coverage-java.csv:

- java.io,37,,40,,15,,,,,,,,,,,,,,,,,,,,,,,,,,,22,,,,,,,,40,
+ java.io,37,,42,,15,,,,,,,,,,,,,,,,,,,,,,,,,,,22,,,,,,,,41,1
- java.lang,13,,75,,,,,,,,,,,,8,,,,,,4,,,1,,,,,,,,,,,,,,,,56,19
+ java.lang,13,,80,,,,,,,,,,,,8,,,,,,4,,,1,,,,,,,,,,,,,,,,53,27
- java.math,,,1,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,1,
- java.sql,11,,1,,,,,,,,4,,,,,,,,,,,,,,,,,7,,,,,,,,,,,,1,
+ java.sql,11,,2,,,,,,,,4,,,,,,,,,,,,,,,,,7,,,,,,,,,,,,1,1
- java.util,44,,461,,,,,,,,,,,,34,,,,,,,5,2,,1,2,,,,,,,,,,,,,,36,425
+ java.util,44,,465,,,,,,,,,,,,34,,,,,,,5,2,,1,2,,,,,,,,,,,,,,38,427

michaelnebel

LGTM!

github-actions · 2023-01-17T13:56:24Z

⚠️ The head of this PR and the base branch were compared for differences in the framework coverage reports. The generated reports are available in the artifacts of this workflow run. The differences will be picked up by the nightly job after the PR gets merged.

Click to show differences in coverage

java

Generated file changes for java

Changes to framework-coverage-java.rst:

-    Java Standard Library,``java.*``,3,602,130,28,,,7,,,10
+    Java Standard Library,``java.*``,3,609,130,28,,,7,,,10
-    Totals,,217,8449,1563,129,6,10,107,33,1,86
+    Totals,,217,8456,1563,129,6,10,107,33,1,86

Changes to framework-coverage-java.csv:

- java.io,37,,40,,15,,,,,,,,,,,,,,,,,,,,,,,,,,,22,,,,,,,,40,
+ java.io,37,,42,,15,,,,,,,,,,,,,,,,,,,,,,,,,,,22,,,,,,,,41,1
- java.lang,13,,75,,,,,,,,,,,,8,,,,,,4,,,1,,,,,,,,,,,,,,,,56,19
+ java.lang,13,,76,,,,,,,,,,,,8,,,,,,4,,,1,,,,,,,,,,,,,,,,53,23
- java.math,,,1,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,1,
- java.sql,11,,1,,,,,,,,4,,,,,,,,,,,,,,,,,7,,,,,,,,,,,,1,
+ java.sql,11,,2,,,,,,,,4,,,,,,,,,,,,,,,,,7,,,,,,,,,,,,1,1
- java.util,44,,461,,,,,,,,,,,,34,,,,,,,5,2,,1,2,,,,,,,,,,,,,,36,425
+ java.util,44,,465,,,,,,,,,,,,34,,,,,,,5,2,,1,2,,,,,,,,,,,,,,38,427

atorralba

Thanks for all the effort! 🙏

Jami Cogswell added 13 commits December 21, 2022 12:34

Java: add java.io models

99ddd48

Java: add java.lang models

ed534b0

Java: add java.math models

1544f49

Java: add java.sql models

b9ce588

Java: add java.text models

8e20aeb

Java: add java.time models

cfe075e

Java: add java.util.concurrent.atomic models

db0d24f

Java: add java.util.concurrent models

a8c55ee

Java: add java.util.function models

573de92

Java: add java.util models

1db829e

Java: add java.util.stream models

16de30e

Java: update TopJdkApis test

c251da7

Java: add initial test cases for summary models

de59655

github-actions Bot added the Java label Dec 21, 2022

Jami Cogswell added 11 commits December 22, 2022 12:29

Java: update test cases

6007827

Java: update test case affected by Class.isAssignableFrom neutral model

997219a

Java: update test case affected by Long.parseLong summary model

e6331dc

Java: updates to order alphabetically

a81c54b

Java: update Math.min test case

673d37c

Java: add comments

939279a

Java: add summary model and test for File.getName

21a018e

Java: add summary model for System.getProperty, adjust comments

29221ae

Java: adjust comments

feaae16

Java: update test case affected by Function.apply model

5d92792

Java: add change note

abe501c

github-actions Bot added the documentation label Jan 3, 2023

jcogs33 marked this pull request as ready for review January 3, 2023 22:23

jcogs33 requested a review from a team as a code owner January 3, 2023 22:23

egregius313 reviewed Jan 6, 2023

View reviewed changes

Comment thread java/ql/lib/ext/java.lang.model.yml Outdated

egregius313 reviewed Jan 6, 2023

View reviewed changes

Jami Cogswell added 2 commits January 11, 2023 12:04

Java: update System.getProperty model

0c7ffb0

Java: make numeric flow models neutral

6bb865a

Jami Cogswell added 2 commits January 11, 2023 22:15

Java: Date models as neutral

ce74c9d

Java: undo changes to tests that were affected by numeric-flow summar…

fd593fd

…y models

Jami Cogswell added 4 commits January 12, 2023 09:58

Java: remove Function.apply model

e044444

Java: add endsWith additionalTaintStep to ConditionalBypassFlowConfig

ffb2679

Java: remove endsWith test case

a39b2aa

Java: update change note

c3a1d08

Jami Cogswell added 2 commits January 12, 2023 17:01

Java: undo change to Function.apply test case

f040ff2

Java: remove Supplier.get model

a43f3cf

michaelnebel reviewed Jan 13, 2023

View reviewed changes

Comment thread java/ql/lib/ext/java.util.model.yml Outdated

michaelnebel reviewed Jan 13, 2023

View reviewed changes

Comment thread java/ql/lib/ext/java.util.model.yml Outdated

michaelnebel previously approved these changes Jan 13, 2023

View reviewed changes

Java: add WithoutElement comment for clear methods

fb6725d

jcogs33 dismissed michaelnebel’s stale review via fb6725d January 13, 2023 18:21

michaelnebel previously approved these changes Jan 16, 2023

View reviewed changes

atorralba reviewed Jan 16, 2023

View reviewed changes

Comment thread java/ql/lib/ext/java.lang.model.yml Outdated

Java: remove models for System.[get|set]Property

10f0975

jcogs33 dismissed michaelnebel’s stale review via 10f0975 January 17, 2023 13:52

atorralba approved these changes Jan 17, 2023

View reviewed changes

jcogs33 merged commit babdee3 into github:main Jan 17, 2023

jcogs33 deleted the jcogs33/model-more-top-jdk-apis branch January 17, 2023 15:20

jcogs33 mentioned this pull request Jan 23, 2023

Java: model remaining top-500 JDK APIs #11968

Merged

Conversation

jcogs33 commented Dec 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description:

Consideration:

Uh oh!

Uh oh!

egregius313 left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Jan 12, 2023

java

Generated file changes for java

Uh oh!

jcogs33 commented Jan 12, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

atorralba commented Jan 12, 2023

Uh oh!

github-actions Bot commented Jan 12, 2023

java

Generated file changes for java

Uh oh!

github-actions Bot commented Jan 12, 2023

java

Generated file changes for java

Uh oh!

Uh oh!

Uh oh!

michaelnebel left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Jan 13, 2023

java

Generated file changes for java

Uh oh!

michaelnebel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions Bot commented Jan 17, 2023

java

Generated file changes for java

Uh oh!

atorralba left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

jcogs33 commented Dec 21, 2022 •

edited

Loading

jcogs33 commented Jan 12, 2023 •

edited

Loading