Data flow: Cache most language-dependent predicates #5773
Conversation
hvitved
force-pushed
the
dataflow/aggressive-caching
branch
5 times, most recently
from
6af292c to
6af0c0d
Compare
hvitved
force-pushed
the
dataflow/aggressive-caching
branch
from
6af0c0d to
8074e13
Compare
hvitved
force-pushed
the
dataflow/aggressive-caching
branch
from
8074e13 to
c35a2b9
Compare
|
For us to evaluate this vs. Go we'll need to bring ourselves up to date with the recent dataflow changes first. I wouldn't block this on our review. |
|
This looks like a clear overall win for C++ on the whole suite, although it comes at the cost of slowdowns in the first query to use IR data flow ( |
|
I've updated Go's dataflow libraries to match |
Thanks. |
| * The value of a parameter at function entry, viewed as a node in a data | ||
| * flow graph. | ||
| */ | ||
| class ParameterNodeExt extends Node { |
There was a problem hiding this comment.
I'm not particularly happy with the introduction of ParameterNodeExt and ArgumentNodeExt as cached aliases. I think we should rename these cached versions to ParameterNode and ArgumentNode and ensure that these cached versions get suitable public scope exposure to make them appear as DataFlow::ParameterNode and DataFlow::ArgumentNode instead of the current ones.
The language-specific definitions with those names then need to be dealt with in some way. Either hidden away in some module to ensure we don't get name clashes, or renamed, or replaced by predicates.
There was a problem hiding this comment.
The only problem with that is that languages may want to have more specific QL doc. Also, languages may want to add more predicates, for example in C# we have a ParameterNode::getParameter() predicate. Lastly, C# does not expose ArgumentNode.
Would it help if we renamed them to ParameterNodeCached and ArgumentNodeCached? (I know that the Ext naming is not good, but I wanted to keep it short.)
There was a problem hiding this comment.
Right. How about this, then: Import the language-specific classes and predicates qualified so we can reuse the names for the cached aliases. That also has the side-effect that we don't accidentally refer to the non-cached version inside the dataflow library.
There was a problem hiding this comment.
That would also allow us to reuse the names for all the predicates that currently use the Cached postfix.
|
Here are the Go results (scroll right to see the last column). They generally look very good, except the 1% increase in time for Kubernetes.
|
That's great. Once this PR is merged, you should no longer need to cache this predicate: https://github.com/github/codeql-go/blob/main/ql/src/semmle/go/dataflow/internal/DataFlowPrivate.qll#L240 |
6 participants
This PR moves the burden of caching just the right predicates from the language-specific implementations into the shared data-flow library. It comes at the cost that we may cache some predicates that are normally trivial to compute, but I think that is outweighed by moving the caching responsibility into shared library. And even though some predicates are quite fast to compute, it may still be a win as the number of data-flow queries increase.
DefaultTaintTracking.qll, which seems to be used a lot. Performance has improved a bit, but not quite as much as I had hoped: https://jenkins.internal.semmle.com/job/Changes/job/CPP-Differences/1955/.https://jenkins.internal.semmle.com/job/Changes/job/Python-Differences/521/.