Skip to content

JS: Add individual per-security-query counting queries #9193

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 25, 2022

Conversation

TomBolton
Copy link
Contributor

The ATM project uses an AML pipeline to select which CodeQL databases to use as the evaluation databases for each security query being boosted.

Previously, evaluation sets for all security queries would be produced at once, using the CountAlertsAndEndpoints.ql query. However, this query does not scale with adding more security queries, and it is not efficient to create evaluation sets for all queries during every pipeline run (when adding a new query, you simply want to create an evaluation set for that one query, and not the others).

Therefore, after a discussion, it was deemed better to specify one security query that you would like to create an evaluation set for. For this, we would need a way to count alerts and endpoints for a single query. The most simple solution is to add per-query counting queries, Count{QUERY_NAME}.ql, which can be used by the selection pipeline.

The selection pipeline has been tested with the CountCodeInjection.ql query.

@TomBolton TomBolton added the JS label May 17, 2022
@TomBolton TomBolton requested a review from a team May 17, 2022 12:33
@github-actions github-actions bot removed the JS label May 17, 2022
@esbena
Copy link
Contributor

esbena commented May 17, 2022

This introduces a lot of duplicate codeql code that can be avoided using imports and query predicates.

I think each query could look something like this:

import semmle.javascript.security.dataflow.CodeInjectionQuery
import CountThings

Where CountThings.qll looks something like:

import javascript
import evaluation.EndToEndEvaluation

query predicate countThings(int numAlerts, int numSinks) { 
  numAlerts =
    count(DataFlow::Node source, DataFlow::Node sink |
      cfg.hasFlow(source, sink) and not isFlowExcluded(source, sink)
    ) and
  numSinks = count(DataFlow::Node sink |
    exists(DataFlow::Configuration cfg | cfg.isSink(sink) or cfg.isSink(sink, _))
  )

Could you consider if such a refactoring is possible / worth it?

@TomBolton
Copy link
Contributor Author

Yes, good point @esbena and I definitely think it's worth it.

Basic QL question: what would the individual queries look like? I.e. how would they actually use the CountThings import? I've checked the documentation and it's not clear how to import and use a predicate

@esbena
Copy link
Contributor

esbena commented May 17, 2022

The trick is that query predicate contributes to the results of the query file when imported. You can view it as an importable from/where/select

@esbena
Copy link
Contributor

esbena commented May 17, 2022

Ah, there was a critical typo: I have updated the example file name to end with qll now!

@TomBolton
Copy link
Contributor Author

I've just attempted the suggestion refactor @esbena - would you mind giving it another look when you have time?

@TomBolton
Copy link
Contributor Author

@github/codeql-ml-powered-queries-reviewers if anyone has a spare moment, could someone provide a quick review?

esbena
esbena previously approved these changes May 23, 2022
Copy link
Contributor

@esbena esbena left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the as CodeInjection/TaintedPath/... namings can be dropped now.
Otherwise LGTM.

@TomBolton TomBolton force-pushed the tombolton/add-counting-queries branch from 5b34741 to 91fa17a Compare May 24, 2022 14:02
@TomBolton
Copy link
Contributor Author

Thanks for the review @esbena - I've now removed the as in the imports.

I accidentally dismissed your approval, would you mind approving again sorry?

Copy link
Contributor

@esbena esbena left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Your edit caused the approval dismissal)

@TomBolton TomBolton merged commit 67572bb into main May 25, 2022
@TomBolton TomBolton deleted the tombolton/add-counting-queries branch May 25, 2022 09:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants