Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

C++ view AST / printAST.ql performance analysis #11221

Open
dwendt opened this issue Nov 10, 2022 · 1 comment
Open

C++ view AST / printAST.ql performance analysis #11221

dwendt opened this issue Nov 10, 2022 · 1 comment
Labels
question Further information is requested

Comments

@dwendt
Copy link

dwendt commented Nov 10, 2022

Is there anything the end user can do to triage extremely slow 24hr+ printAST requests? It's for a single file that has an include that relies on a lot of other compilation units. The 'db-cpp' folder is 92GB - creating a smaller slice with only this file and required compilation units wouldn't make it any faster, I think, as this query should only be accessing data related to this file to begin with? If that's a valid approach to improve this query's perf - is there an easy way to slice the existing DB or does a new build need to be run with some arg?

"Iteration 30-32" took ~4hrs each doing something similar to DumpDeclaration::getTemplateArgumentsString. Java has an 8gb heap and pegs a single core. Is there some preferred tracing for codeql's java components I can capture?

Reason I'm trying to view AST is just so that I don't have to guess at how the code flow I'm looking at is represented by codeql 😄

@dwendt dwendt added the question Further information is requested label Nov 10, 2022
@aibaars
Copy link
Contributor

aibaars commented Nov 11, 2022

Thanks for the report. Wow, 92GB is a large database! You're right that AST building is likely to be a very local analysis.

If possible, one thing to try would be to increase the amount of RAM a lot. CodeQL is a database engine and it likely tries to load the large database tables into memory . If CodeQL runs out of memory it needs to swap things out to disk which impacts performance.

It would be interesting to try if a smaller database helps. There is no way to "slice" an existing database though. You can create a small database by running CodeQL with a build command that only compiles the interesting file(s). For example codeql database create tiny-database -lcpp -s my/source/folder --command "gcc -c the/file.cpp"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants