Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

utPLSQL Performance Test Suite #889

Open
PhilippSalvisberg opened this issue Apr 1, 2019 · 4 comments
Open

utPLSQL Performance Test Suite #889

PhilippSalvisberg opened this issue Apr 1, 2019 · 4 comments
Labels

Comments

@PhilippSalvisberg
Copy link
Member

@PhilippSalvisberg PhilippSalvisberg commented Apr 1, 2019

Motivation

With utPLSQL v3.1.4 - v3.1.6 some performance related issues were introduced. See #882 and #888. Fixing the issue required some collaboration with others, which worked very well. However, we should introduce a mechanism to avoid or at least try to avoid such problems in the future.

Feature Request

I suggest to write performance tests suites with the following main goal:

  • compare overall runtime between utPLSQL releases (current / previous)
    • running reporter consumer in the main process (after test completion)
    • running reporter consumer in dedicated processes (during test run)

When running a reporter consumer in a dedicated process the overhead of a reporter (in most cases) should be reduced to something like an INSERT in the producing main process. In such a setup a reporter should never be the main contributor to the overall end-to-end runtime.

Technically I think the current development environment could be used for these performance tests. It should be good enough to identify relative performance issues between the current develop branch and the latest release. A dedicated shell script could do the job.

@jgebal
Copy link
Member

@jgebal jgebal commented Apr 1, 2019

We could do similar setup for suite setup (parsing/reading cache tables)

@lwasylow
Copy link
Member

@lwasylow lwasylow commented Apr 1, 2019

There are few areas that could benefit from constant monitoring, I think quite important is to decide at what point it will run and on what volume. I believe this should be part of develop only volume wise not sure what we consider big but not too big.

@jgebal
Copy link
Member

@jgebal jgebal commented Apr 1, 2019

The only concern on my side is the build time.
Maybe we could do performance testing:

  • as a separate parallel build (on one DB version only?) - next item on matrix
  • only as part of release build (only on release branch)
  • only to be ran locally (ad-hoc)

Even today I notice that it's quite annoying to have to wait 9-10 minutes for build to finish.

@jgebal
Copy link
Member

@jgebal jgebal commented Apr 1, 2019

I see three main areas of performance testing:

  • cache and parsing
    • initial scanning schema for suites (new schema)
    • consecutive scanning for suites (some package recompiled)
    • running without scanning (read from cache)
  • output buffer on large scale of data produced
  • coverage gathering on large schema

All three scenarios could be covered with a schema containing significant number of packages and lines of code in spec/body.

We could use a package generator. Here is an old script i used(attached).
generate_and_check_parsing_for_big_schema.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.