Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiroot .env file update tests failing on CI. #4468

Closed
DonJayamanne opened this issue Feb 21, 2019 · 3 comments
Closed

Multiroot .env file update tests failing on CI. #4468

DonJayamanne opened this issue Feb 21, 2019 · 3 comments

Comments

@DonJayamanne
Copy link

@DonJayamanne DonJayamanne commented Feb 21, 2019

@d3r3kk /cc

@d3r3kk d3r3kk changed the title Root cause analysis for failing venv tests on CI Multiroot .env file update tests failing on CI. Feb 21, 2019
@d3r3kk
Copy link

@d3r3kk d3r3kk commented Feb 21, 2019

Find out and fix tests that have begun failing intermittently on AzDO CI. The tests that are failing are the last two found in the file src/test/common/variables/envVarsProvider.multiroot.test.ts. The test is failing due to the timeout of 20s being exceeded.

One hypothesis is that the tests are failing and being re-run, exacerbating the trouble with async tests accessing the same file. (Rejected due to PR #4479). Another hypothesis is that the timeouts for the test just aren't long enough.

The tests are both set to having a total timeout of 20 seconds. There are 3 places in both tests where an imposed wait of 5 seconds is being processed. This means that the test must complete all other tasks in < 5 seconds. Perhaps the overall test timeout needs to be set to something a bit higher, like 25 seconds for each test.

If it is at all possible, and we could be notified when the configuration has been updated, we could remove the waits added to the tests altogether and reduce the overall time these tests are set to run for.

@DonJayamanne
Copy link
Author

@DonJayamanne DonJayamanne commented Feb 21, 2019

Hmm, how about we consider re-writing the tests? Wouldn't that be a better approach?
Can get rid of 20s delays, and randomizing file names, etc.

d3r3kk added a commit to d3r3kk/vscode-python that referenced this issue Feb 24, 2019
- only skip for Windows
d3r3kk added a commit to d3r3kk/vscode-python that referenced this issue Feb 24, 2019
d3r3kk added a commit to d3r3kk/vscode-python that referenced this issue Feb 26, 2019
d3r3kk added a commit to d3r3kk/vscode-python that referenced this issue Feb 26, 2019
@DonJayamanne
Copy link
Author

@DonJayamanne DonJayamanne commented Feb 28, 2019

TODO: Refactor the tests.

d3r3kk added a commit to d3r3kk/vscode-python that referenced this issue Mar 1, 2019
d3r3kk added a commit to d3r3kk/vscode-python that referenced this issue Mar 5, 2019
d3r3kk added a commit to d3r3kk/vscode-python that referenced this issue Mar 9, 2019
d3r3kk added a commit to d3r3kk/vscode-python that referenced this issue Mar 12, 2019
@luabud luabud added P1 and removed P1 labels Mar 12, 2019
d3r3kk added a commit to d3r3kk/vscode-python that referenced this issue Mar 13, 2019
@d3r3kk d3r3kk removed their assignment Mar 14, 2019
d3r3kk added a commit to d3r3kk/vscode-python that referenced this issue Mar 14, 2019
@DonJayamanne DonJayamanne self-assigned this Mar 21, 2019
@ghost ghost removed the needs PR label Mar 27, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Apr 24, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants