Skip to content

/node

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: try to reduce flakes on Windows #28035

Merged
merged 4 commits into from Jun 6, 2019

Conversation

@refack
Copy link
Member

commented Jun 2, 2019 •
edited

  • test: always suffix tmpdir
    This makes temp dir names consistent whether we run in stand-alone mode,
    via test.py in single process, or in multi-process.
  • test: shell out to rmdir first on Windows
    cmd's rmdir is hardened to deal with Windows edge cases, like
    lingering processes, indexing, and AV checks. So we give it a try first.
  • test: regression test tmpdir
Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

@refack refack requested a review from Trott Jun 2, 2019

@refack refack self-assigned this Jun 2, 2019

@nodejs-github-bot

This comment was marked as outdated.

Sign in to view
Copy link

commented Jun 2, 2019

CI: https://ci.nodejs.org/job/node-test-pull-request/23573/
@refack

This comment has been minimized.

Sign in to view
Copy link
Member Author

commented Jun 2, 2019

/CC @nodejs/testing
@mscdex

This comment has been minimized.

Sign in to view
Copy link
Contributor

commented Jun 2, 2019

Shouldn't the subsystem prefix here just be test: ?

@refack refack force-pushed the refack:testutils-rmdir-first-on-windows branch from 297daf1 to 9c57dea Jun 2, 2019

@richardlau

This comment has been minimized.

Sign in to view
Copy link
Member

commented Jun 2, 2019

Please could you add a brief sentence or two in the commit message as to why?
@refack

This comment has been minimized.

Sign in to view
Copy link
Member Author

commented Jun 2, 2019

Shouldn't the subsystem prefix here just be test: ?

Fixed.

@refack refack force-pushed the refack:testutils-rmdir-first-on-windows branch from 9c57dea to b557192 Jun 2, 2019

@refack

This comment has been minimized.

Sign in to view
Copy link
Member Author

commented Jun 2, 2019

Please could you add a brief sentence or two in the commit message as to why?

Wrote some words. Didn't have anything too enlightening 🤷‍♂.
WRT to refresh() nothing is extremely robust, it's just a extra heuristic to overcome some recent failures seen in CI:

@nodejs nodejs deleted a comment from nodejs-github-bot Jun 2, 2019

@nodejs-github-bot

This comment was marked as outdated.

Sign in to view
Copy link

commented Jun 2, 2019

CI: https://ci.nodejs.org/job/node-test-pull-request/23575/

@refack refack force-pushed the refack:testutils-rmdir-first-on-windows branch from c869ded to ccf9100 Jun 2, 2019

@nodejs-github-bot

This comment was marked as outdated.

Sign in to view
Copy link

commented Jun 2, 2019

CI: https://ci.nodejs.org/job/node-test-pull-request/23576/

@mscdex mscdex changed the title test,lib: try `rmdir` first on windows test: try `rmdir` first on windows Jun 3, 2019

@Trott
Trott reviewed Jun 3, 2019
View changes
test/async-hooks/test-pipeconnectwrap.js Outdated Show resolved Hide resolved

@refack refack force-pushed the refack:testutils-rmdir-first-on-windows branch 2 times, most recently from 55fd037 to a17d5ba Jun 3, 2019

@nodejs nodejs deleted a comment from nodejs-github-bot Jun 3, 2019

@nodejs-github-bot

This comment was marked as outdated.

Sign in to view
Copy link

commented Jun 3, 2019

CI: https://ci.nodejs.org/job/node-test-pull-request/23578/

@refack refack added the test label Jun 3, 2019

@Trott
Trott approved these changes Jun 3, 2019
View changes
Copy link
Member

left a comment

LGTM if CI is green and once info about opts is added to the tmpdir.refresh() entry in test/common/README.md.
@Trott

This comment has been minimized.

Sign in to view
Copy link
Member

commented Jun 3, 2019

@nodejs/platform-windows
@lpinca
lpinca reviewed Jun 3, 2019
View changes
test/common/tmpdir.js Show resolved Hide resolved
@refack
refack reviewed Jun 3, 2019
View changes
test/common/tmpdir.js Show resolved Hide resolved
@BridgeAR
BridgeAR approved these changes Jun 4, 2019
View changes
Copy link
Member

left a comment

LGTM but it seems like on Windows it will now always try both ways, even though the folder might already be deleted by rmdir.
test/common/tmpdir.js
// On Windows first try to delegate rmdir to a shell.
if (process.platform === 'win32' && st.isDirectory()) {
try {
execSync(`rmdir /q /s ${pathname}`, { timout: 1000 });

This comment has been minimized.

Sign in to view
Copy link
@BridgeAR

BridgeAR Jun 4, 2019

Member

If everything passes, should this not return?

This comment has been minimized.

Sign in to view
Copy link
@refack

refack Jun 6, 2019

Author Member

Not totally sure why, but it's not trivial https://ci.nodejs.org/job/node-test-binary-windows-2/1275/
@benjamingr
benjamingr approved these changes Jun 4, 2019
View changes
@sam-github
sam-github approved these changes Jun 5, 2019
View changes
@sam-github

This comment has been minimized.

Sign in to view
Copy link
Member

commented Jun 5, 2019

Is this intended to fix failures like below?


c:\workspace\node-test-binary-windows-2\test\common\tmpdir.js:27
      throw e;
      ^

Error: ENOTEMPTY: directory not empty, rmdir 'c:\workspace\node-test-binary-windows-2\test\.tmp.6'
    at Object.rmdirSync (fs.js:693:3)
    at rmdirSync (c:\workspace\node-test-binary-windows-2\test\common\tmpdir.js:48:10)
    at rimrafSync (c:\workspace\node-test-binary-windows-2\test\common\tmpdir.js:18:7)
    at Object.refresh (c:\workspace\node-test-binary-windows-2\test\common\tmpdir.js:66:3)
    at Object.<anonymous> (c:\workspace\node-test-binary-windows-2\test\parallel\test-child-process-spawnsync-args.js:18:8)
    at Module._compile (internal/modules/cjs/loader.js:777:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:788:10)
    at Module.load (internal/modules/cjs/loader.js:640:32)
    at Function.Module._load (internal/modules/cjs/loader.js:555:12)
    at Function.Module.runMain (internal/modules/cjs/loader.js:840:10) {
  errno: -4051,
  syscall: 'rmdir',
  code: 'ENOTEMPTY',
  path: 'c:\\workspace\\node-test-binary-windows-2\\test\\.tmp.6'
}

https://ci.nodejs.org/job/node-test-binary-windows-2/1244/COMPILED_BY=vs2017,RUNNER=win2008r2-vs2017,RUN_SUBSET=0/testReport/junit/(root)/test/parallel_test_child_process_spawnsync_args/
@refack

This comment has been minimized.

Sign in to view
Copy link
Member Author

commented Jun 5, 2019

Is this intended to fix failures like below?

Exactly.

@refack refack force-pushed the refack:testutils-rmdir-first-on-windows branch from 91c51b2 to bf5b8e8 Jun 5, 2019

@refack

This comment has been minimized.

Sign in to view
Copy link
Member Author

commented Jun 5, 2019 •
edited

@Trott added doc

@BridgeAR I wanted this to be best-effort with minimal change to current semantics.
had to rebase and fix some more stuff, so I added a return.

I'm going to land this to reduce flakes in CI, and I'll follow up with teaks.
@nodejs-github-bot

This comment was marked as outdated.

Sign in to view
Copy link

commented Jun 5, 2019

CI: https://ci.nodejs.org/job/node-test-pull-request/23638/

@refack refack force-pushed the refack:testutils-rmdir-first-on-windows branch from bf5b8e8 to 4059eba Jun 5, 2019

@nodejs nodejs deleted a comment from nodejs-github-bot Jun 5, 2019

@refack refack force-pushed the refack:testutils-rmdir-first-on-windows branch from 4059eba to e531a68 Jun 5, 2019

@nodejs-github-bot

This comment was marked as outdated.

Sign in to view
Copy link

commented Jun 5, 2019

CI: https://ci.nodejs.org/job/node-test-pull-request/23643/

@refack refack changed the title test: try `rmdir` first on windows test: try to reduce flakes on Windows Jun 5, 2019

@refack refack force-pushed the refack:testutils-rmdir-first-on-windows branch from e531a68 to 803a946 Jun 5, 2019

@nodejs-github-bot

This comment was marked as outdated.

Sign in to view
Copy link

commented Jun 5, 2019

CI: https://ci.nodejs.org/job/node-test-pull-request/23646/
@Trott
Trott reviewed Jun 6, 2019
View changes
test/common/README.md Outdated Show resolved Hide resolved
@Trott
Trott approved these changes Jun 6, 2019
View changes
Copy link
Member

left a comment

LGTM with minor doc update to the function signature

@refack refack force-pushed the refack:testutils-rmdir-first-on-windows branch from 7fce3e9 to 60133a1 Jun 6, 2019

@nodejs-github-bot

This comment has been minimized.

Sign in to view
Copy link

commented Jun 6, 2019

CI: https://ci.nodejs.org/job/node-test-pull-request/23685/
@refack refack referenced this pull request Jun 6, 2019
Closed
refack added 4 commits Jun 2, 2019
@refack
test: shell out to `rmdir` first on Windows 
cmd's `rmdir` is hardened to deal with Windows edge cases, like
lingering processes, indexing, and AV checks. So we give it a try first.

* Added `opts = { spawn = true }` to opt-out of spawning
* test-pipeconnectwrap.js - spawning messes up async_hooks state

PR-URL: #28035
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
59f666c
@refack
test: always suffix `tmpdir` 
This makes temp dir names consistent whether we run in stand-alone mode,
via `test.py` in single process, or in multi-process.

PR-URL: #28035
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
9032ab4
@refack
test: regression test `tmpdir` 
PR-URL: #28035
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
2f8cf5e
@refack
test: mark test-worker-debug as flaky 
Also try to make more traceable.

PR-URL: #28035
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
Loading status checks…
65a5f7b

@refack refack force-pushed the refack:testutils-rmdir-first-on-windows branch from 60133a1 to 65a5f7b Jun 6, 2019

@refack refack merged commit 65a5f7b into nodejs:master Jun 6, 2019
7 checks passed

7 checks passed

Travis CI - Branch Build Passed
Details
Travis CI - Pull Request Build Passed
Details
node-test-commit flaky tests failed
Details
node-test-commit-freebsd tests passed
Details
node-test-commit-linux tests passed
Details
node-test-commit-plinux flaky tests failed
Details
node-test-pull-request flaky tests failed
Details
@refack

This comment has been minimized.

Sign in to view
Copy link
Member Author

commented Jun 6, 2019

landed in:
59f666c
9032ab4
2f8cf5e
65a5f7b

@refack refack deleted the refack:testutils-rmdir-first-on-windows branch Jun 6, 2019

@refack refack removed their assignment Jun 7, 2019

@refack refack referenced this pull request Jun 10, 2019
Open
BridgeAR added a commit that referenced this pull request Jun 17, 2019
@refack @BridgeAR
test: shell out to `rmdir` first on Windows 
cmd's `rmdir` is hardened to deal with Windows edge cases, like
lingering processes, indexing, and AV checks. So we give it a try first.

* Added `opts = { spawn = true }` to opt-out of spawning
* test-pipeconnectwrap.js - spawning messes up async_hooks state

PR-URL: #28035
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
Verified
This commit was signed with a verified signature.
BridgeAR Ruben Bridgewater
GPG key ID: F07496B3EB3C1762 Learn about signing commits
f336236
BridgeAR added a commit that referenced this pull request Jun 17, 2019
@refack @BridgeAR
test: always suffix `tmpdir` 
This makes temp dir names consistent whether we run in stand-alone mode,
via `test.py` in single process, or in multi-process.

PR-URL: #28035
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
Verified
This commit was signed with a verified signature.
BridgeAR Ruben Bridgewater
GPG key ID: F07496B3EB3C1762 Learn about signing commits
e08a98f
BridgeAR added a commit that referenced this pull request Jun 17, 2019
@refack @BridgeAR
test: regression test `tmpdir` 
PR-URL: #28035
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
Verified
This commit was signed with a verified signature.
BridgeAR Ruben Bridgewater
GPG key ID: F07496B3EB3C1762 Learn about signing commits
098cf74
BridgeAR added a commit that referenced this pull request Jun 17, 2019
@refack @BridgeAR
test: mark test-worker-debug as flaky 
Also try to make more traceable.

PR-URL: #28035
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
Verified
This commit was signed with a verified signature.
BridgeAR Ruben Bridgewater
GPG key ID: F07496B3EB3C1762 Learn about signing commits
3feaf3d
@BridgeAR BridgeAR referenced this pull request Jun 17, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lpinca lpinca

@sam-github sam-github

@Trott Trott

@benjamingr benjamingr

@BridgeAR BridgeAR

Assignees
No one assigned
Labels
test
Projects
None yet
Milestone
No milestone
9 participants
@refack @nodejs-github-bot @mscdex @richardlau @Trott @sam-github @benjamingr @lpinca @BridgeAR
You can’t perform that action at this time.