Skip to content

/ node

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: refactor coverage logic #35767

Closed
wants to merge 4 commits into from
+27 −40
Closed

test: refactor coverage logic #35767

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
5877a15
test: refactor coverage logic
bcoe Oct 23, 2020
31359a5
Apply suggestions from code review
bcoe Oct 23, 2020
025a363
chore: address code review
bcoe Oct 23, 2020
cd901bb
Update Makefile
bcoe Oct 23, 2020
File filter...
Filter file types
Jump to…
Jump to file
Failed to load files.

Always

Just for now

32 BUILDING.md
View file
Open in desktop
@@ -389,28 +389,32 @@ $ make coverage
```

A detailed coverage report will be written to `coverage/index.html` for
JavaScript coverage and to `coverage/cxxcoverage.html` for C++ coverage
(if you only want to run the JavaScript tests then you do not need to run
the first command `./configure --coverage`).
JavaScript coverage and to `coverage/cxxcoverage.html` for C++ coverage.

_Generating a test coverage report can take several minutes._

To collect coverage for a subset of tests you can set the `CI_JS_SUITES` and
`CI_NATIVE_SUITES` variables (to run specific suites, e.g., `child-process`, in
isolation, unset the opposing `_SUITES` variable):
If you only want to run the JavaScript tests then you do not need to run
the first command (`./configure --coverage`). Run `make coverage-run-js`,
to execute JavaScript tests independently of the C++ test suite:

```text
$ CI_JS_SUITES=child-process CI_NATIVE_SUITES= make coverage
$ make coverage-run-js
```

The above command executes tests for the `child-process` subsystem and
outputs the resulting coverage report.
If you are updating tests and want to collect coverrage for a single test file
(e.g. `test/parallel/test-stream2-transform.js`):

```text
$ make coverage-clean
$ NODE_V8_COVERAGE=coverage/tmp python tools/test.py test/parallel/test-stream2-transform.js

This comment has been minimized.

Sign in to view
@bcoe

bcoe Oct 23, 2020
Author Member

You can just set NODE_V8_COVERAGE, and then run tests any which way.
$ make coverage-report-js
```

Alternatively, you can run `make coverage-run-js`, to execute JavaScript tests
independently of the C++ test suite:
You can collect coverage for the entire suite of tests for a given subsystem
by providing the name of a subsystem:

```text
$ CI_JS_SUITES=fs CI_NATIVE_SUITES= make coverage-run-js
$ make coverage-clean
$ NODE_V8_COVERAGE=coverage/tmp python tools/test.py -J --mode=release child-process
$ make coverage-report-js
```

The `make coverage` command downloads some tools to the project root directory.
35 Makefile
View file
Open in desktop
@@ -197,20 +197,11 @@ check: test
# Remove files generated by running coverage, put the non-instrumented lib back
# in place
coverage-clean:
if [ -d lib_ ]; then $(RM) -r lib; mv lib_ lib; fi
$(RM) -r node_modules
$(RM) -r gcovr build
$(RM) -r out/$(BUILDTYPE)/.coverage
$(RM) out/$(BUILDTYPE)/obj.target/node/gen/*.gcda
$(RM) out/$(BUILDTYPE)/obj.target/node/src/*.gcda
$(RM) out/$(BUILDTYPE)/obj.target/node/src/tracing/*.gcda
$(RM) out/$(BUILDTYPE)/obj.target/node/gen/*.gcno
$(RM) out/$(BUILDTYPE)/obj.target/node/src/*.gcno
$(RM) out/$(BUILDTYPE)/obj.target/node/src/tracing/*.gcno
$(RM) out/$(BUILDTYPE)/obj.target/cctest/src/*.gcno
$(RM) out/$(BUILDTYPE)/obj.target/cctest/test/cctest/*.gcno
$(RM) out/$(BUILDTYPE)/obj.target/embedtest/src/*.gcno
$(RM) out/$(BUILDTYPE)/obj.target/embedtest/test/embedding/*.gcno
$(RM) -r coverage/tmp
$(FIND) out/$(BUILDTYPE)/obj.target \( -name "*.gcda" -o -name "*.gcno" \) \
-type f -exec $(RM) {} \;

.PHONY: coverage
# Build and test with code coverage reporting. Leave the lib directory
@@ -245,8 +236,8 @@ coverage-test: coverage-build
$(RM) out/$(BUILDTYPE)/obj.target/node/src/*/*.gcda
$(RM) out/$(BUILDTYPE)/obj.target/node_lib/src/*.gcda
$(RM) out/$(BUILDTYPE)/obj.target/node_lib/src/*/*.gcda
-NODE_V8_COVERAGE=out/$(BUILDTYPE)/.coverage \
TEST_CI_ARGS="$(TEST_CI_ARGS) --type=coverage" $(MAKE) $(COVTESTS)
-NODE_V8_COVERAGE=coverage/tmp \
TEST_CI_ARGS="$(TEST_CI_ARGS) --type=coverage" $(MAKE) $(COVTESTS)
$(MAKE) coverage-report-js
-(cd out && "../gcovr/scripts/gcovr" \
--gcov-exclude='.*\b(deps|usr|out|cctest|embedding)\b' -v \
@@ -259,17 +250,10 @@ coverage-test: coverage-build
@grep -A3 Lines coverage/cxxcoverage.html | grep style \
| sed 's/<[^>]*>//g'| sed 's/ //g'

COV_REPORT_OPTIONS = --reporter=html \
--temp-directory=out/$(BUILDTYPE)/.coverage --omit-relative=false \

This comment has been minimized.

Sign in to view
@bcoe

bcoe Oct 23, 2020
Author Member

these options are encapsulated in the .nycrc file.
--resolve=./lib --exclude="benchmark/" --exclude="deps/" --exclude="test/" --exclude="tools/" \
--wrapper-length=0
ifdef COV_ENFORCE_THRESHOLD
COV_REPORT_OPTIONS += --check-coverage --lines=$(COV_ENFORCE_THRESHOLD)
endif

.PHONY: coverage-report-js
coverage-report-js:
$(NODE) ./node_modules/.bin/c8 report $(COV_REPORT_OPTIONS)
-$(MAKE) coverage-build-js
$(NODE) ./node_modules/.bin/c8 report

.PHONY: cctest
# Runs the C++ tests using the built `cctest` executable.
@@ -305,9 +289,8 @@ tooltest:

.PHONY: coverage-run-js
coverage-run-js:
$(RM) -r out/$(BUILDTYPE)/.coverage
$(MAKE) coverage-build-js
-NODE_V8_COVERAGE=out/$(BUILDTYPE)/.coverage CI_SKIP_TESTS=$(COV_SKIP_TESTS) \
$(RM) -r coverage/tmp
-NODE_V8_COVERAGE=coverage/tmp CI_SKIP_TESTS=$(COV_SKIP_TESTS) \
TEST_CI_ARGS="$(TEST_CI_ARGS) --type=coverage" $(MAKE) jstest
$(MAKE) coverage-report-js

ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.