Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add reporting for code coverage #561

Open
andreasprlic opened this issue Aug 15, 2016 · 17 comments
Open

Add reporting for code coverage #561

andreasprlic opened this issue Aug 15, 2016 · 17 comments
Assignees
Labels
Milestone

Comments

@andreasprlic
Copy link
Member

@andreasprlic andreasprlic commented Aug 15, 2016

Currently we don't have any reports on the code coverage. Add support for e.g. https://coveralls.io/

see also

http://blog.eluder.org/2013/06/code-coverage-for-github-hosted-java-projects-with-coveralls

@andreasprlic andreasprlic added the task label Aug 15, 2016
@sbliven
Copy link
Member

@sbliven sbliven commented Aug 15, 2016

I'm scared to see what our coverage is o_o

andreasprlic added a commit to andreasprlic/biojava that referenced this issue Aug 15, 2016
@andreasprlic andreasprlic self-assigned this Aug 15, 2016
@andreasprlic
Copy link
Member Author

@andreasprlic andreasprlic commented Aug 15, 2016

quick status update: there are a few problems with this

  • log files length > 4 MB cause Travis to be unhappy
  • cobertura plugin has problems with MMCifFileTools.java
    (I suspect with getFields(Class<?> c)
  • when running locally, problems with SymmetryTools.getQuaternarySymmetry(CeSymmResult) (stream operation on collections causing problems?)
@andreasprlic
Copy link
Member Author

@andreasprlic andreasprlic commented Aug 15, 2016

running mvn cobertura:cobertura coveralls:report from command line currently fails with:

Build error: Line number 91 is greater than the source file biojava-structure/src/main/java/demo/DemoFATCAT.java size -> [Help 1]

@sbliven
Copy link
Member

@sbliven sbliven commented Aug 16, 2016

I'm guessing that comes from the fact that we have two demo.DemoFATCAT classes. The one from biojava-structure has 87 lines, while biojava-structure-gui has 101.

@andreasprlic
Copy link
Member Author

@andreasprlic andreasprlic commented Aug 16, 2016

With the latest commits on my branch, I could complete a run. Results should show up here soon https://coveralls.io/jobs/17327290

@andreasprlic
Copy link
Member Author

@andreasprlic andreasprlic commented Aug 16, 2016

I don't see an immediate work around to the problem of the long-log file. This is an issue with Travis: travis-ci/travis-ci#3865

For now, #555 allows users to run the coverage reports locally. We can come back to this once the Travis issue can be addressed.

@andreasprlic
Copy link
Member Author

@andreasprlic andreasprlic commented Aug 16, 2016

Btw: currently we have about 38% code coverage.

@phidias51
Copy link

@phidias51 phidias51 commented Aug 17, 2016

Andreas,
If the log file is configured using a rolling appender, you should be able
to skirt the file size issue.

Mark

On Aug 16, 2016 2:21 PM, "Andreas Prlic" notifications@github.com wrote:

Btw: currently we have about 38% code coverage.


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#561 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABOqRylwSH2Eipl571BUSe_GwrVKYNOZks5qginIgaJpZM4JkHn9
.

@andreasprlic
Copy link
Member Author

@andreasprlic andreasprlic commented Aug 17, 2016

so, the problem is not to run this locally, but using the travis-ci engine at https://travis-ci.org/biojava/biojava . This service terminates a build if the log is too long. If I configure a rolling appender, that would influence my local builds, but it would not change the behavior of travis-ci. Do I understand that correctly?

@josemduarte
Copy link
Contributor

@josemduarte josemduarte commented Aug 17, 2016

A possible easy fix for the too-long log problem is to switch the logging level to warn for travis. In any case going forward we should really try to cut the verbosity of some of the tests.

@phidias51
Copy link

@phidias51 phidias51 commented Aug 17, 2016

Hi Andreas,
Changing the log level would help, as would not dumping out sequences. :-)
I assume that every module has its own test log; if not, that might help as
well. Setting the rolling appender size would further reduce the log size,
by causing the log to roll over to another file whenever it reaches a
predefined limit. You end up with a series of log files (test.log.0,
test.log.1, etc).

Hope this helps,

Mark

On Wed, Aug 17, 2016 at 2:22 AM, Jose Manuel Duarte <
notifications@github.com> wrote:

A possible easy fix for the too-long log problem is to switch the logging
level to warn for travis. In any case going forward we should really try to
cut the verbosity of some of the tests.


You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#561 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABOqR_HI3C4Z8LshszpkOVEns4X-0BlZks5qgtLggaJpZM4JkHn9
.

@andreasprlic andreasprlic reopened this Aug 18, 2016
@andreasprlic
Copy link
Member Author

@andreasprlic andreasprlic commented Aug 18, 2016

working on reducing verbosity of unit tests. See #555

@sbliven sbliven added this to the BioJava 5.0.1 milestone Sep 28, 2016
@ghost
Copy link

@ghost ghost commented Nov 24, 2016

Why do not use Atlassian Clover?

A powerful code coverage tool supporting 20+ metrics. Here is a nice Clover/Cobertura comparison.

And it's for free for open source projects https://www.atlassian.com/software/clover/pricing.

BTW: We can set a static code analysis tool on top of that for example SonarQube aggregating additional code quality information.

@andreasprlic
Copy link
Member Author

@andreasprlic andreasprlic commented Nov 28, 2016

I'd prefer a solution that works on the GitHub ecosystem.

@ghost
Copy link

@ghost ghost commented Nov 28, 2016

@andreasprlic

Atlassian products are very powerful but I agree that it is better to choose a solution that works on the GitHub ecosystem.

@andreasprlic
Copy link
Member Author

@andreasprlic andreasprlic commented Nov 28, 2016

Yes they are, I am using many of them in my daily work.

@lafita lafita added the testing label Mar 11, 2018
@josemduarte
Copy link
Contributor

@josemduarte josemduarte commented Sep 2, 2019

SonarQube (mentioned in #789) should be a good tool for coverage and for many other quality controls. They provide github integration with their cloud service for free for open source projects.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.