Re: Coverage of code in our Starman process not counted?

Hi all,

After a full day of fiddling and trying, I've now merged two branches:

1. Use 'nginx' to serve static files and reverse-proxy requests for the app server (reduces/eliminates the intermittent failures)

2. Stop using Starman as the app server while testing: change '.travis.yml' and 'tools/starman.psgi' to use HTTP::Server::PSGI (plack's server)

and include coverage results in Devel::Cover output

With these branches committed (the first is required to make the second even complete the coverage variant of the tests), coverage jumped up from 11.7% to 15.4%.

The primary reason behind the jump is the fact that the BDD tests I had written over the past weeks never made the coverage go up. This inspired me to go search for the reason behind the stable coverage outcome when I at least thought I had added significant coverage to - at least - setup.pm and the modules it uses.

The above jump in coverage proves that indeed coverage should have gone up when I initially committed the BDD tests.

Now that coverage of the Perl code is fully measured and reported, I think it'd be nice to try and write enough tests to get to 70% of our (Perl) code exercised. Why that number? Well, because I think it's too hard to get it to 95% in the short term and because I think that even with 70% we can get a pretty good idea about the quality of our releases. Longer term, I advocate trying to get as high as 95%.

As you can imagine, I could get some help creating tests to get to the 95% - or even the 70% - figure. If you can help out by creating BDD scripts (even if just the end-user part of it) or when you could help out writing unit tests, please drop by in #ledgersmb on irc://chat.freenode.net. I'll gladly help anybody get started!

Kindest regards,

Erik.

On Sun, Feb 7, 2016 at 1:02 PM, Erik Huelsmann <..hidden..> wrote:

Hi all,

As it turned out, using HTTP::Server::PSGI isn't too hard (I've got a prototype running on my branch 'master-try-coverage'). However, the server seems too simple for what we want from it: coverage testing slows the server down, resulting in consistent failure on the coverage-testing-VM (however, the point of failure isn't consistent).

I'm now wondering about a good mechanism to mitigate this problem. Should we run the application server behind nginx and have nginx serve the static files? If we do it that way, pressure on the Perl server might be sufficiently reduced?

Regards,

Erik.

On Sun, Feb 7, 2016 at 10:03 AM, Erik Huelsmann <..hidden..> wrote:
Hi all,

After implementing BDD tests for setup.pm, our coverage % hasn't changed the slightest. While I had serious doubts that the code coverage hadn't changed, I left it alone, since we have lots of items to work on. OTOH, I'd really like to see a coverage figure which resembles actual code tested, so I started looking at it again.

https://github.com/pjcj/Devel--Cover/issues/50 suggests that there's no way to have starman figures extracted correctly. The same post suggests using HTTP::Server::PSGI (which comes with Plack) will work around this problem.

Does anybody have experience with this? Or even better, any help to get a solution which extracts and reports the coverage information would we highly appreciated!

--
Bye,

Erik.

http://efficito.com -- Hosted accounting and ERP.
Robust and Flexible. No vendor lock-in.

--
Bye,

Erik.

http://efficito.com -- Hosted accounting and ERP.
Robust and Flexible. No vendor lock-in.

Bye,

Erik.

http://efficito.com -- Hosted accounting and ERP.

Robust and Flexible. No vendor lock-in.