Progress toward 1.5.0 - status update

For some time now, we've talked about wanting to release 1.5.0 and the beta-4 was planned for last week, but hasn't been released. Time to send an update.

Early January John Locke tested 1.5.0 with his production work flows and found a number of issues indicating that 1.5 might not be as production ready as we believed - having released 3 prior beta's. Now the problems he found aren't really severe as in they are not accounting problems - "just" user interface issues - but they do cripple some very important workflows (like printing invoices).

While his findings can mostly be fixed pretty quickly in an ad-hoc manner, we as a project have been doing that for a long time already. This approach - while really low on the initial investment - has two major drawbacks: it's hard to know if "things stay fixed" and some large amount of time is spent "fixing fixes".

We have testing in place and for some years we have coverage reporting on our tests in place (see https://coveralls.io/github/ledgersmb/LedgerSMB -- you may need to log in using your GitHub credentials to see the individual files and their coverage; I think you can view coverage stats without a login).

When we started out with the coverage reports, our test coverage showed to be as low as 4% (can't remember the exact figures). Over the years we struggled to get to the 11.7% coverage our tests had January 16th, 2016 -- in part because we acknowledged that the "old code" isn't structured to allow unit testing.

For some years now, I've been advocating that we start building browser-based tests, because these tests:

a) test workflows

b) can be run irrespective of the paradigm of the underlying code

c) test the experience in the browser (which includes [optionally broken] _javascript_)

Mid to late 2015, we had some experiments with SauceLabs to see if thteir infrastructure could help, but nothing came immediately from it.

Instead of fixing the reported problems in the same ad-hoc manner we have been for years, I decided to bite the bullet and start developing browser based tests. Taking advantage of our lagging adoption of these, we're able to do it first-time-right: I'm using BDD (Test::BDD::Cucumber) to specify the application behaviours, abstracting away page accesses behind a PageObject model. One of the reasons to go for BDD is to open up contribution of test-scripts to a wider audience than just the core LedgerSMB developers.

With these efforts, I've been able to nearly-double the test coverage in 5 days: 11.7% on Tuesday to 22.3% today!

As you can see, we've been very busy developing infrastructure supportive of releasing higher quality software -- at the expense of visible progress in terms of release size and timing. When we release 1.5.0, we want it to be better than any of the .0 versions we released so far and we're working hard to make it so.

All in all: we're delayed, but we're using that time to improve application quality. Not by doing major (destabilizing) code rewrites, but by implementing rigorous automated quality validations.

If you feel like helping out improving our quality checks: with our choice for BDD, we have lowered the barrier of entry to contribution. To contribute a BDD script, have a look at the scripts we already have in place at https://github.com/ledgersmb/LedgerSMB/tree/master/t/66-cucumber/01-basic ; we don't ask contributors to write the step implementations: just the BDD script. We'll implement the missing steps.

Bye,

Erik.

http://efficito.com -- Hosted accounting and ERP.

Robust and Flexible. No vendor lock-in.