Re: State of Perl-based database setup utilities for LedgerSMB 1.3

["David F. Skoll" <..hidden..>]

On Fri, 27 May 2011 14:01:28 -0700
Chris Travers <..hidden..> wrote:

[...]

> In an upgrade you'd want to run all relevant production-safe tests on
> your database right?  Wouldn't that require a test harness?

No, I think you'd want to run the tests if you are building a
new upgrade release.  But in some distant future if LedgerSMB is packaged
as a deb or rpm, there's no reason to run any unit tests as part of the
installation of the deb or rpm.  At least, I don't think any other
Perl modules that are packaged as debs or rpms run unit-tests as part
of installation.

[...]

> Let's go over a hypothetical problem here......

> Suppose LedgerSMB 1.3 is accidentally installed into template1 and
> nobody notices.

That should be a Can't Happen scenario because ledgersmb-init-database
should prevent that.

> We could document "we highly recommend you go into the
> sql/modules/test directory and run all relevant files there" but I am
> wondering if we really should run automated tests on the database
> after it is set up.

Here's what we do for our software: For every database schema change
from the beginning of time until now, we have a pg_dump sample
database.  As part of our unit-tests, we run tests to make sure that
any previous schema can be upgraded to the current schema and we also
make sure that the so-upgraded schema matches the schema that would be
produced by a brand-new installation.

Once those regression-tests pass, there's no need to run them on deployment
systems because We Know It Will Work.  (Well, OK, there was a change from
PostgreSQL 8.3 to 8.4 or 9.0 in how automatically-created indexes were
named that threw us off, but that was easily worked around.)

LedgerSMB can run into trouble because it hasn't had a proper installation
script until now.  But once it has that, upgrades should be correct almost
by construction.

If you want to have the automated tests available, that might not
be a bad idea, but they should be fairly easy to separate out as a separate
deb or rpm (you could have ledgersmb, ledgersmb-tests, etc.)
It's obviously fine for ledgersmb-tests to depend on the Perl testing
framework.

Regards,

David.