On IRC, we were talking about making the BDD tests be executed in parallel. At the time, I objected to declare them 'parallel' as of now, because there are interactions between the feature files. (Mind you that I'm deliberately not saying that the scenarios can be parallellized at all, which is a completely different problem.)
Let me take this opportunity to explain what the problem for the BDD tests and parallellism is: all feature files use a "template" database to speed up the execution of the scenarios: for each scenario a new database is created, unless specified otherwise. However, database creation takes a *lot* of time. So, instead of creating a new database from scratch, each feature creates a template database once and copies that database later. Which is *much* faster. Also, each feature file creates the template to ensure tests are being run against the latest version of the schema. Especially when running these tests on a developer's local system, this is important. It's much less important on Travis, as that starts out with a clean system anyway.
Note that it uses the global variable `$company_seq` right above the step definition to create sequentially numbered (unique) test databases. These test databases are all called "standard-<sequence-number>". This is a problem under the `prove` execution of the feature files, because the features are executed as separate processes (forks) and by consequence the sequence numbers will be re-used.
Today I was thinking what we can do to speed up the testing of the feature files under `prove` with parallellism. This is what I came up with:
* If we change the 'ensure_template` function to create a template unless the environment variable LSMB_NEW_DB has a value, in which case we assume that to be a valid template.
* If we change the name of the test database instances to include the $$ (process id) (next to the sequence number, that is), we will have unique test database names, at least for the duration of the test run.
The second point should probably include removing the database when the test completes successfully (because otherwise the local developer's database gets poluted with loads of test remnants).
Robust and Flexible. No vendor lock-in.