Re: Test suite configuration

[Chris Travers <..hidden..>]

The goal of the environment variables originally was to provide
high-level configuration for the test suite.  I think we are at a
point where this is breaking down a bit.

I would suggest we go with some sort of test.conf file which provides
specific configuration for the tests themselves but use environment
variables for two specific tasks:

1.  What sorts of tests to run, and
2.  Username and password within the tests.

Best Wishes
Chris Travers

On Wed, Jan 13, 2016 at 6:31 AM, David G <..hidden..> wrote:
> Hi Erik,
>
> As Yoda would say,
> Done well you have.
>
> One thing that stands out for me, test 63-lwp.t runs on the DB
> destructively.
> Can I suggest we have a check in the test that it is running against a
> *very* specifically named database.
> If it's not running against that DB then error out loudly.
>
> I haven't dug into that test yet, so some sort of protection may already
> be there, if so, we should update the document to say so.
>
> During development I am regularly running the tests here although not on
> a production system, but playing safe with destructive tests is probably
> good practice, if we don't someone will eventually blow their database
> away by mistake.
>
> Other responses inline....
>
> On 13/01/16 06:15, Erik Huelsmann wrote:
>> Hi all,
>>
>>
>> In an attempt to document our test suite better (of course in
>> preparation of soliciting contributions :-) ), I've created what is
>> now at
>> https://github.com/ledgersmb/LedgerSMB/tree/master/t#test-file-number-allocation
>>
>> As you can see on that page, we have quite a number of parameters to
>> be configured. To make matters worse, I've just introduced a
>> requirement for even more configuration: on my master-bdd-basics I'm
>> now able to run 2 BDD scenarios.
>>
>> While our current approach (environment variables) works for small
>> numbers (short-valued) of configuration parameters, the more complex
>> our test suite grows, the more likely we are to need more
>> configuration parameters. Also, currently we depend mostly on
>> Travis-CI to run our tests for us (or at least, I do), but I want more
>> tests running locally, which also means that I want my local
>> configuration stored somewhere. Preferably somewhere so I don't have
>> to configure every local repository separately.
>>
>> Am I seeing problems that are not there? Or should we make it easy to
>> run tests locally by storing config parameters in a config file
>> instead of/next to reading the environment?
>>
> I think you are on the right track. in almost all cases the tests can be
> run on any system with the same config, excepting things like Usernames
> and Passwords.
>
> As you are aware I am working on a revamp of our existing Sysconfig.pm
> (handles sane defaults and reading config values from ledgersmb.conf).
> One of the changes to this will allow it to be used with any config file
> rather than a hard coded ledgersmb.conf.
> I was intending to allow 2 methods of setting the config file, An
> Environment variable (easiest within a normal deployment) and as a
> script commandline param (good for direct testing of the script)
>
> Once this is implemented it should be trivial to use the same
> Sysconfig.pm for the test scripts as the LedgerSMB instance will run
> with during the tests.
> The config file would just need to be set in the Environment Variable first.
>
> It should be possible to set 2 config files at the same time, and load
> values from them in order.
> This would allow the "standard" test config to be loaded, then
> overridden by a site local config that only sets values that need to be
> changed like usernames etc
>>
>> --
>> Bye,
>>
>> Erik.
>>
> Regards
> David G
>
>
> ------------------------------------------------------------------------------
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
> _______________________________________________
> Ledger-smb-devel mailing list
> ..hidden..
> https://lists.sourceforge.net/lists/listinfo/ledger-smb-devel

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Ledger-smb-devel mailing list
..hidden..
https://lists.sourceforge.net/lists/listinfo/ledger-smb-devel