Re: Design (more) API; Problems writing tests, global state & Perl "API" requires database knowledge

Hi Erik,

Sorry for the late response, only just catching up on email post trip.

On 08/07/16 15:50, Erik Huelsmann wrote:

Hi,

As you all know, some time ago, I started implementing BDD tests. When I started that work, I had hoped to use Perl APIs from the test scripts -- as we have been developing nice Moose objects for wrapping application functionality (defined in the database most of the time). While implementing the most simple one of all tests (clicking on the menu link and checking a page pops up), I already ran into a number of problems.

A generic step that I've implemented on is:

Given qr/a standard test company/, sub { };

This step requires a few things:

1. Creation of a new database

2. Loading a schema

3. Loading templates

4. Creating an initial user

The fourth step is a problem (the others are encapsulated well in LedgerSMB::Database). This is only 1 occurrence of a much broader problem: our Moose objects basically expect to be instantiated within the context of our web application, including the existence of an HTTP request:

my $user = LedgerSMB::Entity::User->new(%$request);

Even though that works while we're in the web application, when I want to use the Moose objects as a scripting API (as I do for the testing framework), this doesn't work for a number of reasons:

1. There *is* no request object

2. The request object passes "internal values" into each object (i.e. _DBH; maybe others?)

3. The script doesn't have "global state" as normally stored in App_State

I'm thinking that having an API with the following properties would greatly help my coding for the test scripts (and possibly others to code against our company databases?):

Sounds like a good and sane idea to me. Perhaps it will even allow us to abstract and encapsulate things better for the web UI as well.

Proposed API
--------------------

Entry point
---------------

The entire API would be keyed off a LedgerSMB::API::Connection (or other) object, which encapsulates all state specific to one user (normal or super-user) being connected to a company. Such an object will need to be instantiated for every web request in the same way that we currently instantiate a LedgerSMB object, but it would be ignorant of whether it's being instantiated from a web-request or from a Perl script with the correct credentials.

Everything in the company (if access rules allow it) is accessible through methods from this class either directly or indirectly. Methods I envision this class to have:

- users: access to (non-super-users) with access to the

- roles: access to roles and/or role groups available in this company

- contacts: access to the "entity" subsystem (vendors/customers/employees)

(I have thought of many different names for the "Connection" class, all of which I found confusing:

- Connection -- we already have a "database connection" (DBI object)

- Session -- we already have a "User/Browser logged in session"

- Company -- we already have a "Company as a contact" object
)

In the case of this method I'd go for one of
* API::instance
* API::connection
With a strong preference for API::instance

New entities
-----------------

My idea is that if I want to create a new user - which inherently needs database access -

I can simply instantiate a new user object and "create that into the company" like this:

my $new_user = LedgerSMB::API::User->new(username=>'test');

$connection->users->create($new_user);

That is, the user is "created into the company". At that point, any internal administration can be "fixed up" on the user object before the actual database entity is created, including, but not limited to, passing a database connection (DBI) object.

Existing entities
-----------------------

Some existing entities (e.g. roles) are currently directly queried on the database. These should become accessible through the collection accessors introduced at the beginning.

Entity cross-references
--------------------------------

Wondering what to do about entity cross references here: I think API users would expect to be able to request a customer object from an invoice object. In Weasel, I solved this problem by maintaining a reference to the central Session, which allows objects which are related to that session to query attributes of other entities in the session. A similar pattern would probably work for the company as well.

Even though we're not even close to a model like this, I think it's good to have a target model planned so we can start migrating our API to such a model.

Comments? Remarks? Better ideas?

As a rough outline, it looks fine to me.

One comment though, for ease of coding and maintainability (ie: developer understanding with minimal relearning) I think we should try to make the WEB API and Perl API consistent in endpoint naming, and where possibly try and keep the usage the same as well.

I'm also wondering if we perhaps should define an alternative data input / output format that can be used by external tools.
Perhaps JSON, although I'm not convinced that is necessarily the best option, it's just a very common one these days with good tools available for manipulating it in most languages including shell scripts by virtue of `jq`

Regards
David G