Re: [ledgersmb-devel] RFC: Formalize templates interface and split out onto CPAN

On Tue, Aug 29, 2017 at 8:03 AM, David G <..hidden..> wrote:

Hi Chris,

On 27/08/17 19:37, Chris Travers wrote:

Hi;

I have been thinking of what else we can/should split out sometime soon to CPAN and it has occurred to me that the interface of the templates could probably be split out (and maybe the mailer too).

To be honest, I don't think we should be splitting either of these out. Especially at this stage.
Neither of them is a significant quantity of code, and what code there is, is specific to the way we use them.

Both of these however contain LedgerSMB-specific assumptions and would need to be rewritten to some extent to address these. However with the templates in particular, removing these assumptions would help us get rid of a lot of the CGI assumptions we are currently making. In particular, I am thinking of the automatic output options which I think should be abstracted away or even removed.

I believe that most, if not all CGI assumptions have already been removed, and if they haven't should be fairly easily removed once identified.

A major justification for this is that reporting endpoints for my LedgerSMB API gateway are much harder to implement across a Dancer interface than they should be, because we assume we are operating in a CGI environment.

Can you please show some examples of the CGI assumptions that we are still making.
I know there are some still present in the Legacy code, but that's been wrapped at a higher level so it doesn't impact the rest of the codebase.

Look at the _http_output function in LedgerSMB::Template in current master. We print HTTP headers to standard output. We implicitly call that when we render a report which means, for example, that no third party application can use our reporting framework as a library unless it wants to parse out the HTTP headers.

Our overall assumption in internal LedgerSMB code is that we have a CGI app and are wrapping that in Plack.

I would like to do this for the following reasons:

1. I would really like to be able to use the reporting generation in areas outside of a CGI web application, and

2. It would be nice to be able to open up additional formats to third party developers saying "Put it on CPAN and tell people they can install it like so."

I really don't believe it is a sane thing to "hack" in any sort of api/plugin layer right now.
If you have followed the discussion on #ledgersmb you will have noticed that there has been a lot of general discussion (and some on the dev list as well) about designing an API.

I STRONGLY believe that we need to DESIGN the API before ANY code is written.

I guess I am proposing the output handlers should be a component with an API built for that, not an API for LedgerSMB as a system. We have a design for plugins for db types in PGObject which is purpose-build for that and works well.

To do otherwise is counterproductive, and a waste of everyone's time.
You only have to look at the current situation where we have a partial implementation of a perl API, some completely disparate http api like functions, and at least two "API subprojects/libraries"
All of the above are only a tiny portion of what is needed, none of them have even remotely similar interfaces, and as of right now, I don't believe any of them are remotely maintainable in the long term.
They certainly aren't useful beyond the original intent they were created against.

To work towards being ready to start coding for an API we need to....
- Design the API (a lot has been done towards this, but there is a lot more to be done. Many of the notes are in a wiki document on our github repo)
- Document the API endpoints in an industry standard way. (I'm partial to swagger, but it's not the only option)
- Decide on the implementation framework and other technologies to be used for implementing the API (Dancer may be specified here, but there definitely other options)
- Finally get public comment on the design.

ONLY once we have covered all of the above, should we be writing ANY API code.
We don't need to fully design every endpoint first, but we do need to define the API structure, and sufficient endpoints to make some example functionality viable and testable.
Additional endpoints can then be defined following that structure.

I guess there seems to be a philosophical difference here.

I think we are better off with a small responsibility, a small plugin API, and a small direct dependency tree at the expense of a larger extended one than we are trying to take ownership of all possible things ourselves. I think if we focus on the small pieces, the big issues will take care of themselves and so the smaller we can make LedgerSMB *as a system* the more maintainable it will be.

At any rate, I have a need to fork the reporting structure into a different project so my thinking is probably that I will put whatever I come up with in that regard on CPAN and we can decide what to do with it.

I would like to suggest that over the next major version or two I work on refactoring out or abstracting away the LedgerSMB-specific and CGI-specific assumptions to the templating interface, provide a dynamic format registration mechanism, and the like. And then work towards breaking these off once this is complete.

I wouldn't like to see that happen right now, as most of the CGI specific code should only be within the legacy codebase, and we have a declared freeze on the legacy codebase for general changes.

No, it isn't. It is also in our framework. We have just abstracted it away from a lot of new code but it is still there behind the scenes. The entirety of LedgerSMB runs as CGI -> PSGI and that includes *all* of our current code.

Also, I don't think we should make any changes to the templating / reporting mechanism until we have a properly designed API in place, as the correct way to implement any changes here would be to do it right in the API, then implement the client side as an API consumer.

But that means you are suggesting that *all* use of these modules goes over HTTP to LedgerSMB as a system, right? I am suggesting maybe we don't want that and we should aggressively argue that LedgerSMB components should be usable as libraries.

Are there any thoughts on this?

In short, my thoughts are, don't do it until we have a properly designed API
--

Best Wishes,
Chris Travers

Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor lock-in.

http://www.efficito.com/learn_more
_______________________________________________
devel mailing list
..hidden..
https://lists.ledgersmb.org/mailman/listinfo/devel
_______________________________________________
devel mailing list
..hidden..
https://lists.ledgersmb.org/mailman/listinfo/devel

Best Wishes,

Chris Travers

Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor lock-in.

http://www.efficito.com/learn_more