Re: RFC: Proposed Framework Change for 1.4

[John Locke <..hidden..>]

HI, Chris,

I haven't had a chance to try out much of 1.3 since the recent
milestones--I look forward to checking out the progress and see how it's
going! And also to contributing more sometime soon. Some of the changes
below would make it easier for me to contribute, others harder...



-------- Original Message  --------
Subject: [Ledger-smb-devel] RFC: Proposed Framework Change for 1.4
From: Chris Travers <..hidden..>
To: Development discussion for LedgerSMB
<..hidden..>
Date: Mon 20 Jul 2009 02:00:08 PM PDT
> Hi all;
>
> While the 1.3 framework is a great improvement on the 1.2 framework, I
> have noticed a number of things that I think should be changed, as
> well as a few new opportunities to simplify things in 1.4.
>
> First, I would like to propose that 1.4 be targetted at PostgreSQL 8.4
> or higher, and require whatever DBD::Pg modules are current at the
> time when 1.3 is released (i.e. the time when we begin serious work on
> 1.4).  LedgerSMB 1.3 would remain supported for a number of years
> after release (at least 3) so this shouldn't cause any real headaches
> in terms of forcing folks to upgrade.....  8.4 would also allow us to
> do away with the dependency on tablefunc for the menus and move to
> WITH RECURSIVE instead, and it would allow a number of other things
> that could be used to better enforce data integrity between the app
> and the db.
>
This is probably the biggest reason I haven't yet extensively tested out
1.3 -- because my production servers running postgres have been 8.1, and
we try to keep our development environments as close to possible to
production.

We have now moved over to a server with 8.3, just last week in fact. So
I expect to be able to start testing 1.3 shortly.

Adding a limitation of 8.4 would mean I'd need to set up an entirely new
environment to do anything with 1.4. Since we try to keep our dev
servers somewhat synchronized with production, it's hard for me to keep
up with bleeding-edge database releases.

So I'd prefer seeing backwards compatibility for the database supporting
8.3 or newer. 8.4 is probably fine for a full 2.0 release, but it's a
hardship for now--I don't even see any packages available for current
Ubuntu systems, let alone Long Term Support. And I don't see any Debian
packages either. Given that we just upgraded our production
environments, it's probably a couple years before we'd want to do that
again and would even be capable of running 1.4, if it depended on
Postgres 8.4.

> Anyway, here are my other areas I would like to see addressed:
> 1)  The current system presents hazards of customization.  One option
> ot address these is to implement a before/instead/after structure and
> move UI template rendering out of the workflow script.
>
Yes! The current scripts seem awkward. I'd like to see more separation,
and an easy way to switch between loading a template and loading an
equivalent view of an object, either as JSON or XML (or other formats,
too). I'd like to see a template system that is loaded after the main
request is processed, that has objects that expose methods for loading
additional data.

In general, looking for more standard ways of calling LSMB from other
programs, and more ability to customize templates in some way that I can
keep those changes outside core. Haven't done a lot of this to know
whether the template system already does what I'm looking for--it
probably does, from the last time I looked at it, but I was having to
add my own data services...

> 2)  Workflow scripts should be moved to LedgerSMB/Scripts/
>
doesn't matter to me

> 3)  Dialog:  Do we want to continue having top-level scripts past
> 2.0?  I know this has been a point of criticism in the past.  However,
> it is something that will probably have to be changed around 2.0. 
> What do folks think?  What should be the ideal call interface?  Maybe
> http://host/ledger-smb/request.pl?module=journal or something like that?

I'd vote for a single call interface, over multiple top-level scripts,
with parameters as you've shown. I've found having a module, action, and
view parameter for every call, with defaults, to be quite effective.
It's very easy to debug/enhance web service calls when you can switch
between an html template, a json object, or an xml file with a simple
parameter change.

I've also found some of the REST conventions of using http methods for
DELETE and PUT in addition to GET and POST to be very useful.
>
> 4)  Our object structure in 1.3 is better than that in 1.2 but it
> still has some serious shortcomings.  The biggest one is that the
> modules are still grouped in functional units.  This means that object
> properties are not entirely consistent across uses of the same object,
> which makes documentation and tracing a bit difficult.  Fixing this
> will require refactoring.  I would like to have this refactored into a
> properly documented and designed object oriented framework.

Sounds great. I hope to be able to assist with such a refactoring project...
>
> 5)  I would like to revisit a few elements of the database interface
> calls as well.  In particular, the latest DBD::Pg and PostgreSQL allow
> us to easily make use of more complex db-side datatypes.  If we go
> this route, we should be able to define our data structures in the db,
> as complex types, and build database requests out of it.  We could
> also allow primary object attributes to be defined this way, while
> secondary arguments could be defined in a way to allow explicit
> function calls.

I'd like to see this level of functionality available through http and
in whatever templating system is in use. I don't really care whether
these are implemented in Perl or in Postgres -- though as I already
mentioned, requiring the latest Postgres does put a barrier up for my
participation...


>
> What do folks think about these proposals?  Anything else that needs
> to be added?  Anything that should be up for discussion?

Overall, sounds like a good direction to me! But I think you should
reconsider the database version requirements.

It's one thing to be a showcase application for cutting edge Postgres
releases. But this is an accounting application we're talking about--we
shouldn't be requiring server software that isn't anywhere near any
production distribution releases--you're greatly limiting the potential
audience of users, as well as possible contributors. I think it's fine
to say new development branches require at least as new a Postgres
version as is available in the most current long-term support release of
the major enterprise distributions -- Red Hat, SuSE, Debian, and yes I
do include Ubuntu -- but newer than that is going to really limit
participation. In my opinion, anyway.

Cheers,
-- 
John Locke
http://freelock.com