Re: Initial steps for 1.6, and some longer-term questions

As folks know here I have made several major attempts to rewrite the financial logic but it is a very big task and I never get as far as I would like. While 1.6 is already expected to have great improvements in multicurrency support, I am not expecting to do a lot on financial code at present.

While this may seem like a weird move given the mantra of the development team that the "old code" needs to be removed and the recent confirmation that the old code is still very fragile, the existing code has shown to be "mostly working" (for most people). So, I agree that we need to do some maintenance on other aspects of the code which may have degenerated since the project started, due to the focus of replacing code: in the areas you point out below, there's a lot of room for improvement. With the desire to replace old code, it seems like the replacement code wasn't always required to either implement replacements for *all* aspects of the original functionality. While that may be acceptable short term, I don't think that's a good idea longer term. With the release of LedgerSMB 1.3.0 now more than 5 years ago (see: http://ledgersmb.org/release/ledgersmb-130-released -- dated 2011-10-10!), I concur with Chris that time for consolidation has come, even for newer code. Time to iron out some long standing inconveniences and problems which may have been caused by a heavy thrust forward over the past 10 years of LedgerSMB and the past 5 years of 1.3.x and 1.4.x.

There are a few things architecturally however that I would hope to accomplish with 1.6:

1. From 1.2 through 1.4, ease of installation has not been a priority and has degraded We need to reverse that. Ease of installation and management needs to be a top priority. I think we have the back-end stuff down, so the question is front-end and administrator tooling.

I would say this is particularly true: we don't have a web tool which points out which libraries or other dependencies are missing from an installation. Tools like Zabbix have that, allowing an admin to quickly address the issue(s) and assess the level of support for any of the optional features/dependencies. This could seriously improve our "first user experience".

2. Contact and user management has become more feature-rich but harder to use. The code here has also become bit of a mess even though it is rewritten code. I would like to fix these problems too.

Right. Recent consulting support shows that our contact management -- especially the difference between the "entity" and the "eca" -- and the relation between contacts ("entities" and "ecas") and orders/invoices is hard to understand at first.

Personally, I actually completely understand this for multiple reasons (listing them here, because I think these should be on the list to be addressed as part of addressing any issues in "contact management"):

* The system hinges on two thoughts regarding entities and ecas: when selecting a customer an invoice, one has to enter the name of the *entity*

* The system clearly hinges on two thoughts regarding entities and ecas: when there are two ecas (both customers) within 1 entity, every "update" leads to the counterparty selection screen

* The fact that there's a "contact type" (customer/vendor/...) on the entity definition (company/person) screen, leads people to believe that the customer already has been created when they save the company/person (being presented with the ECA screen which lets them define and save the customer for real)

* The way there currently is no extremely obvious indication that addresses/bank accounts/notes/etc are being added to either the entity or the selected ECA, make the use of these items very hard to use

3 Workflow and usability issues need to be the top priority.

Agreed. I think this starts with defining what the various workflows should look like and how we can create screens which support those workflows. I'm thinking of presenting un-editable screens for documents (transactions, orders, ...) which have been saved and/or posted; an "Edit"(for saved documents) or "Copy to new"(for posted documents) button at the bottom should move the document into a state where it can be edited, presenting the user with a fully editable document. This will also help users understand the idea of "Save Info", because the only editable field in an invoice *after posting* will be the "internal notes" field. Completely obvious to what the "Save Info" button will apply, right?

Another one of my gripes is with usability and scheduling of transactions. I think we should present a listing of all scheduled transaction postings when scheduling a transaction. That way the user gets to validate his/her expectations regarding the planned postings. I for one *always* get this wrong...

On a technical side, I would like to switch to external tooling for administration and management. This means one gets the admin tools, and can install and manage LedgerSMB instances from there. It also means these tools need to play really well together.

I'm affraid this sounds pretty vague from just this sentence. I'm affraid people will have a hard time judging how this will impact them. Will we still deliver these tools in such a way that installation will be easy? Will these be part of the source distro so packagers for distributions automatically get the right sources to package and distribute? Could you elaborate a bit more on your plans in that respect?

Other than that, if the tools come with the same source tarball and are a huge improvement over what we have now, I think we only can cheer to that :-)

Could you spend a few lines detailing which problems we'll be solving with these new tools?

I would also like to have the contact and user interfaces primarily client-side _javascript_ driven by Dancer-based web services. This would make things much easier to manage the code.

Both that as well as the fact that having the services makes it much easier for tech savvy people to migrate to LedgerSMB or to interface LedgerSMB to other systems.

As we get into that process, I am hoping that we will be better equipped to discuss options for removing legacy code in an orderly manner.

It's my conception that the problem with the legacy code is that it's too interwoven in order to be disentangled without breaking everything else. My idea is that heavy browser-side widgets powered by webservices, offer the opportunity to modularize the code base, allowing decommissioning of the old codebase in a step-by-step approach. Modules to be factored out first, would be (imo):

- order/invoice/quotation customer/vendor lookup -- we should be able to implement this in a widget, eliminating the need for the intermediate "select customer" screen

- order/invoice/quotation part/service lookup -- we should be able to implement this in a widget in similar ways we do for customers/vendors

- rewrite the entire GL entry screen into a series of widgets and webservices -- although not a big part of the legacy code base, it's definitely a step to decommissioning code

- tax calculation widget (service based automatic calculation) -- eliminates the need for the "update" button, at least as far as tax calculation is required

- ship-to lookup/entry (popup) widget -- eliminates the complexity relating to the existing "ship to" intermediate screen

- e-mail (popup) widget -- for posted documents; eliminates the complexity in dealing with the current intermediate code paths jumping across modules

- widget to hold all invoice lines (and which knows how to add more) -- eliminates the need for the update button while composing the invoice

- widget to hold all the payment lines (and which knows how to add more) -- eliminanets the need for the update button while composing the invoice payments

While I do agree that this doesn't eliminate any of the problems in the database schema yet, it's definitely a way to start decommissioning the Perl code in steps. As of 1.5, we can add any regular or self-developed widget to our interface simply by adding a DIV tag and a data-dojo-type attribute. I don't expect that adding tags and attributes like that, replacing the *use* of other (generated) HTML tags and attributes, will be unmeasurably destabilizing. Note that I'm not saying that we immediately should eliminate the associated code as well, because *that* could be *very* destabilizing (as I found in the 1.4-mc work).

If the external tooling approach works well for administrative tools, we could opt to do this for financial logic. If it doesn't work so well, we will need to find another approach.

We have gone over this idea before and while I do see why you would want to try it, I don't really see how it reduces the size of the problem. Aren't you simply writing the code from scratch either way? [In your earlier attempts *and* in the attempt at externalizing the financial logic?] My thoughts are that in all cases it's a major effort that - in order to produce stable software and a manageable upgrade path for existing users - should be broken into smaller, well-understood steps that we can develop tests for in order to assess stability of the software.

Thanks for writing your mail! I hope for feedback from others too, so we can find a joint way forward with everybody involved in the project, hopefully even finding more contributors as we go!

Bye,

Erik.

http://efficito.com -- Hosted accounting and ERP.

Robust and Flexible. No vendor lock-in.