An Introduction to SODA in LedgerSMB

["Chris Travers" <..hidden..>]

With the added participation in the development of LedgerSMB, there is
a greater need for understanding of our architecture and anticipated
code structure.  Furthermore, some of the structures have evolved in
the process of development, and additional clarification is necessary.

This paper covers the use of Service Oriented Database Architecture
(SODA) in order to help developers understand the methodology and can
use it effectively.

Introducing SODA:
SODA is one attempt to solve the conceptual mismatches between Object
Oriented programming and good relational db design.  Among other
things, SODA is an approach to relational database design which is
specifically designed to make the database play well with object
oriented programs.  We do this by using the same principles behind
service oriented architectures to create interfaces which can be
semantically integrated into other programs quickly and easily, thus
reducing the amount of application code which has to go into database
access.  The service oriented principles used in SODA are rich
semantic interfaces and automatic discovery.  They are no different
than the principles behind SOAP, WSDL, and so forth.  The idea is that
one ought to be able to write code generators and integration classes
which can build object oriented classes out of these discoverable
interfaces.

Major benefits of SODA:
Unlike traditional approaches such as ORMs, SODA allows for a very
loose coupling between the program data structures and the physical
layer of the database design.  This allows us to design the database
using the semantic structure of the data only, and then provide a
loose interface glue on top of this to generate application-specific
or application-independent data structures.  A functional (rather than
relational) interface allows for mapping of object methods rather than
data structures per se.  This also provides greater flexibility in
terms of mapping object behavior to database actions.

A second major benefit of SODA is that business policy logic and
accounting data logic can be encapsulated in the db, ensuring that
they are applied consistently across a business.  This is a similar
benefit of web service API's but in this case, they are available even
when hitting the database directly.  This allows for light-weight (or
even the possibility of automatically generated) code to create
applications which match business tasks.

In short this allows us to create better databases and expose business
policy and accounting rules more consistently across an organization.
This allows LedgerSMB to better become a part of a business
infrastructure than we could through restricting semantic integration
to web services.

How SODA works:
SODA utilizes the general capabilities found in PostgreSQL to create
stored procedures where objects could either dynamically (or through
code generators) look up related functions to execute in the database,
or where objects can execute stored procedures that they merely know
the name of where the mapping of arguments to object properties can be
done automatically through the discovery mechanism.

When exec_method is called, an object looks up the named stored
procedure and the name and of its arguments.  It then strips out the
in_ prefix (added to prevent conflicts with table names) and matches
these by name to object properties.  The stored procedure is then
called with the arguments which were discovered.  This behavior is
common to all objects in LedgerSMB which inherit the base SODA class
(DBObject).

I think we want to suggest that the reliance on AUTOLOAD for 1.3
should go away and be replaced by the usage of stored procedures of
known names.

Required Practices:
Function overloading cannot be safely supported because it introduces
semantic ambiguity into the system.  Therefore all function names must
be unique.

Furthermore the current naming conventions must be followed for all
arguments:  in_ must be prepended to the name of all arguments.  This
is there to prevent name collision between columns and arguments.
Similarly columns must never begin with in_.

A third restriction i that all arguments which are logically the same
and have the same name must have the same type.

Recommended Practices:
In general, since this approach is fairly new, it has taken the core
team a while to develop best practices.  Even this document is
expected to generate some (possibly heated) discussion on the matter.

1:  The database should be designed independently of the application
at first.  All fields in the database should be semantically atomic
and should be broken out as much as possible.  Additionally, we should
be searching for additional opportunities to further normalize the
database to either BCNF or 5NF.
2:  SODA functions should generally be designed to expose necessary
data and policy logic to the application.  This does not mean that all
object logic should be thrown in stored procedures.  SODA can then
provide a logical interface to the database,
3)  SODA pseudoclasses can be created using custom types and function
names.  Function names should have the following format:
lowercase(class_name) . __ . lowercase(method_name).  Note that the
class and method names are separated by a double underscore to remove
semantic ambiguity.

Proposed Future Enhancements longer-term:
Long-run I would like to see more effort made at rich data structures
as discoverable entities.  This probably means developing conventions
which allow for custom types which include, as attributes, arrays of
other custom types.  These issues pose substantial technical changes,
and cannot happen anyway until we are ready to drop support for
PostgreSQL 8.1.  However, it is an area I would suggest we start
thinking of now.  On one hand it does add considerable complexity to
the discovery code, but it also provides for more consistency, an the
discovery code only needs to be written once anyway.  This might also
solve the function overload problem because classes could be mapped to
custom types and hence an API much more like a standard OO environment
could be exposed.

Of course, I expect that the above changes will not be practical for
at least a year and we have a lot that needs to be done in that time
anyway.

Things to Dream About:
SAP has begun a project on policy-oriented management.  The idea is
that managers should be able to design business processes and then
these could be automatically  mapped into sets of web service calls,
allowing non-IT people to generate workflow scripts for in-depth
business processes while at the same time ensuring consistent
application of policies across the enterprise.  Longer-run, it may be
possible to create workflow engines which allow non-technical people
to do exactly this-- to create business processes and have our
application create the actual workflow scripts.  Obviously this is not
likely to happen until SODA is better developed and our application is
fully moved to the new framework.  If this sort of thing is important
for SAP's customers, it is even more important for us.

Where to look in the current code:
To see these processes illustrated, please see the following files in /trunk:
sql/modules/Menu.sql for the interface that builds the menu.  The
interface to the actual code is in LedgerSMB/DBObject/Menu.pm
The sql/modules/Payment.pm and LedgerSMB/DBObject/Payment.pm are
another good set of modules to review.

The core SODA discovery logic is in LedgerSMB/DBObject.pm

Best Wishes,
Chris Travers