Re: Web Services API: URL naming proposal

[Chris Travers <..hidden..>]

I think John's points here raise some important questions I'd like to
raise here for further discussion:

On Sun, Nov 20, 2011 at 9:30 AM, John Locke <..hidden..> wrote:
> Hi, all,
>
> Been quite swamped with work recently, sorry for the delay.
>
> Having just designed a REST API for a client, have some fresh
> perspective. Below...
>
>
> I'm starting to like /api/ or /rest/ for the path that identifies the
> web service. And also adding an API version number to that endpoint, so
> that it becomes possible to change the API in the future while still
> maintaining an old version (should the organization wish to...)

That's not a bad idea.

>
> On authentication, yes we can use http auth headers, but do we want to
> explicitly require a session token, too? We're starting to delve into
> OAuth -- which adds a layer of complexity but also can take away the
> need for the remote system to collect the password at all. This seems
> like a good option to support.

A couple questions:
1)  For an API aimed at other applications, why have a session token?
What does it buy us?  In the main application we use a session token
to enforce a full round trip in some cases (XSRF prevention), and to
handle discretionary locking in long workflows (actually more
properly, timing out of such locks).  Neither of those general
requirements apply to a job from an OS Commerce installation which is
feeding sales orders into LedgerSMB.

2)  How does OAuth affect our ability to pass through credentials to
the database?  Or would a web services handle have to do its own
authentication?

3)  Does the added complexity make sense with general use cases?  I am
assuming we are primarily interested in a web services API for
server->server integration since a piece of software used primarily by
the end user would be more likely to just call the db-level API (which
would provide greater control over db transactions, and the like than
one would get from a web services interface)?

>>> 2. Since companies are separate databases, where do we put the name of
>>> the company in the URL? <prefix>/store.pl/<company>/<etc...>?
>>>
>> What do you think of the above proposal?
>
> I suggest we include the company in the body of the login, and then it's
> represented in the session cookie. If an external application needs to
> work with multiple companies, it can manage multiple session ids.

This is contrary to REST ideals, correct?  Not that departures from
that are necessarily out, but I would prefer to see some justification
for the added complexity of requiring state handling and cookies.

Also I suspect (though I will defer to others here) that debugging an
incorrect company name may be easier if that shows up in the url in
the access logs.

> So one thing is identifying supported formats for the data -- I suggest
> we support JSON, multi-part form (e.g. URL encoded like a regular form
> post) that returns HTML, and some relatively simple XML. Type can then
> be specified via "Accept" header, and also by adding a suffix to the
> URL. For example:
>
> http://myhost/ledgersmb/api/1.3/customer/224.json
> http://myhost/ledgersmb/api/1.3/customer/224.xml

But those formats are not all entirely equivalent are they?  I JSON
and XML are close and could be easily supported together, but they
allow nested data structures while form submissions are flat, right?
If we support form type submissions as a full API, then this means we
have to choose between added maintenance of two very different data
representations and forcing the xml and json to the least common
denominator, correct?  This becomes a bigger deal as time goes on and
more stored procedures expect some form of nesting in argument lists.

This being said, I like the use of extensions here, and I think the
overall idea is sound.  Now, if we are to do this, I would suggest we
go with a plugin model for file types, i.e. require a parser which
converts the incoming file into a Perl hashref, so that we can add
other file types if we ever have to.  That way if someone really needs
plain form submission handling we have an avenue to support that in
the future, even if the API might be more complex in order to handle
lit.

>
> POST/PUT type should get specified by Content-Type header.
>
> I suggest we start with the base entity structure that mostly maps to
> the database structure, then add "sugar" path shortcuts to make this
> easier to use. e.g. all of the below might map to the same item:
>
> http://myhost/ledgersmb/api/1.3/entity?eca=224
> http://myhost/ledgersmb/api/1.3/entity/eca/224
> http://myhost/ledgersmb/api/1.3/eca/224
> http://myhost/ledgersmb/api/1.3/entity/eca/customer/224
> http://myhost/ledgersmb/api/1.3/customer/224
> http://myhost/ledgersmb/api/1.3/customer?meta=557 (which might redirect
> to the actual eca id)

Ok, if you mean the structure of the db api then I would entirely
agree (I think the physical structure of the DB is beside the point).
So let's do this:
1)  A flat namespace for primary types/identifiers below the API base URL
2)  Open discussion for what other shortcuts should be available for
these, and whether they provide additional checks (I am assuming
customer would check the entity_class but eca would not?  Which entity
class would it check?  If grabbing from entity/...  should we require
an entity_id we can check?  That sort of thing)


>
>
> ... then add more "sugar" methods to get related items:
>
> http://myhost/ledgersmb/api/1.3/customer/224/invoice?poststartdate=2011-01-01&poststartoper=gte
>
> ... might return a collection of invoice objects for customer 224 with a
> post date greater than/equal to January 1, 2011.

These can map to db-based search routines, correct?

>
> http://myhost/ledgersmb/api/1.3/customer/224/invoice?status=open
>
> ... might return all open invoices for the customer.

That becomes syntactic sugar above the above?
>
> To create a new customer, you would POST customer data to
>
> http://myhost/ledgersmb/api/1.3/customer

Agreed.

>
> ... and it would come back with the ECA id set, and a Location: header
> set with the resource URL for that item.

 The ECA id would be in the location header, however, right?  I guess
what I am wondering is if we are going to return the ECA id as well,
shouldn't we return the whole object?  Or wouldn't it be better to
just issue a redirect to the new object so that default values can be
pulled?
>
>
> For the API I was just working on, one other key consideration was
> removing a lot of duplicate fields from the object -- the way Drupal
> builds objects (especially with the variety of modules we used on this
> project) resulted in the same data appearing multiple times. This is
> very confusing for non-Drupal developers to figure out, so we are
> creating simplified versions for each type of object that are much
> flatter and contain only a single copy of each field, wherever it comes
> from. It then merges data from PUT requests into the appropriate spot of
> objects to update before saving.
>
> So we should pay attention to the structure of the objects we're
> returning -- and if they are more complicated than necessary, for the
> sake of external developers I suggest we add a mapping layer to simplify.

Agreed.

Best Wishes,
Chris Travers