Re: Thoughts about Problem Releases

[Vladimir Botka <..hidden..>]

Dne Fri, 18 May 2007 22:42:57 -0700
"Chris Travers" <..hidden..> napsal(a):

> Hi all;
> 
> My ideas here are my own.  I do not speak for the core team in this
> but I think that it is important to share my ideas and thoughts with
> everyone and get some feedback.
> 
> 1.2.5 has been one of the more problematic releases in some time
> (since 1.2.0-1.2.2).  I wanted to share my thoughts with people here
> about what to expect in the future, what we are doing to minimize such
> problems as we move forward, and get ideas for ways to help get
> community involvement in QA.
> 
> As of the release of 1.2.5, current policy was to do a review for the
> most common (mostly packaging) issues to date.  The review took no
> less than two days, and could take longer.  1.2.5 was in and out of
> validation at least twice.  And several people had been beta testing
> it for far longer than the validation problem before the particular
> problems were discovered.
> 
> Given the history of 1.2.x and the lengthy feature-freeze that release
> unerwent, I personally do not think that testing alone will resolve
> this problem.  I think it is inherent in the codebase we inherited.  I
> also think that we are going to have occasional problem releases until
> we get all of the inherited code out of our codebase.  To give you an
> idea, 1.2.0 went through about a month of active development and
> nearly six months of post-feature-freeze testing and bugfixes.  No
> codebase should require that sort of overhead in QA.
> 
> A review of the SQL-Ledger changelog indicates that SQL-Ledger runs
> into the same problem from time to time.  The larger issue is that we
> tend to loudly announce and discuss when this sort of thing happens.
> 
> First, I would like to suggest that problem releases may occasionally
> occur through the development of 2.0 but that they should become less
> and less frequent both between major releases and in major trees.  Due
> to the code that has been inherited, this is something we are going to
> have to live with for a while longer.  The simple fact is:  this
> codebase is both disorganized and brittle and it is fundamentally
> impossible to fully predict every way a change could break things
> (which leaves us with Murphy's Law).  Testing alone is not going to
> get us out of these sort of problems.  In fact one of the reasons we
> are moving away from this codebase is that it scores pretty high on
> the unmaintainability sale.
> 
> Secondly, I would like to update you on the attempts we are making to
> minimize these problem releases and solicit specific contributions in
> helping make this happen.
> 
> 1)  Seneca has been working very hard to ensure that we get all new
> code in 1.3 covered by automated tests.  As time goes on, we expect
> full coverage of the application though this may not happen before
> 2.0.
> 
> 2)  There is a general concensus that it would be helpful to form a
> list of testers who would have access to -rc releases prior to general
> release.  We are also discussing the exact minimum hold on an -rc
> within a stable branch before it can be released (too long and we fix
> fewer bugs, too short and we increase the risk of introducing bugs).
> More contribution from users in this area would be of great help.
> 
> 3)  We are working hard to move ourselves away from the current
> codebase towards a structure less condusive of errors.  THis is
> probably the most important aspect of the solution.
> 

My suggestion is to fork CURRENT (lets start with 2.0 right now) branch
while keeping the STABLE (1.x) branch. The CURRENT branch will be for
the development of entirely new concepts and STABLE branch for
conservative changes which will not endanger the production of users. 
-vlado
-- 
Dr.Vladimir Botka
..hidden..