Re: Guile release planning

[Greg Troxel <gdt@ir.bbn.com>]

[-- Attachment #1: Type: text/plain, Size: 6555 bytes --]


"Neil Jerram" <neiljerram@googlemail.com> writes:

>> 2008/11/11 Ludovic Courtès <ludo@gnu.org>:
> But this isn't obvious to me.  _If_ the libtool versioning system
> works in practice, in the senses of
>
> - permitting linking when it ought to be permitted
>
> - failing linking when it ought to fail
>
> - providing a clear error message in the failure cases, so that the
> user knows what to do next

Except for 'clear error message', I think shared library linking (in ELF
at least) satisfies that.

> then it seems to me that a reasonable division of labour is for us
> (upstream) to take care of API compatibility, and ensuring that the
> libtool numbers are a correct description of ABI state, and for
> distributions/users to take care of any consequences of mismatched
> libtool numbers.

That is the standard model, and I think it's fine, modulo the frequency
of ABI breaks (and thus shlib major version increases).

ABI is not just shared library versions - it's all the scheme-level
interfaces that a guile-using program might call.

> I think the latter "consequences" are just having to recompile
> something against the new libguile version.  For a user who has
> already decided to use the source code version of an application, that
> should be trivial; for distributions, that's just what they do all the
> time, isn't it?

Yes, except that it's a pain if it happens often, and it more or less
constrains all the guile-using components in a system that aren't leaves
to use the same guile version.

> Are there other consequences that I'm missing?

Frequent API and ABI breaks are in my view a sign of an immature
development process, and a warning not to rely on something.  guile aims
to be a scripting language to be added to various programs, so we should
hav as a major goal being stable enough to be relied on.

> Sure, we could also take on ABI compatibility ourselves, but I think
> that overconstrains our development, and/or makes it harder.

I think that we should aim to have infrequent ABI breaks.  Every year or
so is probably ok.

>> I think enhancements like the lazy symbol binding in modules
>> could have been in theory added in 1.8.x since it breaks neither the API
>> nor the ABI.
>
> Agreed; and I think they still can be added in 1.8.x.

Sure - this is totally fine.  Programs that worked against 1.8.x, both
compile time and run time, will still work against 1.8.x+1.  But, the
reason we want to add to 1.8 is that 1.10 wasn't release 18 months ago,
about which I'll rant more later.

>> A GC change, or a rewrite of the string/char code with
>> Unicode support would be a big jump, too.  Such big jumps surely need a
>> new major release.
>
> Not necessarily, in my view.  We have an extensive test suite, and I
> think we can have some confidence in that.  After sufficient testing,
> I would see no problem with your proposed BDW-GC changes going into a
> 1.8.x release.
>
> Same for Unicode support - if the API stayed compatible.  If the API
> could not be compatible, then I would agree that we might need a new
> major release.

The biggest issue is API breaks without adequate warning.  To remove
something from the API, it should be marked deprecated in a stable
release, and then have probably 2 years pass during which all depending
programs to update and release.  The key is that one has to be able to
say in good conscience that those depending programs are lame or
unmaintained if they didn't update.

If API breaks other than via the above rules happen too often, then
guile really should support (natively) the installation of multiple
vesrions at once.  pkgsrc has "guile" which is 1.8.x and "guile16", and
guile16 installs to /usr/pkg/guile/1.6 as prefix instead of /usr/pkg.
guile simply doesn't have the mindshare to support this kind of
complexity.

An example of API problems is Linux itself.  I am working on a project
using the kernel and going from 2.6.20 to 2.6.24 is a huge change that
seems to have significant API breaks (for kernel modules).  If these
changes are as bad as they seem, this just isn't adult software
engineering and I would have to recommend against relying on such code.


I think guile's real problem has been that while people hack on head,
this hacking doesn't seem to have the clear goal of producing a stable
release.  Of course all the hard-core guile hackers run guile from
(choose-VCS-weekly :-), but programs that depend on guile should not --
they need to depend on versions which are packaged.  So my general rule
for software is that as soon as people say "you should run HEAD instead
of RELEASE, it's better", a release from head is overdue.

So I don't think there's anything wrong with the
hack-on-head-cut-stable-branches-periodically approach, except:

  1) try really hard to avoid API breaks, and take the ugliness of new
  function names with the new semantics/signatures as a lesser pain

  2) release a new stable version from head every year or so.

For 2, I suggest calling feature freeze and asking for testing of head,
kicking out 1.9.x releases and encouraging fast-moving distributions to
package them as a replacement for 1.8.x so packaging system maintainer
types can test, and then branching for 1.10.0 and unfreezing head.  Even
if there were 3 months a year of feature freeze and 9 months of open
hacking, I think that would be good.  There's a negative feedback cycle,
where people don't want to freeze for release because they want to get
in one more thing before the stable code is frozen for the next 4 years,
and as the stable frequency gets lower the urge to not freeze gets
higher, leading to longer intervals.  NetBSD has been struggling with
this too.

Here's the history

1.4	2000-06
1.6     2002-09
1.8     2006-02
1.10    >> 2008-11 :-(

and these intervals are all significantly too long.  Probably a freeze
is in order 9 months after the last branch cut, to head for 1 year.


Speaking from my experience being the maintainer in pkgsrc of guile, new
versions that don't have API or ABI breaks are nearly trivial to update,
and cause no issues.  ABI breaks but not API breaks are not as bad, but
require touching all the depending packages to ensure cross-package
binary compatibility.  API breaks without all the depending packages
getting fixed leads to choosing between dropping depending packages or
having multiple versions of guile in pkgsrc, both of which are ugly.


[-- Attachment #2: Type: application/pgp-signature, Size: 193 bytes --]