Guile foreign object interface

Guile foreign object interface

[Mike Gran]

Hi All-
I wanted to make a quick post about the foreign object interface.
This is a bit of a placeholder because I haven't had time to
investigate the interface properly, yet. But I intend to poke at
it soon.
But for there record, there are some problematic design patterns
that I want to make sure can be covered by the new interface.
Again, I'll try to make proper examples soon, but, with Andy wanting
to release soon, I want to get this down now.

1. First off is a Lilypond-like pattern.  A C++ vector
is used to dynamically add or remove elements.  The memory
management of those elements lives in the C++ world.  Those elements
are boxed up as SCM.  GCing the SCM should not free its boxed
contents, but, deleting the boxed contents from the C++ side
should render those SCMs invalid in some sense. 
2. Mutually owned information. Two structures (A and B) both
mutually contain a non-GC-malloc'd structure C. C must exist
so long as one of A or B exist.
3. Here's one from guile-ncurses. An element ITEM is created with
some contents. Default memory management suffices: if ITEM is GC'd,
its contents are GC'd.  But the contents of ITEM can later be
attached to a COLLECTION. If ITEM is GC'd when attached to
COLLECTION, its contents are not GC'd.
5. Gobject-like subclassing.  In Gtk, an application window is
a type of a window which is a type of a widget which is a type of
object. If we were to resurrect an effort on GTK3 binding,
how would it work for foreign objects?
Again, forgive this random dump. Wanted to get something down now,
but I'll try to make a more sensible post later.
-Mike Gran

Re: Guile foreign object interface

[Andy Wingo]

Hi :)

On Thu 02 Mar 2017 16:47, Mike Gran <spk121@yahoo.com> writes:

> I wanted to make a quick post about the foreign object interface.
> This is a bit of a placeholder because I haven't had time to
> investigate the interface properly, yet. But I intend to poke at
> it soon.
> But for there record, there are some problematic design patterns
> that I want to make sure can be covered by the new interface.

I am not sure that all cases need to be covered by the new interface to
be able to include it.  I think it would be sufficient to simply replace
SMOBs, unrecommend mark procedures and all that, and get the benefits I
mentioned in my mail.  However that said...

> 1. First off is a Lilypond-like pattern.  A C++ vector
> is used to dynamically add or remove elements.  The memory
> management of those elements lives in the C++ world.  Those elements
> are boxed up as SCM.  GCing the SCM should not free its boxed
> contents, but, deleting the boxed contents from the C++ side
> should render those SCMs invalid in some sense. 

Not really helped by foreign objects AFAIU.

> 2. Mutually owned information. Two structures (A and B) both
> mutually contain a non-GC-malloc'd structure C. C must exist
> so long as one of A or B exist.

Likewise, if I understand A and B to be scheme objects of some kind; the
pattern here would be a gc-managed wrapper around C with a finalizer,
and A and B both use the wrapper.

> 3. Here's one from guile-ncurses. An element ITEM is created with
> some contents. Default memory management suffices: if ITEM is GC'd,
> its contents are GC'd.  But the contents of ITEM can later be
> attached to a COLLECTION. If ITEM is GC'd when attached to
> COLLECTION, its contents are not GC'd.

I understand you mean that ITEM has the only reference on its contents,
and that if it becomes unreachable, the contents also become
unreachable, and may be collected (and possibly finalized if they have a
finalizer and finalizers have a chance to run and all the other
finalizer caveats).  Sure, foreign objects work here, as do any other
Scheme data type, right?  As long as the ITEM finalizer doesn't finalize
its contents.  Each contained object needs its own finalizer I think, if
finalization is necessary.

> 5. Gobject-like subclassing.  In Gtk, an application window is
> a type of a window which is a type of a widget which is a type of
> object. If we were to resurrect an effort on GTK3 binding,
> how would it work for foreign objects?

This is very tricky; there are lots of considerations here.  A new
guile-gobject might well want its own scheme.  However, yes, foreign
objects are subclassable.

Cheers,

Andy

Re: Guile foreign object interface

[Andy Wingo]

Hi :)

On Thu 02 Mar 2017 16:47, Mike Gran <spk121@yahoo.com> writes:

> I wanted to make a quick post about the foreign object interface.
> This is a bit of a placeholder because I haven't had time to
> investigate the interface properly, yet. But I intend to poke at
> it soon.
> But for there record, there are some problematic design patterns
> that I want to make sure can be covered by the new interface.

I am not sure that all cases need to be covered by the new interface to
be able to include it.  I think it would be sufficient to simply replace
SMOBs, unrecommend mark procedures and all that, and get the benefits I
mentioned in my mail.  However that said...

> 1. First off is a Lilypond-like pattern.  A C++ vector
> is used to dynamically add or remove elements.  The memory
> management of those elements lives in the C++ world.  Those elements
> are boxed up as SCM.  GCing the SCM should not free its boxed
> contents, but, deleting the boxed contents from the C++ side
> should render those SCMs invalid in some sense. 

Not really helped by foreign objects AFAIU.

> 2. Mutually owned information. Two structures (A and B) both
> mutually contain a non-GC-malloc'd structure C. C must exist
> so long as one of A or B exist.

Likewise, if I understand A and B to be scheme objects of some kind; the
pattern here would be a gc-managed wrapper around C with a finalizer,
and A and B both use the wrapper.

> 3. Here's one from guile-ncurses. An element ITEM is created with
> some contents. Default memory management suffices: if ITEM is GC'd,
> its contents are GC'd.  But the contents of ITEM can later be
> attached to a COLLECTION. If ITEM is GC'd when attached to
> COLLECTION, its contents are not GC'd.

I understand you mean that ITEM has the only reference on its contents,
and that if it becomes unreachable, the contents also become
unreachable, and may be collected (and possibly finalized if they have a
finalizer and finalizers have a chance to run and all the other
finalizer caveats).  Sure, foreign objects work here, as do any other
Scheme data type, right?  As long as the ITEM finalizer doesn't finalize
its contents.  Each contained object needs its own finalizer I think, if
finalization is necessary.

> 5. Gobject-like subclassing.  In Gtk, an application window is
> a type of a window which is a type of a widget which is a type of
> object. If we were to resurrect an effort on GTK3 binding,
> how would it work for foreign objects?

This is very tricky; there are lots of considerations here.  A new
guile-gobject might well want its own scheme.  However, yes, foreign
objects are subclassable.

Cheers,

Andy

Re: Guile foreign object interface

[Ludovic Courtès]

Hello!

I think Mark made two kinds of comments back then:

  1. There were suggestions about the API itself, nothing deep:
     <https://lists.gnu.org/archive/html/guile-devel/2014-04/msg00070.html>.
     Andy, do you know if they’ve been addressed?

  2. A general concern about “API churn” and our ability to support the
     SMOB API in the future, which I can sympathize with:
     <https://lists.gnu.org/archive/html/guile-devel/2015-11/msg00005.html>.

     Essentially, (1) it should be easy for people to migrate from SMOBs
     to foreign objects (I think it’s largely a matter of
     documentation), and (2) existing code that uses the documented SMOB
     API should just work in 2.2, possibly with a deprecation warning.

It may be that we’re already mostly there (?).  If we get these two
things right, then I have nothing against adding foreign objects in 2.2.

Mark?

Thanks,
Ludo’.

Re: Guile foreign object interface

[Andy Wingo]

Hi :)

Thanks for taking the time to look at the issue!

On Tue 07 Mar 2017 14:44, ludo@gnu.org (Ludovic Courtès) writes:

> I think Mark made two kinds of comments back then:
>
>   1. There were suggestions about the API itself, nothing deep:
>      <https://lists.gnu.org/archive/html/guile-devel/2014-04/msg00070.html>.
>      Andy, do you know if they’ve been addressed?

There were a couple points I think:

In

  SCM scm_make_foreign_object_type (SCM name, SCM slot_names,
                                    scm_t_struct_finalize finalizer);

Mark suggested it should be "scm_t_struct_finalizer".  I.e. with an r.
I can agree but scm_t_struct_finalize is a type that's already in 2.0;
it's just being re-used here.  That said we can do the new name +
typedef + eventually deprecate the old name dance.

He also suggested to get rid of some type punning, and indeed now we
have:

  void* scm_foreign_object_ref (SCM obj, size_t n);
  void scm_foreign_object_set_x (SCM obj, size_t n, void *val);

  scm_t_bits scm_foreign_object_unsigned_ref (SCM obj, size_t n);
  void scm_foreign_object_unsigned_set_x (SCM obj, size_t n,
                                          scm_t_bits val);

  scm_t_signed_bits scm_foreign_object_signed_ref (SCM obj, size_t n);
  void scm_foreign_object_signed_set_x (SCM obj, size_t n,
                                        scm_t_signed_bits val);

I think that was it from an API POV.

>   2. A general concern about “API churn” and our ability to support the
>      SMOB API in the future, which I can sympathize with:
>      <https://lists.gnu.org/archive/html/guile-devel/2015-11/msg00005.html>.
>
>      Essentially, (1) it should be easy for people to migrate from SMOBs
>      to foreign objects (I think it’s largely a matter of
>      documentation), and (2) existing code that uses the documented SMOB
>      API should just work in 2.2, possibly with a deprecation warning.

I am not so sure about about this one.  I think it's not accurate to
characterize beginning to replace a 25-year-old C API (SMOBs) as
"churn".  I will get to Mark's point specifically in a minute but
regarding your points I believe that (1) is somewhat under-documented;
the documentation is more oriented towards new users than migrating
users; we can improve here; and (2) should work fine (the old API is
still there, not even any deprecation warnings currently).

I think Mark's desire (if I understand it) was for the new API to
completely replace the old in all forms, specifically in support for
mark procedures.  I really think that (a) we should not support the SMOB
mark procedure interface, as it's both horrible and insufficient for a
moving collector, and (b) that until we have a moving collector (?), we
shouldn't attempt to speculatively standardize anything in this regard.
Basically the arguments from here:

  https://lists.gnu.org/archive/html/guile-devel/2015-11/msg00003.html

There's still the SMOB API if you need mark procedures, basically, but
hopefully you don't (e.g. when porting GDB to 2.0 I was able to remove
them; they're less necessary in 2.x than they were in 1.8).  Some users
still need them, which is why SMOBs are still around :)

Andy

Re: Guile foreign object interface

[Ludovic Courtès]

Howdy!

Andy Wingo <wingo@pobox.com> skribis:

> On Tue 07 Mar 2017 14:44, ludo@gnu.org (Ludovic Courtès) writes:
>
>> I think Mark made two kinds of comments back then:
>>
>>   1. There were suggestions about the API itself, nothing deep:
>>      <https://lists.gnu.org/archive/html/guile-devel/2014-04/msg00070.html>.
>>      Andy, do you know if they’ve been addressed?
>
> There were a couple points I think:
>
> In
>
>   SCM scm_make_foreign_object_type (SCM name, SCM slot_names,
>                                     scm_t_struct_finalize finalizer);
>
> Mark suggested it should be "scm_t_struct_finalizer".  I.e. with an r.
> I can agree but scm_t_struct_finalize is a type that's already in 2.0;
> it's just being re-used here.  That said we can do the new name +
> typedef + eventually deprecate the old name dance.

Good point; given the type is already there in 2.0, I’d be in favor of
keeping it as is (without the ‘r’).

> He also suggested to get rid of some type punning, and indeed now we
> have:

Perfect.

>>   2. A general concern about “API churn” and our ability to support the
>>      SMOB API in the future, which I can sympathize with:
>>      <https://lists.gnu.org/archive/html/guile-devel/2015-11/msg00005.html>.
>>
>>      Essentially, (1) it should be easy for people to migrate from SMOBs
>>      to foreign objects (I think it’s largely a matter of
>>      documentation), and (2) existing code that uses the documented SMOB
>>      API should just work in 2.2, possibly with a deprecation warning.
>
> I am not so sure about about this one.  I think it's not accurate to
> characterize beginning to replace a 25-year-old C API (SMOBs) as
> "churn".

I think the point is that there’s lots of code out there that rely on
SMOBs and we shouldn’t break it overnight, precisely because that API is
this old.

Of course, I agree that pushing users towards an improved API is the
right thing long term, no argument here.

> I will get to Mark's point specifically in a minute but regarding your
> points I believe that (1) is somewhat under-documented; the
> documentation is more oriented towards new users than migrating users;
> we can improve here; and (2) should work fine (the old API is still
> there, not even any deprecation warnings currently).

Sounds good.

> I think Mark's desire (if I understand it) was for the new API to
> completely replace the old in all forms, specifically in support for
> mark procedures.  I really think that (a) we should not support the SMOB
> mark procedure interface, as it's both horrible and insufficient for a
> moving collector, and (b) that until we have a moving collector (?), we
> shouldn't attempt to speculatively standardize anything in this regard.
> Basically the arguments from here:
>
>   https://lists.gnu.org/archive/html/guile-devel/2015-11/msg00003.html
>
> There's still the SMOB API if you need mark procedures, basically, but
> hopefully you don't (e.g. when porting GDB to 2.0 I was able to remove
> them; they're less necessary in 2.x than they were in 1.8).  Some users
> still need them, which is why SMOBs are still around :)

Understood.

I think we’ve achieved our goal if code that is not ready to migrate
works the same as with 2.0, which should be the case AIUI.

So, green lights for me.  :-)

Thank you,
Ludo’.

Re: Guile foreign object interface

[David Kastrup]

ludo@gnu.org (Ludovic Courtès) writes:

> Andy Wingo <wingo@pobox.com> skribis:
>
>> I am not so sure about about this one.  I think it's not accurate to
>> characterize beginning to replace a 25-year-old C API (SMOBs) as
>> "churn".
>
> I think the point is that there’s lots of code out there that rely on
> SMOBs and we shouldn’t break it overnight, precisely because that API
> is this old.
>
> Of course, I agree that pushing users towards an improved API is the
> right thing long term, no argument here.

Shrug.  LilyPond has all of its SMOB usage condensed into few C++
classes, so it is comparatively easy to migrate to a different API as
long as it offers comparable functionality.

Which it doesn't (namely the ability of marking objects reached through
STL-managed data structures).  So it's pretty pointless to "push users
towards an improved API" and hardly "the right thing".

-- 
David Kastrup

Re: Guile foreign object interface

[Ludovic Courtès]

David Kastrup <dak@gnu.org> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> Andy Wingo <wingo@pobox.com> skribis:
>>
>>> I am not so sure about about this one.  I think it's not accurate to
>>> characterize beginning to replace a 25-year-old C API (SMOBs) as
>>> "churn".
>>
>> I think the point is that there’s lots of code out there that rely on
>> SMOBs and we shouldn’t break it overnight, precisely because that API
>> is this old.
>>
>> Of course, I agree that pushing users towards an improved API is the
>> right thing long term, no argument here.
>
> Shrug.  LilyPond has all of its SMOB usage condensed into few C++
> classes, so it is comparatively easy to migrate to a different API as
> long as it offers comparable functionality.
>
> Which it doesn't (namely the ability of marking objects reached through
> STL-managed data structures).  So it's pretty pointless to "push users
> towards an improved API" and hardly "the right thing".

Thanks for your feedback, this is a kind of use case we’d like to
support.

I’m sure we already discussed it and then I forgot, but would anything
prevent the use of specific C++ allocators in this case?  The STL data
structures could be allocated on GC-scanned memory, in which case mark
procedures are unneeded.

If this cannot be done, there’s always the possibility of having a weak
hash table to keep the C++ and Scheme object life cycles in sync.  That
happens even with C APIs that record a pointer to an object we care
about in non-scanned memory.  Since 2.0 was out I’ve never used mark
procedures for in C bindings.

Thoughts?

As an aside, please keep the tone friendly as is the norm on this
mailing list.

Thanks,
Ludo’.

Re: Guile foreign object interface

[David Kastrup]

ludo@gnu.org (Ludovic Courtès) writes:

> David Kastrup <dak@gnu.org> skribis:
>
>> ludo@gnu.org (Ludovic Courtès) writes:
>>
>>> Andy Wingo <wingo@pobox.com> skribis:
>>>
>>>> I am not so sure about about this one.  I think it's not accurate to
>>>> characterize beginning to replace a 25-year-old C API (SMOBs) as
>>>> "churn".
>>>
>>> I think the point is that there’s lots of code out there that rely on
>>> SMOBs and we shouldn’t break it overnight, precisely because that API
>>> is this old.
>>>
>>> Of course, I agree that pushing users towards an improved API is the
>>> right thing long term, no argument here.
>>
>> Shrug.  LilyPond has all of its SMOB usage condensed into few C++
>> classes, so it is comparatively easy to migrate to a different API as
>> long as it offers comparable functionality.
>>
>> Which it doesn't (namely the ability of marking objects reached through
>> STL-managed data structures).  So it's pretty pointless to "push users
>> towards an improved API" and hardly "the right thing".
>
> Thanks for your feedback, this is a kind of use case we’d like to
> support.
>
> I’m sure we already discussed it and then I forgot, but would anything
> prevent the use of specific C++ allocators in this case?  The STL data
> structures could be allocated on GC-scanned memory, in which case mark
> procedures are unneeded.

We are not talking about STL data structures containing SCM values but
data structures containing other data structures and pointers to other
data structures.  You cannot sensibly garbage collect when whole memory
areas in which allocation and deallocation is done are in GC-scanned
memory and there is no difference between data structures that have been
freed and data structures that haven't.

> If this cannot be done, there’s always the possibility of having a
> weak hash table to keep the C++ and Scheme object life cycles in sync.
> That happens even with C APIs that record a pointer to an object we
> care about in non-scanned memory.  Since 2.0 was out I’ve never used
> mark procedures for in C bindings.
>
> Thoughts?

LilyPond is a large-scale application which will eat up a significant
ratio of the available address space in 32 bit applications and will
process, in a documentation run, thousands of files with little overlap.
Already in Guile 1.8 where _only_ the stack is conservatively marked and
gc is called explicitly at known points of low-stack usage, we have
occasional false positives in garbage collection (the debugging runs
flag them).  With large memory areas being conservatively scanned, this
is going to get a whole lot worse.

The point of time where it makes sense to try to evaluate the impact or
feasibility of such functionality removals (and Boehm GC _does_ support
finalization) is when LilyPond is otherwise running fine on Guile 2.
With the current slowdown by a factor of 5-10 and a number of other
problems preventing the LilyPond test suite from running, there just
isn't any setup where a useful evaluation could take place.

As there is no technical necessity for planning the wreckage of existing
functionality when there hasn't even been a stable release containing
the prospective successor, I cannot see a viable point in raising yet
more hurdles for existing applications to migrate to Guile 2 rather than
bite the bullet and fork Guile 1.8 in order to actually have some
dependable functionality to base other work on.

> As an aside, please keep the tone friendly as is the norm on this
> mailing list.

Disagreement is not the same as unfriendliness.

-- 
David Kastrup

Re: Guile foreign object interface

[Thien-Thi Nguyen]

[-- Attachment #1: Type: text/plain, Size: 1015 bytes --]


() David Kastrup <dak@gnu.org>
() Thu, 09 Mar 2017 00:00:48 +0100

   [...] rather than [...] fork Guile 1.8 in order to actually
   have some dependable functionality to base other work on.

I intend to maintain 1.8 for the time being.  More precisely, i
seek to apply bug fixes, improve documentation (both method and
content), modernize the build system (tracking gnulib, autotools
evolution), and (time and gumption permitting) refactor some
internals.  If the gods smile, maybe a feature or two (always w/
an eye on upward compatibility).

If the changes to 1.8 that Lilypond requires fall into the above
categories, perhaps we can avoid a fork.  Do you have a summary
of those changes handy?

-- 
Thien-Thi Nguyen -----------------------------------------------
 (defun responsep (query)
   (pcase (context query)
     (`(technical ,ml) (correctp ml))
     ...))                              748E A0E8 1CB8 A748 9BFA
--------------------------------------- 6CE4 6703 2224 4C80 7502

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Guile foreign object interface

[David Kastrup]

Thien-Thi Nguyen <ttn@gnu.org> writes:

> () David Kastrup <dak@gnu.org>
> () Thu, 09 Mar 2017 00:00:48 +0100
>
>    [...] rather than [...] fork Guile 1.8 in order to actually
>    have some dependable functionality to base other work on.
>
> I intend to maintain 1.8 for the time being.  More precisely, i
> seek to apply bug fixes, improve documentation (both method and
> content), modernize the build system (tracking gnulib, autotools
> evolution), and (time and gumption permitting) refactor some
> internals.  If the gods smile, maybe a feature or two (always w/
> an eye on upward compatibility).
>
> If the changes to 1.8 that Lilypond requires fall into the above
> categories, perhaps we can avoid a fork.  Do you have a summary
> of those changes handy?

The changes to 1.8 that LilyPond requires is that Guile-1.8 needs to be
compilable with newer compiler versions and standards and that
distributions are confident enough about it to keep distributing it.

That's all: LilyPond works fine with Guile-1.8 as it is.

Yes, migrating to some sort of UTF-8 capable string scheme transcending
the "byte array" approach of Guile-1 would be nice but the Guile-2
approach would likely be a major performance hog and it does not make
sense for a Guile-1.8 fork to develop a different approach: any such
"different approach" fork should, if at all, focus on the Guile-2 code
base as a starting point.

Unless there is an approach that is so lightweight that it just fits
better when starting with the Guile-1.8 code base.

But all in all the most important thing LilyPond would want from
Guile-1.8 is not to die.

-- 
David Kastrup

Re: Guile foreign object interface

[Ludovic Courtès]

David Kastrup <dak@gnu.org> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> David Kastrup <dak@gnu.org> skribis:
>>
>>> ludo@gnu.org (Ludovic Courtès) writes:
>>>
>>>> Andy Wingo <wingo@pobox.com> skribis:
>>>>
>>>>> I am not so sure about about this one.  I think it's not accurate to
>>>>> characterize beginning to replace a 25-year-old C API (SMOBs) as
>>>>> "churn".
>>>>
>>>> I think the point is that there’s lots of code out there that rely on
>>>> SMOBs and we shouldn’t break it overnight, precisely because that API
>>>> is this old.
>>>>
>>>> Of course, I agree that pushing users towards an improved API is the
>>>> right thing long term, no argument here.
>>>
>>> Shrug.  LilyPond has all of its SMOB usage condensed into few C++
>>> classes, so it is comparatively easy to migrate to a different API as
>>> long as it offers comparable functionality.
>>>
>>> Which it doesn't (namely the ability of marking objects reached through
>>> STL-managed data structures).  So it's pretty pointless to "push users
>>> towards an improved API" and hardly "the right thing".
>>
>> Thanks for your feedback, this is a kind of use case we’d like to
>> support.
>>
>> I’m sure we already discussed it and then I forgot, but would anything
>> prevent the use of specific C++ allocators in this case?  The STL data
>> structures could be allocated on GC-scanned memory, in which case mark
>> procedures are unneeded.
>
> We are not talking about STL data structures containing SCM values but
> data structures containing other data structures and pointers to other
> data structures.  You cannot sensibly garbage collect when whole memory
> areas in which allocation and deallocation is done are in GC-scanned
> memory and there is no difference between data structures that have been
> freed and data structures that haven't.

Not sure what you mean.

>> If this cannot be done, there’s always the possibility of having a
>> weak hash table to keep the C++ and Scheme object life cycles in sync.
>> That happens even with C APIs that record a pointer to an object we
>> care about in non-scanned memory.  Since 2.0 was out I’ve never used
>> mark procedures for in C bindings.
>>
>> Thoughts?
>
> LilyPond is a large-scale application which will eat up a significant
> ratio of the available address space in 32 bit applications and will
> process, in a documentation run, thousands of files with little overlap.
> Already in Guile 1.8 where _only_ the stack is conservatively marked and
> gc is called explicitly at known points of low-stack usage, we have
> occasional false positives in garbage collection (the debugging runs
> flag them).  With large memory areas being conservatively scanned, this
> is going to get a whole lot worse.

This sounds like speculation to me.  I don’t think we’ve seen slowdowns
when Guile then 1.9 switched to BDW-GC, quite the opposite.

> The point of time where it makes sense to try to evaluate the impact or
> feasibility of such functionality removals (and Boehm GC _does_ support
> finalization) is when LilyPond is otherwise running fine on Guile 2.
> With the current slowdown by a factor of 5-10 and a number of other
> problems preventing the LilyPond test suite from running, there just
> isn't any setup where a useful evaluation could take place.

LilyPond on Guile 2.0 is 5–10 times slower than on 1.8?

> As there is no technical necessity for planning the wreckage of existing
> functionality when there hasn't even been a stable release containing
> the prospective successor, I cannot see a viable point in raising yet
> more hurdles for existing applications to migrate to Guile 2 rather than
> bite the bullet and fork Guile 1.8 in order to actually have some
> dependable functionality to base other work on.

Note that SMOBs are still around in 2.2, not even deprecated, only
discouraged.

>> As an aside, please keep the tone friendly as is the norm on this
>> mailing list.
>
> Disagreement is not the same as unfriendliness.

I agree.  However I found the tone of your messages patronizing and
aggressive, assuming bad faith and incompetence on the side of the Guile
developers (“planning for wreckage”, “pretty pointless”, etc. etc.)

This is not OK on the Guile mailing lists.

Thanks,
Ludo’.

Re: Guile foreign object interface

[Eli Zaretskii]

> From: ludo@gnu.org (Ludovic Courtès)
> Date: Thu, 09 Mar 2017 13:09:40 +0100
> 
> >> As an aside, please keep the tone friendly as is the norm on this
> >> mailing list.
> >
> > Disagreement is not the same as unfriendliness.
> 
> I agree.  However I found the tone of your messages patronizing and
> aggressive, assuming bad faith and incompetence on the side of the Guile
> developers (“planning for wreckage”, “pretty pointless”, etc. etc.)

FYI, I've communicated (and occasionally disagreed) with David for
many years, and I can assure you that you see something that simply
isn't there.  He sometimes uses such "colorful" descriptions to make a
point more clear, that's all.  People should be allowed to use their
personal style when writing, without being reprimanded, IMO.

Re: Guile foreign object interface

[Ludovic Courtès]

Eli Zaretskii <eliz@gnu.org> skribis:

>> From: ludo@gnu.org (Ludovic Courtès)
>> Date: Thu, 09 Mar 2017 13:09:40 +0100
>> 
>> >> As an aside, please keep the tone friendly as is the norm on this
>> >> mailing list.
>> >
>> > Disagreement is not the same as unfriendliness.
>> 
>> I agree.  However I found the tone of your messages patronizing and
>> aggressive, assuming bad faith and incompetence on the side of the Guile
>> developers (“planning for wreckage”, “pretty pointless”, etc. etc.)
>
> FYI, I've communicated (and occasionally disagreed) with David for
> many years, and I can assure you that you see something that simply
> isn't there.  He sometimes uses such "colorful" descriptions to make a
> point more clear, that's all.  People should be allowed to use their
> personal style when writing, without being reprimanded, IMO.

I’m all for personal style, but I’m against passive-aggressive or downright
aggressive style.

The problem is not whether the person who writes is well-meaning or not;
the problem is how others perceive it.  If I, as an old-timer and
maintainer, feel attacked when reading these messages, I can only think
that newcomers may feel uncomfortable joining the conversation, at best.

We’d be doing a disservice to our group by sending the message that it’s
OK to be harsh to others.

Ludo’.

Re: Guile foreign object interface

[Eli Zaretskii]

> From: ludo@gnu.org (Ludovic Courtès)
> Cc: guile-user@gnu.org
> Date: Thu, 09 Mar 2017 18:26:09 +0100
> 
> > FYI, I've communicated (and occasionally disagreed) with David for
> > many years, and I can assure you that you see something that simply
> > isn't there.  He sometimes uses such "colorful" descriptions to make a
> > point more clear, that's all.  People should be allowed to use their
> > personal style when writing, without being reprimanded, IMO.
> 
> I’m all for personal style, but I’m against passive-aggressive or downright
> aggressive style.

That's what I'm trying to tell you: there's no aggression.  I once
thought like you, but experience of communicating with David taught me
I was wrong.

> The problem is not whether the person who writes is well-meaning or not;
> the problem is how others perceive it.

I'm saying that you are reading into David's words something that
isn't there, not only in meaning, but also not in form.  You are
misinterpreting his words, which might not be surprising, given that
we all are trying to communicate in a language that is not our first
one.  Things might sound aggressive when they really aren't.

> If I, as an old-timer and maintainer, feel attacked when reading
> these messages

I don't see why you should feel attacked.  FWIW, I see no attack in
David's messages.  He's opinionated, that's for sure.  But there's a
difference between that and an attack.

> We’d be doing a disservice to our group by sending the message that it’s
> OK to be harsh to others.

Indeed, we would.  But there's nothing particularly harsh in David's
messages.  I think it would make this list friendlier if people's
words are not taken as an attack just because they disagree with the
project leaders.  We should be able to disagree and still stay
colleagues in our common quest.

Re: Guile foreign object interface

[Andy Wingo]

On Thu 09 Mar 2017 19:31, Eli Zaretskii <eliz@gnu.org> writes:

>> From: ludo@gnu.org (Ludovic Courtès)
>> Cc: guile-user@gnu.org
>> Date: Thu, 09 Mar 2017 18:26:09 +0100
>> 
>> I’m all for personal style, but I’m against passive-aggressive or downright
>> aggressive style.
>
> That's what I'm trying to tell you: there's no aggression.

I understand that different people can have different reactions and it's
great that you can look through "style" to the substance.  I and a
number of other contributors (evidently including Ludovic) find it hard
to do so, and the only reason we try is because we care about Guile.
It's really weird though to try to ignore this "style" when the style
often says precisely that we _don't_ care, in those words, and other
times in as many words.

Style also has an effect on how willing I am to work with someone, and
with their work.  I think for the similar reasons that we should
encourage good coding styles (that it makes it easy to read and
understand what we are doing), we should encourage good communications
styles as well, and give negative feedback on those styles that need
work.

Finally, as Ludovic mentioned, one person's style has an effect on the
composition of the body of Guile users and developers.  Whether someone
feels welcome in a group is a function of lots of subtle and
not-so-subtle things.  Feeling welcome is, for most people, a
prerequisite to further involvement.  Guile does OK here -- some good
aspects, some needs-improvement aspects -- but I think we can all think
of examples of online communities that have atrophied over time, and a
hostile online environment can be one of the reasons for this;
intervention of maintainers to avoid this failure mode should be
expected, I think.

I know it's a complicated issue.  In the particular case of the mail
that Ludovic responded to, I agree with his message and think he did a
fine job -- focussing on issues but also making clear that we should be
kind to each other in the way we communicate.  It was more charitable
than I was going to be :)

Regards,

Andy

Re: Guile foreign object interface

[Eli Zaretskii]

> From: Andy Wingo <wingo@pobox.com>
> Cc: ludo@gnu.org (Ludovic Courtès),  guile-user@gnu.org
> Date: Thu, 09 Mar 2017 20:56:09 +0100
> 
> > That's what I'm trying to tell you: there's no aggression.
> 
> I understand that different people can have different reactions and it's
> great that you can look through "style" to the substance.  I and a
> number of other contributors (evidently including Ludovic) find it hard
> to do so, and the only reason we try is because we care about Guile.
> It's really weird though to try to ignore this "style" when the style
> often says precisely that we _don't_ care, in those words, and other
> times in as many words.

Given the certain amount of frustration over the real problems that
didn't get solved until now, you can understand that, don't you?

I had my share of problems reported here, mostly with working patches,
some of which took many moons, sometimes years to get solved upstream.
So I think I understand some of David's frustration, although the
problems I reported were nowhere near the gravity of those Lilypond
has.

So maybe the project leadership should try to improve the efficiency
of handling bug reports and of catering to the problems raised by
projects using Guile, as a means to lower frustration and make the
environment friendlier?

Re: Guile foreign object interface

[Andy Wingo]

On Fri 10 Mar 2017 08:38, Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andy Wingo <wingo@pobox.com>
>> Cc: ludo@gnu.org (Ludovic Courtès),  guile-user@gnu.org
>> Date: Thu, 09 Mar 2017 20:56:09 +0100
>> 
>> > That's what I'm trying to tell you: there's no aggression.
>> 
>> I understand that different people can have different reactions and it's
>> great that you can look through "style" to the substance.  I and a
>> number of other contributors (evidently including Ludovic) find it hard
>> to do so, and the only reason we try is because we care about Guile.
>> It's really weird though to try to ignore this "style" when the style
>> often says precisely that we _don't_ care, in those words, and other
>> times in as many words.
>
> Given the certain amount of frustration over the real problems that
> didn't get solved until now, you can understand that, don't you?

Oh sure.  Lots of frustration going around :) However I know that if I
need help making something better, then I should avoid antagonizing
other people working on it, especially the maintainers who will be left
maintaining the code.  That's all.

> I had my share of problems reported here, mostly with working patches,
> some of which took many moons, sometimes years to get solved upstream.

Yeah sorry about that :/ For the record I really appreciate your work
and have always enjoyed working with you and I definitely sympathize
with your frustrations.  I also think that it would not have been the
right thing for Guile to simply apply some of the in-line portability
fixes that were the starting points for many of your excellent patches
-- it took work on both sides to get them applied, and that work on my
side at least competed with many other things.

I would like to focus less on individual people and more on behaviors
though.

> So maybe the project leadership should try to improve the efficiency
> of handling bug reports and of catering to the problems raised by
> projects using Guile, as a means to lower frustration and make the
> environment friendlier?

Certainly, I hope we do that :) Of course that depends on good will on
everyone's part, maintainers and contributors, and I think there are
"styles" that create good will and styles that don't help.

Happy hacking,

Andy

Re: Guile foreign object interface

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andy Wingo <wingo@pobox.com>
>> Cc: ludo@gnu.org (Ludovic Courtès),  guile-user@gnu.org
>> Date: Thu, 09 Mar 2017 20:56:09 +0100
>> 
>> > That's what I'm trying to tell you: there's no aggression.
>> 
>> I understand that different people can have different reactions and it's
>> great that you can look through "style" to the substance.  I and a
>> number of other contributors (evidently including Ludovic) find it hard
>> to do so, and the only reason we try is because we care about Guile.
>> It's really weird though to try to ignore this "style" when the style
>> often says precisely that we _don't_ care, in those words, and other
>> times in as many words.
>
> Given the certain amount of frustration over the real problems that
> didn't get solved until now, you can understand that, don't you?
>
> I had my share of problems reported here, mostly with working patches,
> some of which took many moons, sometimes years to get solved upstream.
> So I think I understand some of David's frustration, although the
> problems I reported were nowhere near the gravity of those Lilypond
> has.
>
> So maybe the project leadership should try to improve the efficiency
> of handling bug reports and of catering to the problems raised by
> projects using Guile, as a means to lower frustration and make the
> environment friendlier?

The "project leadership" in Free Software more often than not does not
have any assignable resources so there is little point in venting
frustration in that regard.  My concerns are not as much about lack of
progress but rather about moving Guile in a direction where it becomes
increasingly unfeasible as an extension language not just for LilyPond
because extensive interaction with other programming models and
languages becomes too expensive, invasive, fragile and inflexible.

To a certain degree one can chalk this off as "growing pains" that
long-standing users have to shoulder, at least when working with
development rather than stable versions.  But without a visible intent
of actually prioritizing or even addressing those, you end up with more
like a "one step forward, two steps back" approach rather than the other
way round, at least for already established uses rather than those for
which the steps forward are explicitly intended.

LilyPond is getting removed from distributions these days because it
requires Guile-1.8.  Probably even worse, some distributions deliver
binaries linked with Guile-2.0 (the options available for compiling with
Guile-2.0 support are only intended for developers) which means that the
binaries are not usable for serious work (far too slow and unstable).

This situation has been developing for a number of years.  It would
probably take less effort to address to a tolerable degree than making
Emacs-on-Guile more than an academic option.  And make no mistake: there
is quite an overlap in problems that need solving for making
Emacs-Guile2 viable and making LilyPond-Guile2 viable.

But Emacs does not suffer the same amount of migration pressure since
there is no independent Elisp project in different iterations involved.
It can just stick with what works, short of a political decision (like
the one that put MULE into Emacs 20, costly then, amortized by now).

-- 
David Kastrup

Re: Guile foreign object interface

[Arne Babenhauserheide]

David Kastrup writes:

> To a certain degree one can chalk this off as "growing pains" that
> long-standing users have to shoulder, at least when working with
> development rather than stable versions.

I’d like to chime in here. When looking at the prospects of larger Guile
adoption, I think that Lilypond is a critical component: It is an early
adopter and the absolute top tool in the niche it took.

People thinking about adopting Guile will ask themselves "is this a
viable longterm option?". They will then look at Lilypond, the prime
example of a highly successful Guile-using tool.

So this is not just a growing pain for Lilypond. It is a critical issue
for Guile.

Both communities, Lilypond and Guile, need Lilypond-Guile2 to work well.

And given the speed I see from Guile 2.1.7 at other tasks, there should
be ways to make Lilypond-Guile2.2 outperform Lilypond-Guile1.8
significantly.

Best wishes,
Arne

PS: Just saying "it’s a scripting language now" will not cut it. People
    who adopt Guile now will have to ask whether it will stay a viable
    option as scripting language, and they will again look at Lilypond
    to see whether Guile-as-an-option kept its promises.

Re: Guile foreign object interface

[David Kastrup]

Arne Babenhauserheide <arne_bab@web.de> writes:

> David Kastrup writes:
>
>> To a certain degree one can chalk this off as "growing pains" that
>> long-standing users have to shoulder, at least when working with
>> development rather than stable versions.
>
> I’d like to chime in here. When looking at the prospects of larger Guile
> adoption, I think that Lilypond is a critical component: It is an early
> adopter and the absolute top tool in the niche it took.
>
> People thinking about adopting Guile will ask themselves "is this a
> viable longterm option?". They will then look at Lilypond, the prime
> example of a highly successful Guile-using tool.
>
> So this is not just a growing pain for Lilypond. It is a critical issue
> for Guile.
>
> Both communities, Lilypond and Guile, need Lilypond-Guile2 to work well.
>
> And given the speed I see from Guile 2.1.7 at other tasks, there should
> be ways to make Lilypond-Guile2.2 outperform Lilypond-Guile1.8
> significantly.
>
> Best wishes,
> Arne
>
> PS: Just saying "it’s a scripting language now" will not cut it.

With an implied "rather than an extension language": that _would_ cut
it.  It would imply LilyPond having to work with a fork of Guile-1.8,
and possibly encourage active maintenance of such a fork independently
of LilyPond.  It would also put a clear perspective on Emacs-Guile's
future, namely none.

It would be a valid and clear option to pursue.  In some respects, that
is where I see Guile drifting, but it appears to me as something
happening more by accident than design and it is not apparently what its
developers actively _chose_ for Guile's future.

> People who adopt Guile now will have to ask whether it will stay a
> viable option as scripting language, and they will again look at
> Lilypond to see whether Guile-as-an-option kept its promises.

Well, Guile is not an "option" in LilyPond, and it is clearly more than
just an extension language (as I believe it is designed to be in
GnuCash).  It's more like its programming paradigm.  The C++ part is
structured to fit in with Guile.  But this sort of 1:1 relation is much
more tenuous with Guile-2 since the interaction costs have become quite
larger.  So it works better for either applications that have just a
little Guile, or for applications that have little else.

-- 
David Kastrup

Re: Guile foreign object interface

[Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 1673 bytes --]


David Kastrup <dak@gnu.org> writes:

> Arne Babenhauserheide <arne_bab@web.de> writes:
>> PS: Just saying "it’s a scripting language now" will not cut it.
>
> With an implied "rather than an extension language": that _would_ cut
> it.  It would imply LilyPond having to work with a fork of Guile-1.8,
> and possibly encourage active maintenance of such a fork independently
> of LilyPond.  It would also put a clear perspective on Emacs-Guile's
> future, namely none.

It would also show that Guile leaves people behind who choose it. Yes,
it would be a strategy, but not one which inspires much confidence in
the longterm stability of its goals.

>> People who adopt Guile now will have to ask whether it will stay a
>> viable option as scripting language, and they will again look at
>> Lilypond to see whether Guile-as-an-option kept its promises.
>
> Well, Guile is not an "option" in LilyPond,

I did not mean it as option in Lilypond. I meant it as strategic
possibility.

> and it is clearly more than just an extension language (as I believe
> it is designed to be in GnuCash).  It's more like its programming
> paradigm.  The C++ part is structured to fit in with Guile.  But this
> sort of 1:1 relation is much more tenuous with Guile-2 since the
> interaction costs have become quite larger.  So it works better for
> either applications that have just a little Guile, or for applications
> that have little else.

I hope that will change again. I’m sure most people here want Lilypond
to be faster with Guile 2.x than with Guile 1.8.

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 800 bytes --]

Re: Guile foreign object interface

[David Kastrup]

Arne Babenhauserheide <arne_bab@web.de> writes:

> David Kastrup <dak@gnu.org> writes:
>
>> Arne Babenhauserheide <arne_bab@web.de> writes:
>>> PS: Just saying "it’s a scripting language now" will not cut it.
>>
>> With an implied "rather than an extension language": that _would_ cut
>> it.  It would imply LilyPond having to work with a fork of Guile-1.8,
>> and possibly encourage active maintenance of such a fork independently
>> of LilyPond.  It would also put a clear perspective on Emacs-Guile's
>> future, namely none.
>
> It would also show that Guile leaves people behind who choose it.

We are talking about Free Software here.  Its programmers and
maintainers are the ones who have to find the energy to take it places.
An entity "Guile" that would be able to assign resources does not really
exist.

My personal impression is that the goals and distribution of the energy
right now does not really scale to match the goals that Guile has been
selected for with respect of its intended and announced roles in the GNU
project a long time ago.

That's unfortunate, but a mismatch of hopes and expectations, even
_announced_ hopes and expectations with what actually turns out to be
possible given finite resources and inspiration, is not malice.

> Yes, it would be a strategy, but not one which inspires much
> confidence in the longterm stability of its goals.

"longterm stability of goals" is a mixed blessing.  The world changes.
Guile 2.0 is not as stable as Guile 1.8 was for use, but the version
numbers don't suggest differently.  I was not involved with LilyPond
when it still worked with Guile versions like 1.4: I assume that was
quite more strenuous.

There is no really established and comfortable migration path from
Guile-1.8 to Guile-2.x but there is not a large corpus of applications
that would make working on such a path in general rewarding or even
worthwhile.

So it is natural that projects are "left behind" as default and one
needs to find the best way to help them over into the future with
explicit and manual effort.  With larger adoption of Guile, the choices
and options here might have looked differently.

> I did not mean it as option in Lilypond. I meant it as strategic
> possibility.

As I see it, it's not big enough that strategic guarantees can be given.
If you need guarantees, you need to invest into the resources needed for
that.

>> The C++ part [of LilyPond] is structured to fit in with Guile.  But
>> this sort of 1:1 relation is much more tenuous with Guile-2 since the
>> interaction costs have become quite larger.  So it works better for
>> either applications that have just a little Guile, or for
>> applications that have little else.
>
> I hope that will change again. I’m sure most people here want Lilypond
> to be faster with Guile 2.x than with Guile 1.8.

Where is "here"?  It would look better, for sure.

-- 
David Kastrup

Re: Guile foreign object interface

[Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 767 bytes --]


David Kastrup <dak@gnu.org> writes:

>> I did not mean it as option in Lilypond. I meant it as strategic
>> possibility.
>
> As I see it, it's not big enough that strategic guarantees can be given.
> If you need guarantees, you need to invest into the resources needed for
> that.

Using "The official GNU extension language" is typically pretty safe,
since GNU libraries tend to live and stay stable longer than the typical
middle size company.

>> I hope that will change again. I’m sure most people here want Lilypond
>> to be faster with Guile 2.x than with Guile 1.8.
>
> Where is "here"?  It would look better, for sure.

Here is the Guile community.

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 800 bytes --]

Re: Guile foreign object interface

[David Kastrup]

ludo@gnu.org (Ludovic Courtès) writes:

> David Kastrup <dak@gnu.org> skribis:
>
>> ludo@gnu.org (Ludovic Courtès) writes:
>>
>>> I’m sure we already discussed it and then I forgot, but would
>>> anything prevent the use of specific C++ allocators in this case?
>>> The STL data structures could be allocated on GC-scanned memory, in
>>> which case mark procedures are unneeded.
>>
>> We are not talking about STL data structures containing SCM values
>> but data structures containing other data structures and pointers to
>> other data structures.  You cannot sensibly garbage collect when
>> whole memory areas in which allocation and deallocation is done are
>> in GC-scanned memory and there is no difference between data
>> structures that have been freed and data structures that haven't.
>
> Not sure what you mean.

The code is out there.  Typical stuff looks like

SCM
Context::mark_smob () const
{
  scm_gc_mark (context_list_);
  scm_gc_mark (aliases_);
  scm_gc_mark (definition_);
  scm_gc_mark (definition_mods_);
  scm_gc_mark (properties_scm_);
  scm_gc_mark (accepts_list_);
  scm_gc_mark (default_child_);

  if (implementation_)
    scm_gc_mark (implementation_->self_scm ());

  if (event_source_)
    scm_gc_mark (event_source_->self_scm ());

  if (events_below_)
    scm_gc_mark (events_below_->self_scm ());

  derived_mark ();

  return properties_scm_;
}

The first members are marked directly as SCM.  The others are marked
indirectly.

There is also stuff like

void
Slur_engraver::derived_mark () const
{
  for (vsize i = start_events_.size (); i--;)
    {
      scm_gc_mark (start_events_[i].slur_->self_scm ());
      if (start_events_[i].note_)
        scm_gc_mark (start_events_[i].note_->self_scm ());
    }
  for (vsize i = stop_events_.size (); i--;)
    {
      scm_gc_mark (stop_events_[i].slur_->self_scm ());
      if (stop_events_[i].note_)
        scm_gc_mark (stop_events_[i].note_->self_scm ());
    }
}

where the involved arrays are actually STL vectors.

>> LilyPond is a large-scale application which will eat up a significant
>> ratio of the available address space in 32 bit applications and will
>> process, in a documentation run, thousands of files with little overlap.
>> Already in Guile 1.8 where _only_ the stack is conservatively marked and
>> gc is called explicitly at known points of low-stack usage, we have
>> occasional false positives in garbage collection (the debugging runs
>> flag them).  With large memory areas being conservatively scanned, this
>> is going to get a whole lot worse.
>
> This sounds like speculation to me.  I don’t think we’ve seen
> slowdowns when Guile then 1.9 switched to BDW-GC, quite the opposite.

This is not a problem of slowdowns but of unreclaimed memory because of
false positives in reference detections.  LilyPond has large sessions
where it basically frees all memory after the startup thousands of times
over.

>> The point of time where it makes sense to try to evaluate the impact
>> or feasibility of such functionality removals (and Boehm GC _does_
>> support finalization) is when LilyPond is otherwise running fine on
>> Guile 2.  With the current slowdown by a factor of 5-10 and a number
>> of other problems preventing the LilyPond test suite from running,
>> there just isn't any setup where a useful evaluation could take
>> place.
>
> LilyPond on Guile 2.0 is 5–10 times slower than on 1.8?

Yes.

>> As there is no technical necessity for planning the wreckage of
>> existing functionality when there hasn't even been a stable release
>> containing the prospective successor, I cannot see a viable point in
>> raising yet more hurdles for existing applications to migrate to
>> Guile 2 rather than bite the bullet and fork Guile 1.8 in order to
>> actually have some dependable functionality to base other work on.
>
> Note that SMOBs are still around in 2.2, not even deprecated, only
> discouraged.

I have no problem with discouragement.

>>> As an aside, please keep the tone friendly as is the norm on this
>>> mailing list.
>>
>> Disagreement is not the same as unfriendliness.
>
> I agree.  However I found the tone of your messages patronizing and
> aggressive, assuming bad faith and incompetence on the side of the
> Guile developers (“planning for wreckage”, “pretty pointless”,
> etc. etc.)
>
> This is not OK on the Guile mailing lists.

Given the current state of things and the current discussion and
wordings being used, the terms I used are describing the effects as they
appear when viewed through the lens of LilyPond development.  I saw no
convincing _technical_ reason for the enthusiasm towards stopping users
from being able to use their existing code without a workable
replacement in light of the existing mechanisms.

As I said: LilyPond has its Guile object management encapsulated and
modularized to a degree where it was simple to implement some finalizer
workaround on the LilyPond side until the problem finally also got fixed
in Guile 2.0.13 or so.

But it would be a huge step backwards if you had to use STL containers
with separate allocation if and only if SCM data structures got
involved.  It would mean that creating and maintaining standard LilyPond
code would require a lot more low-level expertise from programmers than
is the case now.  I've been working hard on encapsulating special
knowledge into modules that make creating a lot of LilyPond code
accessible to programmers with moderate skill levels.

I'll readily grant that foregoing mark procedures would be beneficial in
that respect, but it would be more than offset by having to meddle with
custom allocators: that is _way_ more complex for average programmers.

As long as there are no actual good answers to the problems posed by
LilyPond's manner of wrapping C++ classes containing STL data types
referencing SCM with various degrees of indirection (which is done for
good reasons), the lighthearted manner in which burning bridges with
existing code (which addresses existing problems in reasonably
straightforward manner) is proposed is disconcerting.

I am aware that not working with finalizers/mark routines has its own
attractions: if it didn't, bugs like the one addressed with

commit 8442211ef0029581b35f784489afcf210491fc41
Author: David Kastrup <dak@gnu.org>
Date:   Sat Sep 20 05:17:54 2014 -0400

    Fix SCM_SMOB_OBJECT{_,_0_,_1_,_2_,_3_}LOC.
    
    Fixes <http://bugs.gnu.org/18495>.
    
    * libguile/smob.h (SCM_SMOB_OBJECT_LOC, SCM_SMOB_OBJECT_0_LOC)
      (SCM_SMOB_OBJECT_1_LOC, SCM_SMOB_OBJECT_2_LOC)
      (SCM_SMOB_OBJECT_3_LOC): These elementary API macros have been broken
      by commit 56164dc47f6616b359f0ad23be208f01a77b55fa in 2009.
    
    Signed-off-by: David Kastrup <dak@gnu.org>

wouldn't have survived as long.

-- 
David Kastrup