bug#20209: GUILE 2.0.11: crash in set_port_filename_x for bytevector ports

bug#20209: GUILE 2.0.11: crash in set_port_filename_x for bytevector ports

[David Kastrup]

[-- Attachment #1: Type: text/plain, Size: 21 bytes --]


The following code


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: nextbug.c --]
[-- Type: text/x-csrc, Size: 273 bytes --]

#include <libguile.h>

int main()
{
  scm_init_guile ();
  SCM str = scm_c_make_bytevector (0);
  SCM port = scm_open_bytevector_input_port (str, SCM_UNDEFINED);
  scm_set_port_filename_x (port, scm_from_locale_string ("/usr/local/tmp/lilypond/ly/init.ly"));
  return 0;
}

[-- Attachment #3: Type: text/plain, Size: 2392 bytes --]


crashes with the backtrace

Program terminated with signal SIGSEGV, Segmentation fault.
#0  0xb7699059 in scm_iprlist () from /usr/lib/libguile-2.0.so.22
(gdb) bt
#0  0xb7699059 in scm_iprlist () from /usr/lib/libguile-2.0.so.22
#1  0xb7699e95 in ?? () from /usr/lib/libguile-2.0.so.22
#2  0xb7699153 in scm_iprlist () from /usr/lib/libguile-2.0.so.22
#3  0xb7699e95 in ?? () from /usr/lib/libguile-2.0.so.22
#4  0xb7699153 in scm_iprlist () from /usr/lib/libguile-2.0.so.22
#5  0xb7699e95 in ?? () from /usr/lib/libguile-2.0.so.22
#6  0xb7698ec9 in scm_prin1 () from /usr/lib/libguile-2.0.so.22
#7  0xb769a606 in scm_simple_format () from /usr/lib/libguile-2.0.so.22
#8  0xb76e5f0b in ?? () from /usr/lib/libguile-2.0.so.22
#9  0xb76e6539 in ?? () from /usr/lib/libguile-2.0.so.22
#10 0xb76e664c in scm_call_with_vm () from /usr/lib/libguile-2.0.so.22
#11 0xb764cc67 in scm_apply () from /usr/lib/libguile-2.0.so.22
#12 0xb764e90e in scm_apply_1 () from /usr/lib/libguile-2.0.so.22
#13 0xb76cdb7b in scm_throw () from /usr/lib/libguile-2.0.so.22
#14 0xb76ce04c in scm_ithrow () from /usr/lib/libguile-2.0.so.22
#15 0xb764b6a3 in scm_error_scm () from /usr/lib/libguile-2.0.so.22
#16 0xb764b778 in scm_error () from /usr/lib/libguile-2.0.so.22
#17 0xb764ba22 in scm_wrong_type_arg () from /usr/lib/libguile-2.0.so.22
#18 0xb76927ca in scm_set_port_filename_x () from /usr/lib/libguile-2.0.so.22
#19 0x08048723 in main () at nextbug.c:8

after outputting

guile: uncaught throw to wrong-type-arg: (set-port-filename! Wrong type argument in position ~A: ~S (1 (Segmentation fault (core dumped)

(by the way: what's up with the non-call of format here?  It's not the
first time I see wrong-type-arg spitting out a format string unmodified)

Compilation was done using
gcc -Wall -ggdb `guile-config compile` nextbug.c `guile-config link` -o nextbug

Version is gcc (Ubuntu 4.9.1-16ubuntu6) 4.9.1

platform is
Linux version 3.16.0-33-generic (buildd@brownie) (gcc version 4.9.1 (Ubuntu 4.9.1-16ubuntu6) ) #44-Ubuntu SMP Thu Mar 12 12:25:57 UTC 2015


A few simple experiments for triggering the same crash from Scheme
rather than C were not successful.  That does not necessarily mean much.

It's not hard to guess that this is yet another roadblock on the way to
get LilyPond working with GUILE2.

Any ideas regarding what may be causing this error and how to work
around it?

-- 
David Kastrup

bug#20209: GUILE 2.0.11: crash in set_port_filename_x for bytevector ports

[Mark H Weaver]

David Kastrup <dak@gnu.org> writes:

> The following code
>
>
> #include <libguile.h>
>
> int main()
> {
>   scm_init_guile ();
>   SCM str = scm_c_make_bytevector (0);
>   SCM port = scm_open_bytevector_input_port (str, SCM_UNDEFINED);
>   scm_set_port_filename_x (port, scm_from_locale_string ("/usr/local/tmp/lilypond/ly/init.ly"));
>   return 0;
> }
>
>
> crashes with the backtrace
>
> Program terminated with signal SIGSEGV, Segmentation fault.
> #0  0xb7699059 in scm_iprlist () from /usr/lib/libguile-2.0.so.22
> (gdb) bt
> #0  0xb7699059 in scm_iprlist () from /usr/lib/libguile-2.0.so.22
> #1  0xb7699e95 in ?? () from /usr/lib/libguile-2.0.so.22
> #2  0xb7699153 in scm_iprlist () from /usr/lib/libguile-2.0.so.22
> #3  0xb7699e95 in ?? () from /usr/lib/libguile-2.0.so.22
> #4  0xb7699153 in scm_iprlist () from /usr/lib/libguile-2.0.so.22
> #5  0xb7699e95 in ?? () from /usr/lib/libguile-2.0.so.22
> #6  0xb7698ec9 in scm_prin1 () from /usr/lib/libguile-2.0.so.22
> #7  0xb769a606 in scm_simple_format () from /usr/lib/libguile-2.0.so.22
> #8  0xb76e5f0b in ?? () from /usr/lib/libguile-2.0.so.22
> #9  0xb76e6539 in ?? () from /usr/lib/libguile-2.0.so.22
> #10 0xb76e664c in scm_call_with_vm () from /usr/lib/libguile-2.0.so.22
> #11 0xb764cc67 in scm_apply () from /usr/lib/libguile-2.0.so.22
> #12 0xb764e90e in scm_apply_1 () from /usr/lib/libguile-2.0.so.22
> #13 0xb76cdb7b in scm_throw () from /usr/lib/libguile-2.0.so.22
> #14 0xb76ce04c in scm_ithrow () from /usr/lib/libguile-2.0.so.22
> #15 0xb764b6a3 in scm_error_scm () from /usr/lib/libguile-2.0.so.22
> #16 0xb764b778 in scm_error () from /usr/lib/libguile-2.0.so.22
> #17 0xb764ba22 in scm_wrong_type_arg () from /usr/lib/libguile-2.0.so.22
> #18 0xb76927ca in scm_set_port_filename_x () from /usr/lib/libguile-2.0.so.22
> #19 0x08048723 in main () at nextbug.c:8
>
> after outputting
>
> guile: uncaught throw to wrong-type-arg: (set-port-filename! Wrong
> type argument in position ~A: ~S (1 (Segmentation fault (core dumped)

The problem turns out to be that binary ports are not initialized until
the (ice-9 binary-ports) module is loaded.  As a result, the port
returned by 'scm_open_bytevector_input_port' has a bad type tag.

The workaround is to call scm_c_resolve_module ("ice-9 binary-ports")
after initializing Guile before the first call to
'scm_open_bytevector_input_port'.  The following code works:

--8<---------------cut here---------------start------------->8---
#include <libguile.h>

int main()
{
  scm_init_guile ();
  (void) scm_c_resolve_module ("ice-9 binary-ports");
  SCM str = scm_c_make_bytevector (0);
  SCM port = scm_open_bytevector_input_port (str, SCM_UNDEFINED);
  scm_set_port_filename_x (port, scm_from_locale_string ("/usr/local/tmp/lilypond/ly/init.ly"));
  return 0;
}
--8<---------------cut here---------------end--------------->8---

      Mark

bug#20209: GUILE 2.0.11: crash in set_port_filename_x for bytevector ports

[David Kastrup]

Mark H Weaver <mhw@netris.org> writes:

> The problem turns out to be that binary ports are not initialized until
> the (ice-9 binary-ports) module is loaded.  As a result, the port
> returned by 'scm_open_bytevector_input_port' has a bad type tag.
>
> The workaround is to call scm_c_resolve_module ("ice-9 binary-ports")
> after initializing Guile before the first call to
> 'scm_open_bytevector_input_port'.

It kind of defeats the point of having scm_open_bytevector_input_port in
the C library part if you have to call scm_c_resolve_module before use.
It's also a rather inscrutable error symptom.  Any chance wrong-type-arg
can guard against uninitialized/invalid types specifically?  There is a
reasonably high chance that bad/uninitialized SCM will wash up there.

The more thorough way would be to check the type tag to be in valid
range before doing any smob callback.

-- 
David Kastrup

bug#20209: GUILE 2.0.11: crash in set_port_filename_x for bytevector ports

[Mark H Weaver]

Hi Ludovic,

There's a problem with the C interfaces for bytevectors and binary
ports, introduced in 1ee2c72eafaae5f91f4c899bc4b4853af5c16f28.  Neither
of those modules are initialized until the associated Scheme modules are
loaded, but this is not documented.

There are undocumented public C API functions 'scm_init_r6rs_ports' and
'scm_init_bytevectors', but if they are called and the associated Scheme
modules (rnrs bytevectors) or (ice-9 binary-ports) are also loaded, then
those initialization functions will be called more than once.

In the case of 'scm_init_r6rs_ports', the duplicate initialization would
cause 'scm_make_port_type' to be called more than once, leading to
multiple copies of the same port types to be created.  The snarfed code
in r6rs-ports.x and bytevectors.x would also be run more than once.  I
don't know whether these duplicate initializations would cause problems,
but it seems like a bad idea.

Another issue is that 'scm_init_r6rs_ports' does not call
'scm_init_bytevectors', and yet that module calls C functions from
bytevectors.c.

For these reasons, I decided against recommending those undocumented
initialization functions.  Instead, I suggested that David initialize
binary ports by loading (ice-9 binary-ports):

  (void) scm_c_resolve_module ("ice-9 binary-ports");

For Guile 2.0.12, I suggest that we unconditionally do the equivalent of
'scm_init_bytevectors' and 'scm_init_r6rs_ports' during Guile
initialization, and make those functions into deprecated no-ops.

What do you think?

      Mark
      

Mark H Weaver <mhw@netris.org> writes:

> David Kastrup <dak@gnu.org> writes:
>
>> The following code
>>
>>
>> #include <libguile.h>
>>
>> int main()
>> {
>>   scm_init_guile ();
>>   SCM str = scm_c_make_bytevector (0);
>>   SCM port = scm_open_bytevector_input_port (str, SCM_UNDEFINED);
>>   scm_set_port_filename_x (port, scm_from_locale_string ("/usr/local/tmp/lilypond/ly/init.ly"));
>>   return 0;
>> }
>>
>>
>> crashes with the backtrace
>>
>> Program terminated with signal SIGSEGV, Segmentation fault.
>> #0  0xb7699059 in scm_iprlist () from /usr/lib/libguile-2.0.so.22
>> (gdb) bt
>> #0  0xb7699059 in scm_iprlist () from /usr/lib/libguile-2.0.so.22
>> #1  0xb7699e95 in ?? () from /usr/lib/libguile-2.0.so.22
>> #2  0xb7699153 in scm_iprlist () from /usr/lib/libguile-2.0.so.22
>> #3  0xb7699e95 in ?? () from /usr/lib/libguile-2.0.so.22
>> #4  0xb7699153 in scm_iprlist () from /usr/lib/libguile-2.0.so.22
>> #5  0xb7699e95 in ?? () from /usr/lib/libguile-2.0.so.22
>> #6  0xb7698ec9 in scm_prin1 () from /usr/lib/libguile-2.0.so.22
>> #7  0xb769a606 in scm_simple_format () from /usr/lib/libguile-2.0.so.22
>> #8  0xb76e5f0b in ?? () from /usr/lib/libguile-2.0.so.22
>> #9  0xb76e6539 in ?? () from /usr/lib/libguile-2.0.so.22
>> #10 0xb76e664c in scm_call_with_vm () from /usr/lib/libguile-2.0.so.22
>> #11 0xb764cc67 in scm_apply () from /usr/lib/libguile-2.0.so.22
>> #12 0xb764e90e in scm_apply_1 () from /usr/lib/libguile-2.0.so.22
>> #13 0xb76cdb7b in scm_throw () from /usr/lib/libguile-2.0.so.22
>> #14 0xb76ce04c in scm_ithrow () from /usr/lib/libguile-2.0.so.22
>> #15 0xb764b6a3 in scm_error_scm () from /usr/lib/libguile-2.0.so.22
>> #16 0xb764b778 in scm_error () from /usr/lib/libguile-2.0.so.22
>> #17 0xb764ba22 in scm_wrong_type_arg () from /usr/lib/libguile-2.0.so.22
>> #18 0xb76927ca in scm_set_port_filename_x () from /usr/lib/libguile-2.0.so.22
>> #19 0x08048723 in main () at nextbug.c:8
>>
>> after outputting
>>
>> guile: uncaught throw to wrong-type-arg: (set-port-filename! Wrong
>> type argument in position ~A: ~S (1 (Segmentation fault (core dumped)
>
> The problem turns out to be that binary ports are not initialized until
> the (ice-9 binary-ports) module is loaded.  As a result, the port
> returned by 'scm_open_bytevector_input_port' has a bad type tag.
>
> The workaround is to call scm_c_resolve_module ("ice-9 binary-ports")
> after initializing Guile before the first call to
> 'scm_open_bytevector_input_port'.  The following code works:
>
> #include <libguile.h>
>
> int main()
> {
>   scm_init_guile ();
>   (void) scm_c_resolve_module ("ice-9 binary-ports");
>   SCM str = scm_c_make_bytevector (0);
>   SCM port = scm_open_bytevector_input_port (str, SCM_UNDEFINED);
>   scm_set_port_filename_x (port, scm_from_locale_string ("/usr/local/tmp/lilypond/ly/init.ly"));
>   return 0;
> }
>
>       Mark

bug#20209: GUILE 2.0.11: crash in set_port_filename_x for bytevector ports

[Mark H Weaver]

David Kastrup <dak@gnu.org> writes:

> It's also a rather inscrutable error symptom.  Any chance wrong-type-arg
> can guard against uninitialized/invalid types specifically?

In this case the "bad type tag" was a valid tag, but not the one
intended for ports.  The tags of the returned port indicated that it was
a pair those car contained a non-immediate object located at an unmapped
address, which caused a segfault while trying to print it.

     Mark

bug#20209: GUILE 2.0.11: crash in set_port_filename_x for bytevector ports

[Ludovic Courtès]

Mark H Weaver <mhw@netris.org> skribis:

> For these reasons, I decided against recommending those undocumented
> initialization functions.  Instead, I suggested that David initialize
> binary ports by loading (ice-9 binary-ports):
>
>   (void) scm_c_resolve_module ("ice-9 binary-ports");

That sounds like the best option, indeed.

> For Guile 2.0.12, I suggest that we unconditionally do the equivalent of
> 'scm_init_bytevectors' and 'scm_init_r6rs_ports' during Guile
> initialization, and make those functions into deprecated no-ops.

Agreed, that’s what I was going to suggest.

That these init functions are not marked as internal is really an
oversight.

I wouldn’t be against simply making them SCM_INTERNAL in 2.0.12, but
making them public + deprecated as you suggest is even better.

Thanks,
Ludo’.

bug#20209: GUILE 2.0.11: crash in set_port_filename_x for bytevector ports

[Andy Wingo]

[-- Attachment #1: Type: text/plain, Size: 955 bytes --]

On Sun 29 Mar 2015 15:06, ludo@gnu.org (Ludovic Courtès) writes:

> Mark H Weaver <mhw@netris.org> skribis:
>
>> For these reasons, I decided against recommending those undocumented
>> initialization functions.  Instead, I suggested that David initialize
>> binary ports by loading (ice-9 binary-ports):
>>
>>   (void) scm_c_resolve_module ("ice-9 binary-ports");
>
> That sounds like the best option, indeed.
>
>> For Guile 2.0.12, I suggest that we unconditionally do the equivalent of
>> 'scm_init_bytevectors' and 'scm_init_r6rs_ports' during Guile
>> initialization, and make those functions into deprecated no-ops.
>
> Agreed, that’s what I was going to suggest.
>
> That these init functions are not marked as internal is really an
> oversight.
>
> I wouldn’t be against simply making them SCM_INTERNAL in 2.0.12, but
> making them public + deprecated as you suggest is even better.

WDYT about something like this?


[-- Attachment #2: 0001-Register-R6RS-port-and-bytevector-internals-early.patch --]
[-- Type: text/plain, Size: 4644 bytes --]

From f763f353e438de985577ef4b88b805bd4d4c9b77 Mon Sep 17 00:00:00 2001
From: Andy Wingo <wingo@pobox.com>
Date: Thu, 23 Jun 2016 18:31:55 +0200
Subject: [PATCH] Register R6RS port and bytevector internals early

* libguile/bytevectors.c (sym_big, sym_little): Rename from scm_sym_big
  and scm_sym_little, and don't use the snarf mechanism as we need to
  initialize this value eagerly in case the C API is used before the
  Scheme module is loaded.
  (scm_bootstrap_bytevectors): Initialize the endianness symbols here.
* libguile/r6rs-ports.c (scm_register_r6rs_ports): Register the R6RS
  port kinds here, for the same reason.
---
 libguile/bytevectors.c | 24 ++++++++++++------------
 libguile/r6rs-ports.c  | 12 ++++++------
 2 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/libguile/bytevectors.c b/libguile/bytevectors.c
index cb7f294..86d1bdf 100644
--- a/libguile/bytevectors.c
+++ b/libguile/bytevectors.c
@@ -418,8 +418,8 @@ scm_i_print_bytevector (SCM bv, SCM port, scm_print_state *pstate SCM_UNUSED)
 \f
 /* General operations.  */
 
-SCM_SYMBOL (scm_sym_big, "big");
-SCM_SYMBOL (scm_sym_little, "little");
+static SCM sym_big;
+static SCM sym_little;
 
 SCM scm_endianness_big, scm_endianness_little;
 
@@ -799,13 +799,13 @@ bytevector_large_ref (const char *c_bv, size_t c_size, int signed_p,
 
   if (signed_p)
     {
-      if (scm_is_eq (endianness, scm_sym_big))
+      if (scm_is_eq (endianness, sym_big))
 	negative_p = c_bv[0] & 0x80;
       else
 	negative_p = c_bv[c_size - 1] & 0x80;
     }
 
-  c_endianness = scm_is_eq (endianness, scm_sym_big) ? 1 : -1;
+  c_endianness = scm_is_eq (endianness, sym_big) ? 1 : -1;
 
   mpz_init (c_mpz);
   mpz_import (c_mpz, 1 /* 1 word */, 1 /* word order doesn't matter */,
@@ -832,7 +832,7 @@ bytevector_large_set (char *c_bv, size_t c_size, int signed_p,
   mpz_t c_mpz;
   int c_endianness, c_sign, err = 0;
 
-  c_endianness = scm_is_eq (endianness, scm_sym_big) ? 1 : -1;
+  c_endianness = scm_is_eq (endianness, sym_big) ? 1 : -1;
 
   mpz_init (c_mpz);
   scm_to_mpz (value, c_mpz);
@@ -1864,9 +1864,9 @@ utf_encoding_name (char *name, size_t utf_width, SCM endianness)
 		       ? "32"
 		       : "??"))));
   strcat (name,
-	  ((scm_is_eq (endianness, scm_sym_big))
+	  ((scm_is_eq (endianness, sym_big))
 	   ? "BE"
-	   : ((scm_is_eq (endianness, scm_sym_little))
+	   : ((scm_is_eq (endianness, sym_little))
 	      ? "LE"
 	      : "unknown")));
 }
@@ -1884,7 +1884,7 @@ utf_encoding_name (char *name, size_t utf_width, SCM endianness)
                                                                         \
   SCM_VALIDATE_STRING (1, str);                                         \
   if (scm_is_eq (endianness, SCM_UNDEFINED))                            \
-    endianness = scm_sym_big;                                           \
+    endianness = sym_big;                                           \
   else                                                                  \
     SCM_VALIDATE_SYMBOL (2, endianness);                                \
                                                                         \
@@ -2001,7 +2001,7 @@ SCM_DEFINE (scm_string_to_utf32, "string->utf32",
 									\
   SCM_VALIDATE_BYTEVECTOR (1, utf);					\
   if (scm_is_eq (endianness, SCM_UNDEFINED))                            \
-    endianness = scm_sym_big;						\
+    endianness = sym_big;						\
   else									\
     SCM_VALIDATE_SYMBOL (2, endianness);				\
 									\
@@ -2266,13 +2266,13 @@ scm_bootstrap_bytevectors (void)
       (scm_i_array_element_types[SCM_ARRAY_ELEMENT_TYPE_VU8],
        scm_make_bytevector);
   }
+
+  scm_endianness_big = sym_big = scm_from_latin1_symbol ("big");
+  scm_endianness_little = sym_little = scm_from_latin1_symbol ("little");
 }
 
 void
 scm_init_bytevectors (void)
 {
 #include "libguile/bytevectors.x"
-
-  scm_endianness_big = scm_sym_big;
-  scm_endianness_little = scm_sym_little;
 }
diff --git a/libguile/r6rs-ports.c b/libguile/r6rs-ports.c
index bd8a1c4..5d283d2 100644
--- a/libguile/r6rs-ports.c
+++ b/libguile/r6rs-ports.c
@@ -1400,12 +1400,6 @@ scm_register_r6rs_ports (void)
                             "scm_init_r6rs_ports",
 			    (scm_t_extension_init_func) scm_init_r6rs_ports,
 			    NULL);
-}
-
-void
-scm_init_r6rs_ports (void)
-{
-#include "libguile/r6rs-ports.x"
 
   initialize_bytevector_input_ports ();
   initialize_custom_binary_input_ports ();
@@ -1413,3 +1407,9 @@ scm_init_r6rs_ports (void)
   initialize_custom_binary_output_ports ();
   initialize_transcoded_ports ();
 }
+
+void
+scm_init_r6rs_ports (void)
+{
+#include "libguile/r6rs-ports.x"
+}
-- 
2.8.3

bug#20209: GUILE 2.0.11: crash in set_port_filename_x for bytevector ports

[David Kastrup]

Andy Wingo <wingo@pobox.com> writes:

> On Sun 29 Mar 2015 15:06, ludo@gnu.org (Ludovic Courtès) writes:
>
>> Mark H Weaver <mhw@netris.org> skribis:
>>
>>> For these reasons, I decided against recommending those undocumented
>>> initialization functions.  Instead, I suggested that David initialize
>>> binary ports by loading (ice-9 binary-ports):
>>>
>>>   (void) scm_c_resolve_module ("ice-9 binary-ports");
>>
>> That sounds like the best option, indeed.
>>
>>> For Guile 2.0.12, I suggest that we unconditionally do the equivalent of
>>> 'scm_init_bytevectors' and 'scm_init_r6rs_ports' during Guile
>>> initialization, and make those functions into deprecated no-ops.
>>
>> Agreed, that’s what I was going to suggest.
>>
>> That these init functions are not marked as internal is really an
>> oversight.
>>
>> I wouldn’t be against simply making them SCM_INTERNAL in 2.0.12, but
>> making them public + deprecated as you suggest is even better.
>
> WDYT about something like this?

I don't see anything protecting sym_big or sym_little (more accurately,
'big or 'little which are non-immediate SCM values) from collection
which would make sym_big and sym_little useless for comparison.

I'm assuming that not the whole bss segment is getting scanned by
BoehmGC.

-- 
David Kastrup

bug#20209: GUILE 2.0.11: crash in set_port_filename_x for bytevector ports

[Andy Wingo]

On Thu 23 Jun 2016 18:59, David Kastrup <dak@gnu.org> writes:

> I don't see anything protecting sym_big or sym_little (more accurately,
> 'big or 'little which are non-immediate SCM values) from collection
> which would make sym_big and sym_little useless for comparison.
>
> I'm assuming that not the whole bss segment is getting scanned by
> BoehmGC.

The whole bss segment is scanned by the GC :)  See the third paragraph
here.

  https://www.gnu.org/software/guile/docs/master/guile.html/Garbage-Collection.html#Garbage-Collection

This section applies to 2.0 as well.

Andy

bug#20209: GUILE 2.0.11: crash in set_port_filename_x for bytevector ports

[David Kastrup]

Andy Wingo <wingo@pobox.com> writes:

> On Thu 23 Jun 2016 18:59, David Kastrup <dak@gnu.org> writes:
>
>> I don't see anything protecting sym_big or sym_little (more accurately,
>> 'big or 'little which are non-immediate SCM values) from collection
>> which would make sym_big and sym_little useless for comparison.
>>
>> I'm assuming that not the whole bss segment is getting scanned by
>> BoehmGC.
>
> The whole bss segment is scanned by the GC :)  See the third paragraph
> here.
>
>   https://www.gnu.org/software/guile/docs/master/guile.html/Garbage-Collection.html#Garbage-Collection
>
> This section applies to 2.0 as well.

Sigh.

    In practice, the wasted memory seems to be no problem, as the static
    C root set is almost always finite and small, given that the Scheme
    stack is separate from the C stack."

"almost always" means "for trivial applications and/or applications only
written in Scheme".  But Guile is marketed as an _extension_ language.

LilyPond has a table-driven lexer and parser and quite a bit of other
static data.

A quick peek with ld shows

.bss            0x0000000000aa2340    0x21218
.data           0x0000000000aa1080     0x12a0

Ok, .data seems absurdly compact for a table-driven parser with
something like 800 states alone, let alone all the rest.  So I might
have problems interpreting the output.

At any rate, .bss does not look exactly small.

Obviously, I am not overly happy with the design choices and rationale
taken for Guile-2.x memory management.

    These “false positives” might keep SCM objects alive that would
    otherwise be considered dead. While this might waste memory, keeping
    an object around longer than it strictly needs to is harmless.

Except that a single object may keep a whole lot of other stuff alive.
LilyPond is usually called once for compiling literally thousands of
files in one session sequentially.  It makes sure to call garbage
collection explicitly between files, when the return stack is indeed
guaranteed to be quite small.

Using Guile-1.8 we actually track non-freed objects at that time in our
testing versions (by setting a flag that will let the mark hooks
complain for object types that should not be around at that time).  It
does deliver false positives but at a pretty moderate rate: I'd guess
about 5% or so of complete runs through the regression test database are
affected.  We won't be able to do these tests with Guile-2 anyway since
they format output in the marking phase and that crashes the GC
subsystem (it's likely not kosher in Guile-1 either but the GC character
is more sequential there).

I doubt that the false positive rate would stay that low when
conservatively marking all the data segments rather than just the stack
at a known low mark.

And I don't really see how to optimize for application layout patterns
_not_ matching Guile's assumptions which again takes us to the question
how much an extension language should determine the design of the main
application.

Obviously, I have no problem with Guile deciding to conservatively scan
the libguile data segments: the layout and size of those are under its
own control.

Either way, sym_big and sym_little seem to be covered.

-- 
David Kastrup

bug#20209: GUILE 2.0.11: crash in set_port_filename_x for bytevector ports

[Andy Wingo]

On Thu 23 Jun 2016 18:34, Andy Wingo <wingo@pobox.com> writes:

> From: Andy Wingo <wingo@pobox.com>
> Date: Thu, 23 Jun 2016 18:31:55 +0200
> Subject: [PATCH] Register R6RS port and bytevector internals early
>
> * libguile/bytevectors.c (sym_big, sym_little): Rename from scm_sym_big
>   and scm_sym_little, and don't use the snarf mechanism as we need to
>   initialize this value eagerly in case the C API is used before the
>   Scheme module is loaded.
>   (scm_bootstrap_bytevectors): Initialize the endianness symbols here.
> * libguile/r6rs-ports.c (scm_register_r6rs_ports): Register the R6RS
>   port kinds here, for the same reason.

Already applied to 2.0; now applied to master too.

Andy