From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: David Kastrup Newsgroups: gmane.lisp.guile.bugs Subject: bug#20209: GUILE 2.0.11: crash in set_port_filename_x for bytevector ports Date: Thu, 23 Jun 2016 20:45:51 +0200 Message-ID: <87oa6reobk.fsf@fencepost.gnu.org> References: <87fv8r1pio.fsf@fencepost.gnu.org> <87egob8irs.fsf@netris.org> <87y4mhasqb.fsf@netris.org> <87twx40x7c.fsf@gnu.org> <87r3bnn9tq.fsf@pobox.com> <87shw3et9j.fsf@fencepost.gnu.org> <87eg7nn5sk.fsf@pobox.com> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: ger.gmane.org 1466707644 4316 80.91.229.3 (23 Jun 2016 18:47:24 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 23 Jun 2016 18:47:24 +0000 (UTC) Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , 20209@debbugs.gnu.org To: Andy Wingo Original-X-From: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Thu Jun 23 20:47:15 2016 Return-path: Envelope-to: guile-bugs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1bG9ek-0005by-KD for guile-bugs@m.gmane.org; Thu, 23 Jun 2016 20:47:14 +0200 Original-Received: from localhost ([::1]:38738 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bG9ej-0005Ml-Vf for guile-bugs@m.gmane.org; Thu, 23 Jun 2016 14:47:14 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:53831) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bG9ea-0005JK-Ch for bug-guile@gnu.org; Thu, 23 Jun 2016 14:47:05 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bG9eY-0003sD-33 for bug-guile@gnu.org; Thu, 23 Jun 2016 14:47:03 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:40642) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bG9eX-0003s8-Vr for bug-guile@gnu.org; Thu, 23 Jun 2016 14:47:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1bG9eX-0003Gj-PV for bug-guile@gnu.org; Thu, 23 Jun 2016 14:47:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: David Kastrup Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Thu, 23 Jun 2016 18:47:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20209 X-GNU-PR-Package: guile X-GNU-PR-Keywords: Original-Received: via spool by 20209-submit@debbugs.gnu.org id=B20209.146670756212480 (code B ref 20209); Thu, 23 Jun 2016 18:47:01 +0000 Original-Received: (at 20209) by debbugs.gnu.org; 23 Jun 2016 18:46:02 +0000 Original-Received: from localhost ([127.0.0.1]:52979 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bG9da-0003FE-Gk for submit@debbugs.gnu.org; Thu, 23 Jun 2016 14:46:02 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:34260) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bG9dY-0003Eh-JY for 20209@debbugs.gnu.org; Thu, 23 Jun 2016 14:46:00 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bG9dR-0003ew-PY for 20209@debbugs.gnu.org; Thu, 23 Jun 2016 14:45:55 -0400 Original-Received: from fencepost.gnu.org ([2001:4830:134:3::e]:41544 helo=lola.localdomain) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bG9dR-0003ek-Gk; Thu, 23 Jun 2016 14:45:53 -0400 Original-Received: by lola.localdomain (Postfix, from userid 1000) id C3921DF783; Thu, 23 Jun 2016 20:45:51 +0200 (CEST) In-Reply-To: <87eg7nn5sk.fsf@pobox.com> (Andy Wingo's message of "Thu, 23 Jun 2016 20:01:15 +0200") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Original-Sender: "bug-guile" Xref: news.gmane.org gmane.lisp.guile.bugs:8178 Archived-At: Andy Wingo writes: > On Thu 23 Jun 2016 18:59, David Kastrup writes: > >> I don't see anything protecting sym_big or sym_little (more accurately, >> 'big or 'little which are non-immediate SCM values) from collection >> which would make sym_big and sym_little useless for comparison. >> >> I'm assuming that not the whole bss segment is getting scanned by >> BoehmGC. > > The whole bss segment is scanned by the GC :) See the third paragraph > here. > > https://www.gnu.org/software/guile/docs/master/guile.html/Garbage-Colle= ction.html#Garbage-Collection > > This section applies to 2.0 as well. Sigh. In practice, the wasted memory seems to be no problem, as the static C root set is almost always finite and small, given that the Scheme stack is separate from the C stack." "almost always" means "for trivial applications and/or applications only written in Scheme". But Guile is marketed as an _extension_ language. LilyPond has a table-driven lexer and parser and quite a bit of other static data. A quick peek with ld shows .bss 0x0000000000aa2340 0x21218 .data 0x0000000000aa1080 0x12a0 Ok, .data seems absurdly compact for a table-driven parser with something like 800 states alone, let alone all the rest. So I might have problems interpreting the output. At any rate, .bss does not look exactly small. Obviously, I am not overly happy with the design choices and rationale taken for Guile-2.x memory management. These =E2=80=9Cfalse positives=E2=80=9D might keep SCM objects alive th= at would otherwise be considered dead. While this might waste memory, keeping an object around longer than it strictly needs to is harmless. Except that a single object may keep a whole lot of other stuff alive. LilyPond is usually called once for compiling literally thousands of files in one session sequentially. It makes sure to call garbage collection explicitly between files, when the return stack is indeed guaranteed to be quite small. Using Guile-1.8 we actually track non-freed objects at that time in our testing versions (by setting a flag that will let the mark hooks complain for object types that should not be around at that time). It does deliver false positives but at a pretty moderate rate: I'd guess about 5% or so of complete runs through the regression test database are affected. We won't be able to do these tests with Guile-2 anyway since they format output in the marking phase and that crashes the GC subsystem (it's likely not kosher in Guile-1 either but the GC character is more sequential there). I doubt that the false positive rate would stay that low when conservatively marking all the data segments rather than just the stack at a known low mark. And I don't really see how to optimize for application layout patterns _not_ matching Guile's assumptions which again takes us to the question how much an extension language should determine the design of the main application. Obviously, I have no problem with Guile deciding to conservatively scan the libguile data segments: the layout and size of those are under its own control. Either way, sym_big and sym_little seem to be covered. --=20 David Kastrup