From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: David Kastrup Newsgroups: gmane.lisp.guile.bugs Subject: bug#19883: Correction for backtrace Date: Thu, 26 Feb 2015 16:30:28 +0100 Message-ID: <87wq34bsh7.fsf@fencepost.gnu.org> References: <87twyln70f.fsf@fencepost.gnu.org> <873865n2rr.fsf@fencepost.gnu.org> <87twy91vtc.fsf@gnu.org> <878ufld1iw.fsf@fencepost.gnu.org> <87k2z4dhx7.fsf@gnu.org> <874mq8dfb3.fsf@fencepost.gnu.org> <87fv9sahxx.fsf@gnu.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: ger.gmane.org 1424964690 5677 80.91.229.3 (26 Feb 2015 15:31:30 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 26 Feb 2015 15:31:30 +0000 (UTC) Cc: 19883@debbugs.gnu.org To: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-X-From: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Thu Feb 26 16:31:20 2015 Return-path: Envelope-to: guile-bugs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1YR0P8-0003mS-FO for guile-bugs@m.gmane.org; Thu, 26 Feb 2015 16:31:10 +0100 Original-Received: from localhost ([::1]:59584 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YR0P7-0004x0-RD for guile-bugs@m.gmane.org; Thu, 26 Feb 2015 10:31:09 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:57287) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YR0P2-0004uP-0b for bug-guile@gnu.org; Thu, 26 Feb 2015 10:31:07 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YR0P0-0005cY-L7 for bug-guile@gnu.org; Thu, 26 Feb 2015 10:31:03 -0500 Original-Received: from debbugs.gnu.org ([140.186.70.43]:55246) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YR0P0-0005cU-HQ for bug-guile@gnu.org; Thu, 26 Feb 2015 10:31:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.80) (envelope-from ) id 1YR0P0-0006lg-7V for bug-guile@gnu.org; Thu, 26 Feb 2015 10:31:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: David Kastrup Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Thu, 26 Feb 2015 15:31:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 19883 X-GNU-PR-Package: guile X-GNU-PR-Keywords: Original-Received: via spool by 19883-submit@debbugs.gnu.org id=B19883.142496463425976 (code B ref 19883); Thu, 26 Feb 2015 15:31:02 +0000 Original-Received: (at 19883) by debbugs.gnu.org; 26 Feb 2015 15:30:34 +0000 Original-Received: from localhost ([127.0.0.1]:58844 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YR0OX-0006kt-L7 for submit@debbugs.gnu.org; Thu, 26 Feb 2015 10:30:34 -0500 Original-Received: from fencepost.gnu.org ([208.118.235.10]:51552 ident=Debian-exim) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YR0OU-0006kk-El for 19883@debbugs.gnu.org; Thu, 26 Feb 2015 10:30:31 -0500 Original-Received: from localhost ([127.0.0.1]:58856 helo=lola) by fencepost.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YR0OT-0003Zl-6F; Thu, 26 Feb 2015 10:30:29 -0500 Original-Received: by lola (Postfix, from userid 1000) id 93259E0575; Thu, 26 Feb 2015 16:30:28 +0100 (CET) In-Reply-To: <87fv9sahxx.fsf@gnu.org> ("Ludovic =?UTF-8?Q?Court=C3=A8s?="'s message of "Thu, 26 Feb 2015 15:03:22 +0100") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.0.50 (gnu/linux) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 140.186.70.43 X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Original-Sender: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.bugs:7730 Archived-At: ludo@gnu.org (Ludovic Court=C3=A8s) writes: > David Kastrup skribis: > >> Is there a reason you are not using the test program provided with this >> bug report? > > Sorry, I had overlooked it. Please assume good faith. > > With this patch: > > diff --git a/smobs.tcc b/smobs.tcc > index c8b9dfe..b1d61b7 100644 > --- a/smobs.tcc > +++ b/smobs.tcc > @@ -132,9 +132,6 @@ void Smob_base::init () > // types in order to be able to compare them. The other comparisons > // are for static member functions and thus are ordinary function > // pointers which work without those contortions. > - if (static_cast(&Super::mark_smob) !=3D > - static_cast(&Smob_base::mark_smob)) > - scm_set_smob_mark (smob_tag_, Super::mark_trampoline); > scm_set_smob_print (smob_tag_, Super::print_trampoline); > if (&Super::equal_p !=3D &Smob_base::equal_p) > scm_set_smob_equalp (smob_tag_, Super::equal_p); > > > =E2=80=9C./test 1000 1000 10000=E2=80=9D works fine. That's not really surprising: this specifies a "fork factor" of 1000 and 1000 points of data which means that the data structures are nested only one level deep. Try ./test 2 2000 200 which will cause a nesting of about 10 levels, making it much more likely for collection/allocation problems to strike. > However, it=E2=80=99s not representative since the C++ objects in this te= st do > not hold =E2=80=98SCM=E2=80=99 references, AFAICS. Why isn't it representative? That's _very_ much the situation in LilyPond's code base. >> LilyPond's GUILEv2 branch is currently out of order again since >> 2.0.11 changed encoding mechanisms _again_ > > Please submit a different bug report. There were separate bug reports. This is considered a feature. >> It would not help since many of the references are stored in STL >> containers (like std::vector ) which have their data >> allocated/deallocated separately from the memory area of the >> structure itself. > > Oh, OK. Still, I don=E2=80=99t think this is a problem because each C++ > object has a corresponding SMOB, and said SMOB is GC-protected; thus > the C++ object itself is also GC-protected until the SMOB is > unprotected. The code given in test.cc is representative for LilyPond: most of the C++ objects refer to other C++ objects via pointers, and the protection of SMOB and C++ objects is managed through the mark callbacks. Complex C++ objects contain their own SCM value as part of the Smob base class. Simple C++ objects (derived from Simple_smob) don't and are only optionally managed by GUILE. The test program currently does not demonstrate use of Simple_smob: at the current point of time getting the crashes of Smob under control is the first target. > Here=E2=80=99s the patch I=E2=80=99ve ended up with: > > diff --git a/smobs.hh b/smobs.hh > index 3701280..a41a645 100644 > --- a/smobs.hh > +++ b/smobs.hh > @@ -263,6 +263,20 @@ private: > protected: > Smob () : self_scm_ (SCM_UNDEFINED), protection_cons_ (SCM_EOL) { }; > public: > + static void *operator new (std::size_t size) > + { > + /* This C++ object is referred to by the corresponding SMOB, which is > + itself GC-protected. Thus, no need to protect the C++ object. */ > + return scm_gc_malloc (size, "lily-smob"); > + } > + > + static void operator delete (void *thing) > + { > + /* Nothing to do: the GC will reclaim memory for THING when it deems > + appropriate. */ > + // printf ("delete %p\n", thing); > + } > + As I stated: this will not help with STL containers which are extensively used in pretty much every data structure of LilyPond. >> Frankly, I don't get the current strategy of GUILE: basically any use >> of scm_set_smob_mark will result in a function that can be called >> with garbage from a smob that has already been deallocated via the >> function registered with scm_set_smob_free. > > That=E2=80=99s not true. Shrug. I posted the backtrace and the example program. > Let me explain what my personal expectations are. First, like many > others, I=E2=80=99m a volunteer with limited bandwidth. Thus, I really p= refer > bug reports that are as concise as possible and to-the-point; I can=E2=80= =99t > afford to read so much. That's what I consider the primary reason that every offer to help with LilyPond so far has gone dead the moment I provided a branch and recipe to work on. LilyPond is a large and complex application. That's why I=C2=A0went to the pain of providing standalone code demonstrating the memo= ry allocation and management problems so that this could be worked on independently from other problems. > Second, I don=E2=80=99t want to mix issues: this bug report is about a > specific GC issue, not about an encoding issue. It=E2=80=99s better for = me > (and everyone I guess) when a bug report remains focused. Which is why I provided a separate test program that does not break whenever the "stable" Guile version changes its encoding APIs again. Several weeks ago I promised RMS to update the GUILEv2 branch to the current version of LilyPond as he got a renewed promise of cooperation. It turned out that in the mean time Ubuntu had updated Guilev2 from 2.0.9 (or something) to 2.0.11, and the encoding issues I got under control for 2.0.9 were back with a vengeance where a speedy solution was not in sight. So I spent the time to create test code _only_ involving the memory management issues. Now the complaint appears to be that the code is not representative for LilyPond. It most certainly is. > Third, we must fix these LilyPond vs. Guile 2 issues. You and others > have spent a lot of time on this already. In the last two years, nobody except myself spent any time on GUILEv2/LilyPond. Previous to that, Ian Hulin spent time on reorganizing our code base and startup code in order not to get the byte compilation, module system and other interpreter/compiler differences of GUILEv2 mess up the basic operation, mostly by disabling a lot of stuff. There will be a lot of followup work to get code management and performance into reasonable shape once it works at all. When it is not possible to run the test suites without random crashes, there is no reasonably effective way for people to work on those aspects however. > I think it would help to get everyone involved on both sides. Thus, > could you Cc: this bug report to the LilyPond developer list, or the > corresponding LilyPond bug report? This is really important to me. Shrug. I'll put a link to this bug report to a suitable LilyPond issue. It will not likely trigger a response or other feedback, however. Nobody but myself is working on GUILEv2 migration. The current C++ template class implementation/abstraction of the GUILE memory management has been written by me. No other currently active developer has had either significant involvement with the current code, or the functionally similar previous code that used C preprocessor macros instead of language features to do essentially the same. > Please don=E2=80=99t take it personally or anything. I=E2=80=99m really = trying hard > to see how we can improve the collaboration between the two projects. For better or worse, LilyPond has no other developer to offer than myself for that task. There is nobody else available who even has had exposure to GUILEv2 or even GUILE1 internals. --=20 David Kastrup