From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: David Kastrup Newsgroups: gmane.lisp.guile.bugs Subject: bug#18520: string ports should not have an encoding Date: Mon, 22 Sep 2014 19:20:54 +0200 Message-ID: <87bnq7lgg9.fsf@fencepost.gnu.org> References: <87iokgmttc.fsf@fencepost.gnu.org> <87mw9rq20u.fsf@gnu.org> <87sijjlqx0.fsf@fencepost.gnu.org> <87sijjmvlr.fsf@gnu.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: ger.gmane.org 1411407441 17930 80.91.229.3 (22 Sep 2014 17:37:21 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Mon, 22 Sep 2014 17:37:21 +0000 (UTC) Cc: 18520@debbugs.gnu.org To: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-X-From: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Mon Sep 22 19:37:15 2014 Return-path: Envelope-to: guile-bugs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1XW7Y1-0002Db-VE for guile-bugs@m.gmane.org; Mon, 22 Sep 2014 19:37:14 +0200 Original-Received: from localhost ([::1]:47950 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XW7Y1-0004aa-J8 for guile-bugs@m.gmane.org; Mon, 22 Sep 2014 13:37:13 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:44897) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XW7Xx-0004Zd-7X for bug-guile@gnu.org; Mon, 22 Sep 2014 13:37:10 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XW7Xv-0000xo-TU for bug-guile@gnu.org; Mon, 22 Sep 2014 13:37:09 -0400 Original-Received: from debbugs.gnu.org ([140.186.70.43]:57519) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XW7Xv-0000wd-Qg for bug-guile@gnu.org; Mon, 22 Sep 2014 13:37:07 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.80) (envelope-from ) id 1XW7IM-0005AO-1s for bug-guile@gnu.org; Mon, 22 Sep 2014 13:21:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: David Kastrup Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Mon, 22 Sep 2014 17:21:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 18520 X-GNU-PR-Package: guile X-GNU-PR-Keywords: Original-Received: via spool by 18520-submit@debbugs.gnu.org id=B18520.141140646019842 (code B ref 18520); Mon, 22 Sep 2014 17:21:02 +0000 Original-Received: (at 18520) by debbugs.gnu.org; 22 Sep 2014 17:21:00 +0000 Original-Received: from localhost ([127.0.0.1]:49075 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XW7IJ-00059y-Fe for submit@debbugs.gnu.org; Mon, 22 Sep 2014 13:20:59 -0400 Original-Received: from fencepost.gnu.org ([208.118.235.10]:35519) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XW7IG-00059p-LT for 18520@debbugs.gnu.org; Mon, 22 Sep 2014 13:20:57 -0400 Original-Received: from localhost ([127.0.0.1]:42824 helo=lola) by fencepost.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XW7IF-0000ou-If; Mon, 22 Sep 2014 13:20:56 -0400 Original-Received: by lola (Postfix, from userid 1000) id 6CE69E620D; Mon, 22 Sep 2014 19:20:54 +0200 (CEST) In-Reply-To: <87sijjmvlr.fsf@gnu.org> ("Ludovic =?UTF-8?Q?Court=C3=A8s?="'s message of "Mon, 22 Sep 2014 19:08:16 +0200") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.4.50 (gnu/linux) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 140.186.70.43 X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Original-Sender: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.bugs:7572 Archived-At: ludo@gnu.org (Ludovic Court=C3=A8s) writes: > David Kastrup skribis: > >> I'm currently migrating LilyPond over to GUILE 2.0. LilyPond has its >> own UTF-8 verification, error flagging, processing and indexing. > > Do I understand correctly that LilyPond expects Guile strings to be byte > vectors, which it can feed with UTF-8 byte sequences that it built by > itself? Not really. LilyPond reads and parses its own files but it does divert parts through GUILE occasionally in the process. Some stuff is passed through GUILE with time delays and parts wrapped into closures and flagged with machine-identifiable source locations. >> If you take a look at >> , >> ftell on a string port is here used for correlating the positions of >> parsed subexpressions with the original data. Reencoding strings in >> utf-8 is not going to make this work with string indexing since ftell >> does not bear a useful relation to string positions. > > AIUI the result of =E2=80=98ftell=E2=80=99 is used in only one place, whi= le > =E2=80=98port-line=E2=80=99 and =E2=80=98port-column=E2=80=99 are used in= other places. The ftell information is wrapped into an alist together with a closure corresponding to the source location. At a later point of time, the surrounding string may be interpreted, and the source location is correlated with the closure and the closure used instead of a call to local-eval (which does not have the same power of evaluating materials in a preserved lexical environment as a closure has). > The latter seems more appropriate to me when it comes to tracking > source location. For error messages, yes. For associating a position in a string with a previously parsed closure, no. > How is the result of =E2=80=98ftell=E2=80=99 used by callers of > =E2=80=98read-lily-expression=E2=80=99? See above. >> I=C2=A0have more than enough crashes and obscure errors to contend with = as >> it stands, > > Could you open a separate bug with the backtrace of such crashes, if you > think it may be Guile=E2=80=99s fault? The backtraces are usually quite useless for diagnosing the crashes. For example, there are crashes in scm_sloppy_assq. If you look at the code, it is clear that they can only happen for pairs that have already been collected by garbage collection. So the bug has occured quite a bit previously to the crash. So one has to figure out how the collection could possibly have happened (naturally, it didn't with GUILE 1.8). You can try doing that with the rather expensive process of "reverse execution" (which basically traces and keeps a history you can then explore backwards from the crash), but that requires that the bugs are reproducible, and with collection in a separate thread, that is not really the case. Sometimes a crash segfaults, more often you get std::exception triggered. All with the same input and executable. --=20 David Kastrup