From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: taylanbayirli@gmail.com (Taylan Ulrich =?UTF-8?Q?Bay=C4=B1rl=C4=B1/Kammer?=) Newsgroups: gmane.lisp.guile.bugs Subject: bug#26058: utf16->string and utf32->string don't conform to R6RS Date: Mon, 13 Mar 2017 19:10:36 +0100 Message-ID: <87h92xyrmr.fsf@gmail.com> References: <87o9x83t0f.fsf@gmail.com> <87shmhqqgd.fsf@pobox.com> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: blaine.gmane.org 1489428273 17265 195.159.176.226 (13 Mar 2017 18:04:33 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Mon, 13 Mar 2017 18:04:33 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux) Cc: 26058@debbugs.gnu.org To: Andy Wingo Original-X-From: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Mon Mar 13 19:04:29 2017 Return-path: Envelope-to: guile-bugs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cnUKT-0003WI-TE for guile-bugs@m.gmane.org; Mon, 13 Mar 2017 19:04:22 +0100 Original-Received: from localhost ([::1]:53660 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cnUKa-0002yb-0m for guile-bugs@m.gmane.org; Mon, 13 Mar 2017 14:04:28 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:42118) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cnUKD-0002s7-EN for bug-guile@gnu.org; Mon, 13 Mar 2017 14:04:06 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cnUKA-0000Uk-8G for bug-guile@gnu.org; Mon, 13 Mar 2017 14:04:05 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:55982) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cnUKA-0000Ud-5D for bug-guile@gnu.org; Mon, 13 Mar 2017 14:04:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1cnUK9-0004cf-RM for bug-guile@gnu.org; Mon, 13 Mar 2017 14:04:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: taylanbayirli@gmail.com (Taylan Ulrich =?UTF-8?Q?Bay=C4=B1rl=C4=B1/Kammer?=) Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Mon, 13 Mar 2017 18:04:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 26058 X-GNU-PR-Package: guile X-GNU-PR-Keywords: Original-Received: via spool by 26058-submit@debbugs.gnu.org id=B26058.148942818417690 (code B ref 26058); Mon, 13 Mar 2017 18:04:01 +0000 Original-Received: (at 26058) by debbugs.gnu.org; 13 Mar 2017 18:03:04 +0000 Original-Received: from localhost ([127.0.0.1]:54180 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cnUJE-0004bG-Ab for submit@debbugs.gnu.org; Mon, 13 Mar 2017 14:03:04 -0400 Original-Received: from mail-wr0-f180.google.com ([209.85.128.180]:36442) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cnUJC-0004al-HE for 26058@debbugs.gnu.org; Mon, 13 Mar 2017 14:03:03 -0400 Original-Received: by mail-wr0-f180.google.com with SMTP id u108so108866328wrb.3 for <26058@debbugs.gnu.org>; Mon, 13 Mar 2017 11:03:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=1iDW5oFOoNWsCRBbDC3ZGY3DebRF/iMclwKml0m6qdY=; b=suqRNMC9aE1Rdsr2cR+BNQ4IAdeFynhrsbmi+PMpr7C3R36aesjLLIJRUOZySE3AUx xfVCR54B8LEvDRAyUh8cJ17bti4vUg2J1y7DzsWypxhgwQTRkRtlqlpfpJF4asL5wSal jRXMAjo8vFBzZAv9IoHV8xqzScSDUfEcCOaphiDu/vmTHwa8rqztW5GV7QzOSzYHwSao no9f5Uhk5/skDEc0dSlaWwtn8UCBvpmq+Gprm97bFBUTShOIYv0OGW5n0TqDmw3fLhok 4keHgzwONvWs3g/gu+AwGo3DRJ+/i10s8v99IVBJaXd7qs81KYAe1R4pex9NQC9MdHVl dWyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=1iDW5oFOoNWsCRBbDC3ZGY3DebRF/iMclwKml0m6qdY=; b=QZQ/uMzAlZ6NlBwk4WQNsfQKa+V0Y95159UUx6VPZqHUl+Cc4gycgO1+cMjynGpv2/ dTGSY+HZGiTu1XtfLVEg8NAyOEedN84GAK6wqoEIuT7AUV6Nx4uRXJi4nl8b9NDYWN4t t4t17+8BXKohxYZJKEmYPVFvGN+3LMmRE6hGMcGX796H8XsjCcbEz3dR77suLuM+0HEz sFw0wnqjuhfL1jx/kkcAbLRwwnMXB7NEbmOCe4B7KYF2DZBVaZLEf2e0ewWQIwfxBVlX l02J/MIdqVo8Oj/CebQsDgszKJmUQAyNOg7qX5HpKDXWENLN7HVkQ2Ph+Du76cA/F4BG K+tQ== X-Gm-Message-State: AMke39l+JrS6FxCz0kTfJdNybBmrnXCC8wYnGqeDrb3dqG3t4DurNeXWnfDq8bxdUgIXRA== X-Received: by 10.223.182.167 with SMTP id j39mr27559082wre.152.1489428176620; Mon, 13 Mar 2017 11:02:56 -0700 (PDT) Original-Received: from T420 ([2a02:908:c30:3540:221:ccff:fe66:68f0]) by smtp.gmail.com with ESMTPSA id j26sm25940374wrb.69.2017.03.13.11.02.55 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 13 Mar 2017 11:02:55 -0700 (PDT) In-Reply-To: <87shmhqqgd.fsf@pobox.com> (Andy Wingo's message of "Mon, 13 Mar 2017 14:03:14 +0100") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Original-Sender: "bug-guile" Xref: news.gmane.org gmane.lisp.guile.bugs:8675 Archived-At: Andy Wingo writes: > Hi, > > this is a tricky area that is not so amenable to quick patches :) Have > you looked into what Guile already does for byte-order marks? Can you > explain how the R6RS specification relates to this? > > https://www.gnu.org/software/guile/manual/html_node/BOM-Handling.html > > Andy Hmm, interesting. I noticed the utf{16,32}->string procedures ignoring a BOM at the start of the given bytevector, but didn't look at it from a ports perspective. TL;DR of the below though: the R6RS semantics offer a strict enrichment of the feature-set of the utfX->string procedures relative to the Guile semantics, so at most we would end up with spurious features. (The optional ability to handle any possible BOM at the start of the bytevector, with a fall-back endianness in case none is found.) That said, let's see... If I do a textual read from a port, I already get a string and not a bytevector, so the behavior of utfX->string operations is irrelevant. If I do binary I/O, the following situations are possible: 1. I'm guaranteed to get any possible bytes that happen to form a valid BOM at the start of the stream as-is in the returned bytevector; the binary I/O interface doesn't see such bytes as anything special, as it could simply be coincidence that the stream starts with such bytes. 2. I'm guaranteed *not* to get bytes that form a BOM at the start of the stream; instead they're consumed to set the port encoding for any future text I/O. 3. The behavior is unspecified and either of the above may happen. In the case of #1, it's probably good for utfX->string procedures to be able to handle BOMs, but also allow explicitly ignoring any possible BOM. The R6RS semantics cover this. In the case of #2, the utfX->string procedures don't need to be able to handle BOMs as far as we're talking about passing them bytevectors returned by port I/O, but it also doesn't hurt if they optionally support it. The R6RS semantics are fine here as well I think. As for #3... first of all it's bad IMO; the behavior ought to be specified. :-) But in any case, the additional features of the R6RS semantics won't hurt. WDYT? As far as I understand the page you linked, Guile currently implements #3, which I think is unfortunate but can kinda understand too. In any case, the additional R6RS features won't hurt, right? Taylan