From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Mark H Weaver Newsgroups: gmane.lisp.guile.bugs Subject: bug#22667: string->bytevector encoding error Date: Thu, 03 Mar 2016 12:55:55 -0500 Message-ID: <87oaavsaxg.fsf@netris.org> References: <56C0FD28.9050709@primfilat.com> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1457027843 30863 80.91.229.3 (3 Mar 2016 17:57:23 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 3 Mar 2016 17:57:23 +0000 (UTC) Cc: 22667@debbugs.gnu.org, Ludovic =?UTF-8?Q?Court=C3=A8s?= To: Josep Portella Florit Original-X-From: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Thu Mar 03 18:57:14 2016 Return-path: Envelope-to: guile-bugs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1abXUu-0002Bx-Q9 for guile-bugs@m.gmane.org; Thu, 03 Mar 2016 18:57:12 +0100 Original-Received: from localhost ([::1]:36618 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1abXUq-0006mL-Sx for guile-bugs@m.gmane.org; Thu, 03 Mar 2016 12:57:08 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:36706) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1abXUn-0006lo-93 for bug-guile@gnu.org; Thu, 03 Mar 2016 12:57:06 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1abXUj-0000U6-Uq for bug-guile@gnu.org; Thu, 03 Mar 2016 12:57:05 -0500 Original-Received: from debbugs.gnu.org ([208.118.235.43]:35071) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1abXUj-0000Ty-RS for bug-guile@gnu.org; Thu, 03 Mar 2016 12:57:01 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84) (envelope-from ) id 1abXUj-0005p6-L9 for bug-guile@gnu.org; Thu, 03 Mar 2016 12:57:01 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Mark H Weaver Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Thu, 03 Mar 2016 17:57:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 22667 X-GNU-PR-Package: guile X-GNU-PR-Keywords: Original-Received: via spool by 22667-submit@debbugs.gnu.org id=B22667.145702777522329 (code B ref 22667); Thu, 03 Mar 2016 17:57:01 +0000 Original-Received: (at 22667) by debbugs.gnu.org; 3 Mar 2016 17:56:15 +0000 Original-Received: from localhost ([127.0.0.1]:60431 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1abXTz-0005o5-Lr for submit@debbugs.gnu.org; Thu, 03 Mar 2016 12:56:15 -0500 Original-Received: from world.peace.net ([50.252.239.5]:38296 ident=hope5) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1abXTx-0005nx-HS for 22667@debbugs.gnu.org; Thu, 03 Mar 2016 12:56:13 -0500 Original-Received: from pool-71-174-32-69.bstnma.east.verizon.net ([71.174.32.69] helo=jojen) by world.peace.net with esmtpsa (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.72) (envelope-from ) id 1abXTr-0005Kr-12; Thu, 03 Mar 2016 12:56:07 -0500 In-Reply-To: <56C0FD28.9050709@primfilat.com> (Josep Portella Florit's message of "Sun, 14 Feb 2016 23:18:16 +0100") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.0.91 (gnu/linux) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Original-Sender: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.bugs:7973 Archived-At: Josep Portella Florit writes: > 'string->bytevector' throws 'encoding-error' when the encoding is > "UTF-16" and the length of the string is greater than 128. The same > for the encoding "UTF-32" and a string of length greater than 64. > > Tested on Guile 2.0.11 and 2.1.2 with the same result. > > How to reproduce: > > (use-modules (ice-9 iconv)) > > ;; These expressions evaluate as expected: > (string->bytevector (make-string 128 #\A) "UTF-16") > ;; => #vu8(254 255 0 65 0 65 ...) > (string->bytevector (make-string 64 #\A) "UTF-32") > ;; => #vu8(0 0 254 255 0 0 0 65 0 0 0 65 ...) > > ;; These expressions throw 'encoding-error': > (string->bytevector (make-string 129 #\A) "UTF-16") > (string->bytevector (make-string 65 #\A) "UTF-32") Thee bug is in 'display_string_using_iconv', introduced in commit f4bc4e5934, which improperly assumes that a 256-byte buffer (encoded_output) will be large enough to hold the results of converting up to 256 bytes of UTF-8 (utf8_buf) to an arbitrary encoding using 'iconv'. When this assumption fails, the call to 'iconv' on line 997 done = iconv (id->output_cd, &input, &input_left, &output, &output_left); reports a failure due to the output buffer being too small, and the code fails to handle this case. To be continued... Mark