From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.bugs Subject: bug#23750: 25.0.95; bug in url-retrieve or json.el Date: Mon, 20 Jun 2016 17:54:23 +0300 Message-ID: <367c9750-4aaa-e7cd-1a1a-1a61107fbb12@yandex.ru> References: <358304f6-98e1-fa10-8805-aa9b73db406a@yandex.ru> <83k2hl828z.fsf@gnu.org> <83h9co8twu.fsf@gnu.org> <838ty07wno.fsf@gnu.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Trace: ger.gmane.org 1466435329 28283 80.91.229.3 (20 Jun 2016 15:08:49 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Mon, 20 Jun 2016 15:08:49 +0000 (UTC) Cc: 23750@debbugs.gnu.org, monnier@IRO.UMontreal.CA, sdl.web@gmail.com To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Mon Jun 20 17:08:38 2016 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1bF0oX-0000LC-Q6 for geb-bug-gnu-emacs@m.gmane.org; Mon, 20 Jun 2016 17:08:38 +0200 Original-Received: from localhost ([::1]:44225 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bF0oW-0000t6-U5 for geb-bug-gnu-emacs@m.gmane.org; Mon, 20 Jun 2016 11:08:36 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:33245) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bF0bT-0002pr-78 for bug-gnu-emacs@gnu.org; Mon, 20 Jun 2016 10:55:09 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bF0bO-0000Sk-0t for bug-gnu-emacs@gnu.org; Mon, 20 Jun 2016 10:55:06 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:35503) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bF0bN-0000Sf-TM for bug-gnu-emacs@gnu.org; Mon, 20 Jun 2016 10:55:01 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1bF0bN-0008K9-Ma for bug-gnu-emacs@gnu.org; Mon, 20 Jun 2016 10:55:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 20 Jun 2016 14:55:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 23750 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 23750-submit@debbugs.gnu.org id=B23750.146643447331958 (code B ref 23750); Mon, 20 Jun 2016 14:55:01 +0000 Original-Received: (at 23750) by debbugs.gnu.org; 20 Jun 2016 14:54:33 +0000 Original-Received: from localhost ([127.0.0.1]:47840 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bF0av-0008JN-4r for submit@debbugs.gnu.org; Mon, 20 Jun 2016 10:54:33 -0400 Original-Received: from mail-wm0-f49.google.com ([74.125.82.49]:38453) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bF0at-0008JB-7Y for 23750@debbugs.gnu.org; Mon, 20 Jun 2016 10:54:31 -0400 Original-Received: by mail-wm0-f49.google.com with SMTP id r201so65639197wme.1 for <23750@debbugs.gnu.org>; Mon, 20 Jun 2016 07:54:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=puE6F8XzaIMmszW/qvW72AUsM69VHf9ISD3zoC1zZbY=; b=FFcyBL688Y7AOdtafR40R6era2NMT4quulVtoABszKN3eu2nZVoqjLErILeO4uS0H9 DItXc+7fOY1+RQxGyuCFrBI/YnwVLIGgkcGQ83eqQP8hzMWWrv6gMF+a2/LR1hnBjYZo /W+JfoHVTqip9QBxGeqSpano+MrI2ZpbGBmuOLfDaga6Eru+aPrZVKp9slKuCRpvihD7 QcjTpcX8+ishJpwP0bNYdC/CqyUAp5Wxwda5iyC3YZiz9NaMwlosxntInmfe4DWzvp/o LB53SOo8nuFjHQYBdLGdApx+1o8GP/ljXPzz+T0So8LdOKuo9mUKFWFhVGRe02PWy1zE Q1NA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=puE6F8XzaIMmszW/qvW72AUsM69VHf9ISD3zoC1zZbY=; b=ODpahLHcvXtHw4LZ3SfILQCchwhMCXcW2jbqgRErhzRmcNEJkxu12Z6j52lteAM4j1 BAXUFBPfSuMGrIdWaO1YSTGFUzddMUBkQWeVpTLpzkF1i49OOAEpVlTWhitbcH5ypip5 92kXxmurSsFoEktQJs0pbbRL72YU0+/iTrid9Ylbo1JExovK8T2OU/Q5DcwFFnh8qZus C6/WS9DDfGUE28WrX/+iAQimg/jmPkz5yu1lqsOEo0fszYHO/dxdhzNtzfcNTxXSUqPO AiyEITm8U97dse9kPUtKEIJWLRqr61jPkKjY20lfgh4ATzU40PkXboZ2Y/DBtkDiIpS1 xANQ== X-Gm-Message-State: ALyK8tJME1NKDF6p/bhoTodGZzDqbbm/YzR20Boi8RcUsm687OaXyOPGwUDKssI3jmPHeA== X-Received: by 10.194.94.69 with SMTP id da5mr9301421wjb.158.1466434465506; Mon, 20 Jun 2016 07:54:25 -0700 (PDT) Original-Received: from [192.168.1.2] ([185.105.173.135]) by smtp.googlemail.com with ESMTPSA id r127sm7971130wmf.21.2016.06.20.07.54.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 20 Jun 2016 07:54:24 -0700 (PDT) User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.2 In-Reply-To: <838ty07wno.fsf@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:119837 Archived-At: On 06/20/2016 05:38 PM, Eli Zaretskii wrote: > This all sounds like my response is not welcome, but in that case why > did you ask the question? I was kind of hoping for "yes, let's get it into 25.1!"? :) > No, the bug is where the invalid input is generated in the first > place. Each API has its contract; if you violate the contract, you > invoke undefined behavior. It's a bug in the API, or bad API, if you will. It needs stricter contract, and the submitted patch added it. Or to look at it another way, the current contract allows url-http-data to be multibyte, because the requirement to the contrary is not documented anywhere that I can see. The variable is simply undocumented. >> If this is what you need, why not simply test the payload for being a >> unibyte string? There a function, multibyte-string-p, for that. >> >> There are a lot of variables to test (see the comment above the mapconcat call). > > Looks like mapc will be able to deal with that. Or just use concat, > and test the result with multibyte-string-p before sending. Or encode > it with UTF-8, if it is not unibyte already. I don't know if we want to be that permissive that we'll encode to UTF-8 silently. > Btw, I don't think the comment which explains why we started using > mapconcat is accurate these days. It was written before the move to > Unicode in Emacs 23, but we stopped converting raw bytes into Latin-1 > characters in Emacs 23 and later. So maybe we should just go back to > using concat (with erroring out, if the result is multibyte, and/or > maybe with replacing 'length' with 'string-bytes'). Better error out: the payload's encoding is something only the caller should be concerned with. Unless we're fine with the users assuming that Emacs's internal encoding is close enough to UTF-8. > Bottom line: like I said, there should be no reason to use > string-*-unibyte in modern Emacs code on the url-http level or higher > (maybe not at all). Its use is a sign of some basic misunderstanding, > or a bug elsewhere, or remnant of old problems that no longer exist. > So I think we should reconsider the solution on master as well. I don't mind. Would you advocate for having this fix on emacs-25 if I implement it the way you described? >> And you'll have to come up with the error message(s). > > Are you saying you like the error message from string-to-unibyte? > > Cannot convert 123th character to unibyte It's an order of magnitude better than what was before (no error and silent corruption), but yes, there is space for improvement.