From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Philipp Stephani Newsgroups: gmane.emacs.devel Subject: Re: bug#23750: 25.0.95; bug in url-retrieve or json.el Date: Wed, 28 Dec 2016 18:45:43 +0000 Message-ID: References: <6d0c8c2e-8428-2fdb-0d6e-899f7b9d7ffd@nifty.com> <8053af81-80e1-a24a-f649-8ffc86963ed5@nifty.com> <0cc7fab4-9a2c-6a8d-def7-36bd50317ca3@yandex.ru> <7f9a799f-de88-fd78-0cdc-dac0928f1503@nifty.com> <308bb78f-8be3-092d-d877-e129d340242b@nifty.com> <4dc615e7-ec73-60a5-426e-0d6986f15d76@yandex.ru> <0cb406fb-ffc4-a4ad-557a-2cacc99b8e75@nifty.com> <86ccb4af-5719-c017-26bb-fc06b4c904d2@yandex.ru> <83r35uxkr5.fsf@gnu.org> <4e12d4ad-cd6b-3087-5d7c-449d4c1886e2@yandex.ru> <83lgw1q9uu.fsf@gnu.org> <83eg1tq8is.fsf@gnu.org> <787e5206-53e0-752f-a339-4608d2f7ad39@yandex.ru> <8360n5q6j4.fsf@gnu.org> <8337i8rkbe.fsf@gnu.org> <83polcpzwk.fsf@gnu.org> <83lguzvr63.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=001a114b41a0ec7b480544bc6023 X-Trace: blaine.gmane.org 1482950859 9505 195.159.176.226 (28 Dec 2016 18:47:39 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Wed, 28 Dec 2016 18:47:39 +0000 (UTC) Cc: larsi@gnus.org, dgutov@yandex.ru, kentaro.nakazawa@nifty.com, emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed Dec 28 19:47:34 2016 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cMJG3-00014I-UD for ged-emacs-devel@m.gmane.org; Wed, 28 Dec 2016 19:47:28 +0100 Original-Received: from localhost ([::1]:60519 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cMJG8-0000Yr-Mi for ged-emacs-devel@m.gmane.org; Wed, 28 Dec 2016 13:47:32 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:44563) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cMJEh-0000ID-TT for emacs-devel@gnu.org; Wed, 28 Dec 2016 13:46:07 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cMJEe-00034P-5a for emacs-devel@gnu.org; Wed, 28 Dec 2016 13:46:03 -0500 Original-Received: from mail-wm0-x231.google.com ([2a00:1450:400c:c09::231]:38866) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cMJEZ-00032a-H3; Wed, 28 Dec 2016 13:45:55 -0500 Original-Received: by mail-wm0-x231.google.com with SMTP id k184so120127342wme.1; Wed, 28 Dec 2016 10:45:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=vsRktpA7D8WLb1zaC5kZbtDCJksNQD8TTg8C4YwtMcc=; b=nLz4ssfBYeKVLuXyb+G2WBIuJYSd2IKOMKiMta9VZhjcTolWAFW9tmVguqk2Wdejmw K6Y2Nt2dMgD5z/2cN0KZB7D+kMl6DuiO5Aj5aIGFftQCTRTAMg45yTLLGctoXZCbZlYJ LchNDd36750bGwTx5od2tzdBTSaAwWZXIxdKesEOaqt4RXYfJjs8A+wdlJwNJlCP7rfq 1rM1lIsJhnJ9aZFOjqLCzu0Es4XIO3tiiDU82UASMGnf7QYtqjz0lSPh1kZLaD9n/HoX HuMU4rHy4RCGeoL5mRfxv5SgLrB3scA0tjXRIF+lEMV0k9iAt7AR20QgXZf3ceRwOfWp QfOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=vsRktpA7D8WLb1zaC5kZbtDCJksNQD8TTg8C4YwtMcc=; b=pQ4D4AGNWp9d5rzpvZLgM2TsIyrdVdTuRCHpi7/t7nHOedBhuE4k4BvYn5K+spdIS6 A9qoqbvh0sb/g4o/QwKVAEDtWYxaA+0Q944G3yUvwZfRxget9ZCUCidt0vEuo9Sirw2i lgpWLfKuCaCUrxTDV3p+3ediqCgrmldp26YBugtAgyINYIZ1hSu7KGHw0WZ4rs7xguHI lz5llGnIeFv9ZeMaW0xkWJAJaaXn881kb4vKx+aEGk2ckmiR6GhT2x4meJOJcNqmyiSu bKMHaRNZY/UTia/kdrFneNHpO4PBCuf2b9n41mxPzDtc3sKWHzXQPC39o+mbEj+XMPbj 493A== X-Gm-Message-State: AIkVDXJgSWZCi859NHsMJiz/WCnSW+35Zw2K1SKKa8jdebGvJ24JhgdXdCUnPSX8IsD+jsxQNoatjwUQz7xeAw== X-Received: by 10.28.31.23 with SMTP id f23mr37952253wmf.94.1482950754202; Wed, 28 Dec 2016 10:45:54 -0800 (PST) In-Reply-To: <83lguzvr63.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::231 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:210924 Archived-At: --001a114b41a0ec7b480544bc6023 Content-Type: text/plain; charset=UTF-8 Eli Zaretskii schrieb am Mi., 28. Dez. 2016 um 19:35 Uhr: > > From: Philipp Stephani > > Date: Wed, 28 Dec 2016 18:18:25 +0000 > > Cc: larsi@gnus.org, emacs-devel@gnu.org, kentaro.nakazawa@nifty.com, > > dgutov@yandex.ru > > > > > > That's right -- why should any code care? Yet url.el does. > > > > > > No, it doesn't, not if the string is plain ASCII. > > > > > > But in that case it isn't, it's morally a byte array. > > > > Yes, because the internal representation of characters in Emacs is a > > superset of UTF-8. > > > > That has nothing to do with characters. A byte array is conceptually > different from a character string. > > In Emacs, they are both implemented using very similar objects. > Yes, that's why I said "conceptually different". The concepts may be the different, but the implementation might still be the same. > > > > What Emacs lacks is good support for byte arrays. > > > > Unibyte strings are byte arrays. What do you think we lack in that > regard? > > > > If unibyte strings should be used for byte arrays, then the URL > functions should indeed signal an error > > whenever url-request-data is a multibyte string, as HTTP requests are > conceptually byte arrays, not character > > strings. > > Which is what we do now. > There is no such check for url-request-data. There's an overall check for the complete request, but that also doesn't check for unibyte-ness. > > > > For HTTP, process-send-string shouldn't need to deal > > > with encoding or EOL conversion, it should just accept a byte array > and send that, unmodified. > > > > I disagree. Handling unibyte strings is a nuisance, so Emacs allows > > most applications be oblivious about them, and just handle > > human-readable text. > > > > That is the wrong approach (byte arrays and character strings are > fundamentally different types, and mixing > > them together only causes pain), and it cannot work when implementing > network protocols. HTTP requests > > are *not* human-readable text, they are byte arrays. Attempting to > handle Unicode strings can't work because > > we wouldn't know the number of encoded bytes. > > You are arguing against a long and quite painful history of non-ASCII > strings in Emacs. What we have now is based on a lot of experience > and at least two very large refactoring jobs. Going back would be a > very bad idea indeed, as we've been there already, and users didn't > like that. Some of us are old enough to remember the notorious \201 > bytes creeping into text files and mail messages, due to that. Never > again. > I'm not suggesting going back, too much would be broken. > > Our experience is that we should keep use of unibyte strings in Lisp > application code to the absolute minimum, ideally zero. Once we > arrived at that conclusion, we've been living happily ever after. > This minor issue we are discussing here is certainly not worth > repeating past mistakes for which we paid plenty in sweat and blood. > If you want unibyte strings to represent octet streams, then unibyte strings must be usable in application code, because octet streams are a concept that exists in reality, and applications must be able to support them in some way. If you don't want unibyte strings, then you need to provide some different way to represent octet streams. --001a114b41a0ec7b480544bc6023 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable


Eli Za= retskii <eliz@gnu.org> schrieb am= Mi., 28. Dez. 2016 um 19:35=C2=A0Uhr:
> From: Philipp Stephani <p.stephani2@gmail.com>
> Date: Wed, 28 Dec 2016 18:18:25 +0000
> Cc: larsi@gnus.org, emacs-devel@gnu.org, kentaro.nakaz= awa@nifty.com,
>=C2=A0 =C2=A0 =C2=A0 =C2=A0dgutov@yandex.ru
>
>=C2=A0 > > That's right -- why should any code care? Yet url.= el does.
>=C2=A0 >
>=C2=A0 > No, it doesn't, not if the string is plain ASCII.
>=C2=A0 >
>=C2=A0 > But in that case it isn't, it's morally a byte arra= y.
>
>=C2=A0 Yes, because the internal representation of characters in Emacs = is a
>=C2=A0 superset of UTF-8.
>
> That has nothing to do with characters. A byte array is conceptually d= ifferent from a character string.

In Emacs, they are both implemented using very similar objects.

Yes, that's why I said &qu= ot;conceptually different". The concepts may be the different, but the= implementation might still be the same.
=C2=A0

>=C2=A0 > What Emacs lacks is good support for byte arrays.
>
>=C2=A0 Unibyte strings are byte arrays. What do you think we lack in th= at regard?
>
> If unibyte strings should be used for byte arrays, then the URL functi= ons should indeed signal an error
> whenever url-request-data is a multibyte string, as HTTP requests are = conceptually byte arrays, not character
> strings.

Which is what we do now.

There is no such check for url-request-data. There's an overall c= heck for the complete request, but that also doesn't check for unibyte-= ness.
=C2=A0

>=C2=A0 > For HTTP, process-send-string shouldn't need to deal >=C2=A0 > with encoding or EOL conversion, it should just accept a by= te array and send that, unmodified.
>
>=C2=A0 I disagree. Handling unibyte strings is a nuisance, so Emacs all= ows
>=C2=A0 most applications be oblivious about them, and just handle
>=C2=A0 human-readable text.
>
> That is the wrong approach (byte arrays and character strings are fund= amentally different types, and mixing
> them together only causes pain), and it cannot work when implementing = network protocols. HTTP requests
> are *not* human-readable text, they are byte arrays. Attempting to han= dle Unicode strings can't work because
> we wouldn't know the number of encoded bytes.

You are arguing against a long and quite painful history of non-ASCII
strings in Emacs.=C2=A0 What we have now is based on a lot of experience and at least two very large refactoring jobs.=C2=A0 Going back would be a very bad idea indeed, as we've been there already, and users didn't=
like that.=C2=A0 Some of us are old enough to remember the notorious \201 bytes creeping into text files and mail messages, due to that.=C2=A0 Never<= br class=3D"gmail_msg"> again.

I'm not = suggesting going back, too much would be broken.
=C2=A0

Our experience is that we should keep use of unibyte strings in Lisp
application code to the absolute minimum, ideally zero.=C2=A0 Once we
arrived at that conclusion, we've been living happily ever after.
This minor issue we are discussing here is certainly not worth
repeating past mistakes for which we paid plenty in sweat and blood.

If you want unibyte strin= gs to represent octet streams, then unibyte strings must be usable in appli= cation code, because octet streams are a concept that exists in reality, an= d applications must be able to support them in some way. If you don't w= ant unibyte strings, then you need to provide some different way to represe= nt octet streams.=C2=A0
--001a114b41a0ec7b480544bc6023--