From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Philipp Stephani Newsgroups: gmane.emacs.devel Subject: Re: JSON/YAML/TOML/etc. parsing performance Date: Sat, 09 Dec 2017 23:05:35 +0000 Message-ID: References: <87poaqhc63.fsf@lifelogs.com> <8360ceh5f1.fsf@gnu.org> <83h8vl5lf9.fsf@gnu.org> <83r2um3fqi.fsf@gnu.org> <43520b71-9e25-926c-d744-78098dad6441@cs.ucla.edu> <83o9pnzddc.fsf@gnu.org> <472176ce-846b-1f24-716b-98eb95ceaa47@cs.ucla.edu> <83d163z6dy.fsf@gnu.org> <73477c99-1600-a53d-d84f-737837d0f91f@cs.ucla.edu> <83poa2ya8j.fsf@gnu.org> <21b0ba97-ed49-43ae-e86f-63fba762353a@cs.ucla.edu> <83lgkqxe3l.fsf@gnu.org> <903b4aaa-36e6-9985-7313-177d9d34b7ec@cs.ucla.edu> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="001a114f1f025d12a6055ff057e3" X-Trace: blaine.gmane.org 1512860790 19272 195.159.176.226 (9 Dec 2017 23:06:30 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sat, 9 Dec 2017 23:06:30 +0000 (UTC) Cc: Eli Zaretskii , emacs-devel@gnu.org To: Paul Eggert Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sun Dec 10 00:06:26 2017 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eNoCQ-0004ph-BL for ged-emacs-devel@m.gmane.org; Sun, 10 Dec 2017 00:06:26 +0100 Original-Received: from localhost ([::1]:42834 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eNoCV-0006XP-H3 for ged-emacs-devel@m.gmane.org; Sat, 09 Dec 2017 18:06:31 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:39324) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eNoBp-0006X8-Jv for emacs-devel@gnu.org; Sat, 09 Dec 2017 18:05:50 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eNoBo-00085F-GL for emacs-devel@gnu.org; Sat, 09 Dec 2017 18:05:49 -0500 Original-Received: from mail-qk0-x22e.google.com ([2607:f8b0:400d:c09::22e]:33908) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1eNoBm-000802-Ps; Sat, 09 Dec 2017 18:05:46 -0500 Original-Received: by mail-qk0-x22e.google.com with SMTP id d66so3027973qkg.1; Sat, 09 Dec 2017 15:05:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=gtlQ9l+cyKmH71ox2AUQIWC+0+whjEEnaKbvH0RSYzY=; b=CO6aXvALgjOnwq0NLPJX+fDeepffUxDZynnv9mAfpuDbOqOTNqvmeRk55zWgmkIt4s of9ZmJMnYAl5RHcKZJOaikYuCO+2Ra33wz+yjCvtQQs+AhLwbq4+hUeIgBsB3H0xfqT6 6bFfE4o1arDCPBIKeYuAyZ2pEXNNsAHm3f/8bgdgUXlGqEck66ldh8vr3oH+3EylftXN JSgp9d7L1GupcjHrCslKPRrjlT9m3qi7fvu65QzfLlV/h/HqA8gR2PQSyVyMGwOeCSsL /32QyCTHLGyrZwoE6+cMgGym9S6/9dHGqqfQ4oeUUhGYKlzZb7CNd9X2JqIRti4ySNHH BJ3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=gtlQ9l+cyKmH71ox2AUQIWC+0+whjEEnaKbvH0RSYzY=; b=fDDpibqV6tR1ynodETKj48Kz/mUq2pTEuKbaQCccOZxce1ft3MBvuLztNK1eUfC+8r aUWTqbWRA7Te8t1//dlOa+4qQDGyoBqKD0Pf5ufTB79qffcdM5h2jmP6Wg/41YSPNBKi ILdZRzz7F+W34phTV+dLC6S/aSGDIVEEzHQlfNkzE7HIdflS7peJWbsmQ/rMmCDtlP34 7PVtoD/pQ4FSQNXnqey9QwDvg9Dty8TDQHvoPL2gLleHAOKszO+5AddDAJMzY61ADDEF S40FgRGsgWlTOrWTXLXbrlpqFF1Oy8dN+jBGtu1MOL8m/GKinRqAviBloC3tIYg/0oyb S64w== X-Gm-Message-State: AKGB3mIdf2d1RxCkY7gonhp60pCMhHx/sshmAjv0sbZ00wOr7vsyZr5W 5IIS/xCWY+1+/+jtaQmSucEyp8nEXs8C/gnyTg4= X-Google-Smtp-Source: AGs4zMZEEncrbI86tBtKqOclu45hl+SkgdjBZMk9hSGm2B1RNPei4y9MUwceXByeI24C9OTJ+BgVVGjW0KnqCJcxs+o= X-Received: by 10.233.235.207 with SMTP id b198mr42944905qkg.55.1512860746058; Sat, 09 Dec 2017 15:05:46 -0800 (PST) In-Reply-To: X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400d:c09::22e X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:220838 Archived-At: --001a114f1f025d12a6055ff057e3 Content-Type: text/plain; charset="UTF-8" Philipp Stephani schrieb am So., 29. Okt. 2017 um 23:49 Uhr: > Philipp Stephani schrieb am So., 29. Okt. 2017 um > 21:48 Uhr: > >> Paul Eggert schrieb am Mo., 9. Okt. 2017 um >> 08:19 Uhr: >> >>> Philipp Stephani wrote: >>> > I don't think Jansson can use xmalloc because xmalloc can exit >>> nonlocally, >>> > which is not expected by a third-party library such as Jansson. It >>> could >>> > use a suitable wrapper of lmalloc, though. >>> >>> That would be overkill, as lmalloc arranges for Lisp alignment, which >>> Jansson >>> does not need. We could define new functions (smalloc and srealloc, >>> say), that >>> act like malloc and realloc except they return NULL for requests larger >>> than >>> PTRDIFF_MAX. Right now, I expect only the JSON code needs this sort of >>> thing so >>> we could put the new functions in json.c. If other code needs it later >>> we could >>> move these new functions to alloc.c. >>> >> >> Yes, that sounds reasonable. >> > > Here's a new patch that incorporates some of these changes. Specifically: > > - I've removed some of the assertions > - I've installed a custom allocator, as you suggested > - Reverted back to creating a temporary string and inserting that into the > buffer. Anything else just doesn't seem to work or seems way too complex. > - Introduced explicit encoding and decoding. I suspect that will lead to a > massive performance hit, but I haven't done any benchmarks yet. > - Added manual section and NEWS entry > Is that patch OK for master? --001a114f1f025d12a6055ff057e3 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


Philip= p Stephani <p.stephani2@gmail.c= om> schrieb am So., 29. Okt. 2017 um 23:49=C2=A0Uhr:
Philipp Stephani <p.stephani2@gmail.com> schrieb am So., 29. Okt. 2017= um 21:48=C2=A0Uhr:
Paul Eggert <eggert@cs.ucla.edu> schri= eb am Mo., 9. Okt. 2017 um 08:19=C2=A0Uhr:
Philipp Stephani wrote:
> I don't think Jansson can use xmalloc because xmalloc can exit non= locally,
> which is not expected by a third-party library such as Jansson. It cou= ld
> use a suitable wrapper of lmalloc, though.

That would be overkill, as lmalloc arranges for Lisp alignment, which Janss= on
does not need. We could define new functions (smalloc and srealloc, say), t= hat
act like malloc and realloc except they return NULL for requests larger tha= n
PTRDIFF_MAX. Right now, I expect only the JSON code needs this sort of thin= g so
we could put the new functions in json.c. If other code needs it later we c= ould
move these new functions to alloc.c.

<= /div>
Yes, that sounds reas= onable.=C2=A0

Here's a new patch that in= corporates some of these changes. Specifically:

- = I've removed some of the assertions
- I've installed a cu= stom allocator, as you suggested
- Reverted back to creating a te= mporary string and inserting that into the buffer. Anything else just doesn= 't seem to work or seems way too complex.
- Introduced explic= it encoding and decoding. I suspect that will lead to a massive performance= hit, but I haven't done any benchmarks yet.
- Added manual s= ection and NEWS entry=C2=A0

Is that patch OK for master?=C2=A0
--001a114f1f025d12a6055ff057e3--