From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Philipp Stephani Newsgroups: gmane.emacs.devel Subject: Re: JSON/YAML/TOML/etc. parsing performance Date: Sun, 17 Sep 2017 20:27:02 +0000 Message-ID: References: <87poaqhc63.fsf@lifelogs.com> <83vakhi1rx.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="001a113cc3627f27c7055968732f" X-Trace: blaine.gmane.org 1505680093 4025 195.159.176.226 (17 Sep 2017 20:28:13 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sun, 17 Sep 2017 20:28:13 +0000 (UTC) Cc: emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sun Sep 17 22:28:06 2017 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dtgAe-0000ez-9S for ged-emacs-devel@m.gmane.org; Sun, 17 Sep 2017 22:28:04 +0200 Original-Received: from localhost ([::1]:33596 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dtgAj-0001Cz-TY for ged-emacs-devel@m.gmane.org; Sun, 17 Sep 2017 16:28:09 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:33170) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dtgA0-0001Co-BQ for emacs-devel@gnu.org; Sun, 17 Sep 2017 16:27:25 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dtg9y-00023c-Hw for emacs-devel@gnu.org; Sun, 17 Sep 2017 16:27:24 -0400 Original-Received: from mail-io0-x22c.google.com ([2607:f8b0:4001:c06::22c]:52382) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dtg9p-0001v7-MI; Sun, 17 Sep 2017 16:27:13 -0400 Original-Received: by mail-io0-x22c.google.com with SMTP id i197so14483272ioe.9; Sun, 17 Sep 2017 13:27:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=SRdIBGOVhomPgbZb4aA1a8VuMhOzxscbJCXAZ9S2ntU=; b=NwM9jupIwibz8b4/+tLmqZjo87p14eMQhXQ64ttpuvjVzirJUjaW11+GJEjmufkJKL tyMJcMhRtYAjrR6gVx4GEuh1IjThaYxZWJLbBd4sXr69MrAhsuHkT+eCl+pzSUwEwVuK X6dXx0uSPhOJ43Xz34LsRrJcKujQMF7gGj40tka58uTl3RD4Th4IHImcRlaGxZ6uREbq Sg0bjyCfLMC2g1nv7/hLQdFkTaCcaofqeyfDx2zY4BEOAxbBIL0em2ICOzILtwC5i6xS T7xzQGHF53pxTK9vIAMaQCHlfgf0aerRkRVMaLVP9iQJ4rdP6+YJ+hNE7cMHrypUjlAp em/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=SRdIBGOVhomPgbZb4aA1a8VuMhOzxscbJCXAZ9S2ntU=; b=Yle1KjAyf0a1A2Vgq+rAL2CyPj0OFM0XApJNPs4kH2j23pATMJ0kX/6fMdoEogbVo7 UNdyjd0PZJIYO2UOr4JKvjoARLuYgeYvEt9RAPzJGdyjhBEHiwftBOCSwnzM6D0GJGDO 4JkinrKa31QoikHi+AWNhu3SmBZCscXhww7yvTTXrwU2RdEsr0Hr8zW3JxDQUxT9wxoI RwnN44HR8rQ+utsyhumm5NsODEyqOXlxI3IkkjZy/up4mUGcf2tOIWc1tvsN84Z0Iexu wRgDaXdWqUvnsKGxYUX7wXRUH5LpiXVVOtg/RLMy0aBpRRBNI0suX7Iiqt4lS1C2dHDP oQxQ== X-Gm-Message-State: AHPjjUgBMftlsXtOzHdVl6k2ux41I9e9LH4EEcMBvpGYZm6wsQYSWX8q ZUlZVEBOcD/tc+xF3ab77L9oUJ/1gMRVjI1GWnIoDg== X-Google-Smtp-Source: AOwi7QDFjy+QD/WzqAti9sltvLdiYZOyUwPFgs/CMPk6n8f0GWgM4fh+wTtP3/ymHBvvRCZyXfqCaNkLivSRCVi7J9Q= X-Received: by 10.202.171.147 with SMTP id u141mr15820679oie.312.1505680032726; Sun, 17 Sep 2017 13:27:12 -0700 (PDT) In-Reply-To: <83vakhi1rx.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4001:c06::22c X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:218425 Archived-At: --001a113cc3627f27c7055968732f Content-Type: text/plain; charset="UTF-8" Eli Zaretskii schrieb am So., 17. Sep. 2017 um 21:05 Uhr: > > From: Philipp Stephani > > Date: Sun, 17 Sep 2017 18:46:45 +0000 > > > > I don't know much about the others, but given the importance of JSON as > data exchange and serialization > > format, I think it's worthwhile to invest some time here. I've > implemented a wrapper around the json-c library > > (license: Expat/X11/MIT), resulting in significant speedups using the > test data from > > https://github.com/miloyip/nativejson-benchmark: a factor of 3.9 to 6.4 > for parsing, and a factor of 27 to 67 for > > serializing. If people agree that this is useful I can send a patch. > > Before we make a decision on which library to use, I'd prefer some > kind of survey of available free software libraries, including their > popularity and development activity. The survey doesn't have to be > exhaustive, but I think we should compare at least a few candidates. > We already have 2, so maybe we should start by comparing them. > > Sure, I've made a quick overview based on https://github.com/miloyip/nativejson-benchmark. I've only used the libraries that are written in C and have been tested in that benchmark; it's still quite a few. I've checked the conformance and speed metrics from the benchmark as well as number of GitHub stars (as proxy for popularity) and number of commit in the last month (as proxy for development activity): Here are the results: https://docs.google.com/spreadsheets/d/e/2PACX-1vTqKxqo47s67L3EJ9AWvZclNuT2xbd9rgoRuJ_UYbXgnV171owr8h2mksHjrjNGADDR3DVTWQvUMBpe/pubhtml?gid=0&single=true Note that some of the libraries (jsmn, ujson4c) don't appear to support serialization at all; I'd suggest to avoid them, because we'd then need to wrap another library for serialization. Also, even though JSMN advertises itself as "world's fastest JSON parser", it's actually the slowest of the libraries in the survey. json-c appears to be reasonably conformant and fast for both parsing and serialization, and has by far the largest development activity. --001a113cc3627f27c7055968732f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


Eli Za= retskii <eliz@gnu.org> schrieb am= So., 17. Sep. 2017 um 21:05=C2=A0Uhr:
> From: Philipp Stephani <p.stephani2@gmail.com>
> Date: Sun, 17 Sep 2017 18:46:45 +0000
>
> I don't know much about the others, but given the importance of JS= ON as data exchange and serialization
> format, I think it's worthwhile to invest some time here. I've= implemented a wrapper around the json-c library
> (license: Expat/X11/MIT), resulting in significant speedups using the = test data from
> https://github.com/miloyip/nativejson-benchmark<= /a>: a factor of 3.9 to 6.4 for parsing, and a factor of 27 to 67 for
> serializing. If people agree that this is useful I can send a patch.
Before we make a decision on which library to use, I'd prefer some
kind of survey of available free software libraries, including their
popularity and development activity.=C2=A0 The survey doesn't have to b= e
exhaustive, but I think we should compare at least a few candidates.
We already have 2, so maybe we should start by comparing them.


Sure, I've made a quick overview b= ased on https:/= /github.com/miloyip/nativejson-benchmark. I've only used the librar= ies that are written in C and have been tested in that benchmark; it's = still quite a few. I've checked the conformance and speed metrics from = the benchmark as well as number of GitHub stars (as proxy for popularity) a= nd number of commit in the last month (as proxy for development activity): = Here are the results:=C2=A0https://docs.google.com/= spreadsheets/d/e/2PACX-1vTqKxqo47s67L3EJ9AWvZclNuT2xbd9rgoRuJ_UYbXgnV171owr= 8h2mksHjrjNGADDR3DVTWQvUMBpe/pubhtml?gid=3D0&single=3Dtrue
Note that some of the libraries (jsmn, ujson4c) don't appear to suppo= rt serialization at all; I'd suggest to avoid them, because we'd th= en need to wrap another library for serialization. Also, even though JSMN a= dvertises itself as "world's fastest JSON parser", it's a= ctually the slowest of the libraries in the survey. json-c appears to be re= asonably conformant and fast for both parsing and serialization, and has by= far the largest development activity.
--001a113cc3627f27c7055968732f--