From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Julian Scheid Newsgroups: gmane.emacs.devel Subject: Re: Fast JSON codec based on YAJL Date: Tue, 15 Jan 2013 04:04:37 +0100 Message-ID: References: NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=e89a8f3ba0a1a8362004d34b054f X-Trace: ger.gmane.org 1358219086 16965 80.91.229.3 (15 Jan 2013 03:04:46 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 15 Jan 2013 03:04:46 +0000 (UTC) Cc: emacs-devel@gnu.org To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Tue Jan 15 04:05:04 2013 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1Tuwpk-0000Uw-2E for ged-emacs-devel@m.gmane.org; Tue, 15 Jan 2013 04:05:04 +0100 Original-Received: from localhost ([::1]:54993 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TuwpT-00023M-Lb for ged-emacs-devel@m.gmane.org; Mon, 14 Jan 2013 22:04:47 -0500 Original-Received: from eggs.gnu.org ([208.118.235.92]:38254) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TuwpO-00023A-FE for emacs-devel@gnu.org; Mon, 14 Jan 2013 22:04:45 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TuwpL-0003YV-E7 for emacs-devel@gnu.org; Mon, 14 Jan 2013 22:04:42 -0500 Original-Received: from mail-ie0-f173.google.com ([209.85.223.173]:54463) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TuwpL-0003YQ-8H for emacs-devel@gnu.org; Mon, 14 Jan 2013 22:04:39 -0500 Original-Received: by mail-ie0-f173.google.com with SMTP id e13so6400246iej.4 for ; Mon, 14 Jan 2013 19:04:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=QBH4/x4A0FkYGN5/U9NHGoeAnjcIqCpzXOpT7N+XWu8=; b=ZXZ2YL9dnkbg4SIEmYR2QW5czxXB+IEnweafxjwXlCPyJEw0a57hXznd3oAocnwsYb UUyWlU942RB40TRRoqAH3MkOcSqVPPHP2dTuYj1CVM1bzAI08r5sVCV11yzvmlvk0yjs mMH97w+hu6F9wW8M3ZueP9Mla2jGdCnC6plXt7QiFYJ1AhXFAWufLL5eCgxUKr20ZDWp DbhzljPibAPtokOAivkV00WspOb97PdLqX4Ps1z3YQmZvbKMcreXvLy9w0vr2CaMCwmC 8MEMDYx+xzw+hqZDDiCboH0B6zb95YbCx6s9osTOvDahez9FCP3kvb9gYOo14oECAOkh erAw== Original-Received: by 10.50.91.230 with SMTP id ch6mr494666igb.92.1358219077760; Mon, 14 Jan 2013 19:04:37 -0800 (PST) Original-Received: by 10.231.45.78 with HTTP; Mon, 14 Jan 2013 19:04:37 -0800 (PST) In-Reply-To: X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 209.85.223.173 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:156369 Archived-At: --e89a8f3ba0a1a8362004d34b054f Content-Type: text/plain; charset=ISO-8859-1 On Tue, Jan 15, 2013 at 3:26 AM, Stefan Monnier wrote: > You mean that json.elc is only 5 to 25 times slower than > a C implementation? I actually expected significantly worse. > Both share quite a few code paths actually, for example number-to-string conversion (and vice versa) and multibyte handling. I wrote the code a little while ago but IIRC data conversion between Emacs-land and native land accounts for a non-negligible share of the runtime, and of course the same goes for building Emacs data structures from parsed data. (It doesn't help that with JSON, you don't know how big an array or map will be until you're finished parsing it.) Also, although performance is my primary objective, I didn't pick the C library on that aspect alone. For example, libjson bills itself as a "JSON reader and writer which is super-effiecient [sic] and usually runs circles around other JSON libraries." But it doesn't support streaming and thus would add a significant memory overhead for large documents, and also it's not nearly as portable. I did spend some time making sure that the interface performs well and there are no obvious low-hanging-fruit-type optimizations left, but any help with making it even faster is welcome! I think that an order of magnitude is a good start though, it certainly takes my other app from "painfully slow" to "perfectly usable". --e89a8f3ba0a1a8362004d34b054f Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
On Tue, Jan 15, 2013 at 3:26 AM= , Stefan Monnier <monnier@iro.umontreal.ca> wrote:
You mean that json.elc is only 5 to 25 ti= mes slower than
a C implementation? =A0I actually expected significantly worse.

Both share quite a few code paths actually, for = example number-to-string
conversion (and vi= ce versa) and multibyte handling. I wrote the code a little
while ago but IIRC data conversion between Emacs-land and native land accou= nts
for a non-negligible share of the runti= me, and of course the same goes for
buildin= g Emacs data structures from parsed data. (It doesn't help that with
JSON, you don't know how big an array or map= will be until you're finished
parsing = it.)

A= lso, although performance is my primary objective, I didn't pick the C = library
on that aspect alone. For example, libjson bills= itself as a "JSON reader and
writer w= hich is super-effiecient [sic] and usually runs circles around other
<= div class=3D"gmail_extra"> JSON libraries." But it doesn't support streaming and thus would a= dd a
significant memory overhead for large = documents, and also it's not nearly as
portable.

I did spend some time making sure that the interface performs well and = there are
no obvious low-hanging-fruit-type= optimizations left, but any help with making
it even faster is welcome! I think that an order= of magnitude is a good start
though, it ce= rtainly takes my other app from "painfully slow" to "perfect= ly
usable".



--e89a8f3ba0a1a8362004d34b054f--