From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id 0BFDA431FAF for ; Sun, 29 Apr 2012 13:11:38 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: -0.7 X-Spam-Level: X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5 tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XBT1hPpDCdP8 for ; Sun, 29 Apr 2012 13:11:35 -0700 (PDT) Received: from dmz-mailsec-scanner-3.mit.edu (DMZ-MAILSEC-SCANNER-3.MIT.EDU [18.9.25.14]) by olra.theworths.org (Postfix) with ESMTP id 60DBC431FAE for ; Sun, 29 Apr 2012 13:11:35 -0700 (PDT) X-AuditID: 1209190e-b7fd86d0000008b4-aa-4f9da076cf97 Received: from mailhub-auth-1.mit.edu ( [18.9.21.35]) by dmz-mailsec-scanner-3.mit.edu (Symantec Messaging Gateway) with SMTP id 2E.F5.02228.670AD9F4; Sun, 29 Apr 2012 16:11:34 -0400 (EDT) Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103]) by mailhub-auth-1.mit.edu (8.13.8/8.9.2) with ESMTP id q3TKBXNg010169; Sun, 29 Apr 2012 16:11:34 -0400 Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91]) (authenticated bits=0) (User authenticated as amdragon@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id q3TKBWLx024979 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT); Sun, 29 Apr 2012 16:11:32 -0400 (EDT) Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.77) (envelope-from ) id 1SOaSx-0003js-RY; Sun, 29 Apr 2012 16:11:31 -0400 Date: Sun, 29 Apr 2012 16:11:31 -0400 From: Austin Clements To: Chris Gray Subject: Re: [RFC] Use JSON in emacs interface Message-ID: <20120429201131.GM2704@mit.edu> References: <87ty02v786.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87ty02v786.fsf@gmail.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprAKsWRmVeSWpSXmKPExsUixCmqrFu2YK6/wc8N3BZLz/xntrh+cyaz A5PHzll32T2erbrFHMAUxWWTkpqTWZZapG+XwJVxcN1C1oJXQhWnXi9gamD8ydfFyMkhIWAi 0fz6ATuELSZx4d56ti5GLg4hgX2MEnt/3mCBcDYwSkzaOIUJwjnJJPH33Q8oZwmjxJvl/Swg /SwCqhI3nm1lArHZBDQktu1fzghiiwDFd3fPB7OZBaQlvv1uBqsRFtCTmPDiHVgvr4C2xKU7 85lBbCEBNYkjb95CxQUlTs58wgLRqyVx499LoF4OsDnL/3GAhDkF1CWam9eAtYoKqEhMObmN bQKj0Cwk3bOQdM9C6F7AyLyKUTYlt0o3NzEzpzg1Wbc4OTEvL7VI11gvN7NELzWldBMjOLAl +XYwfj2odIhRgINRiYdXKHeOvxBrYllxZe4hRkkOJiVR3q+z5/oL8SXlp1RmJBZnxBeV5qQW H2KU4GBWEuHVFgcq501JrKxKLcqHSUlzsCiJ86ppvfMTEkhPLEnNTk0tSC2CycpwcChJ8O6e DzRUsCg1PbUiLTOnBCHNxMEJMpwHaPgckBre4oLE3OLMdIj8KUZdjnlTtl5lFGLJy89LlRLn vQ9SJABSlFGaBzcHlpBeMYoDvSXMuxykigeYzOAmvQJawgS0hMlzFsiSkkSElFQDY1TVsQWe hzumP0vsU/zivDG6JN9rt5rVWs7WKi23LcYbX3+SVWf8bdJTzO/w7Nf1ia2i/y1Vkks0mgp5 T64+5BLmOkGVj2lWxIxY0R1nFrb7Mv1z7D3roxnWuuOheljg5Gnln5dp6G9wcq5NbuG8dPPu xP5s/zNxG6dpXPv1+J9y9Csr1uzZSizFGYmGWsxFxYkAYEuIASMDAAA= Cc: notmuch@notmuchmail.org X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Apr 2012 20:11:38 -0000 Quoth Chris Gray on Apr 29 at 10:22 am: > Hi, > > My thinking about this arises from the fact that there is a person on > one of the lists that I read who puts a semicolon after his name. Of > course, this confuses the regex in notmuch-search-process-filter, which > expects that the first semicolon in the string representing a thread is > after all the authors. > > I first thought of changing the regex so that it looked for the last > semicolon in the string or something like that, but that would just move > the problem. (Semicolons are probably more frequent in subject lines > than in author names.) So it seems to me that what is needed is for > notmuch and emacs to talk with each other in a format that is > unambiguously parseable. Since notmuch search already has the option of > outputting to JSON, that seems like a natural fit. > > Emacs has an existing JSON parser, > , > but it doesn't appear that it is able to parse progressively, meaning > that it wouldn't be able to display results as they come in from notmuch > search if used as-is. My guess is that its parts could be hacked > together to overcome this limitation though. > > Anyway, if others think this is a good idea, I'm willing to do the > coding. This is definitely a good idea. In fact, there's been quite a bit of discussion about it in the past, and even some (very old and outdated) implementation work: id:"878vs28dvo.fsf@praet.org" A rather lengthy discussion of another motivation for using JSON search. Includes performance results for using JSON in search. id:"20120214152114.GQ27039@mit.edu" Discussion of "framed" JSON that would be a valid JSON object, but could easily be consumed in a streaming fashion without hacking Emacs' JSON parser or resorting to paging. id:"1290777202-14040-1-git-send-email-dme@dme.org" The implementation from long ago. Definitely outdated, but could be a good starting point. It looks like most of the discussion about streaming JSON parsing has been on the IRC channel, rather than the mailing list, but I'd be happy to summarize it or dig out the IRC logs. I agree with Adam that it probably makes the most sense to get an implementation working without streaming first and then add streaming support.