From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id 343D4431FB6 for ; Tue, 19 Jun 2012 11:52:09 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: -2.29 X-Spam-Level: X-Spam-Status: No, score=-2.29 tagged_above=-999 required=5 tests=[RCVD_IN_DNSWL_MED=-2.3, T_MIME_NO_TEXT=0.01] autolearn=disabled Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tZmcVK9aGbCF for ; Tue, 19 Jun 2012 11:52:08 -0700 (PDT) Received: from outgoing-mail.its.caltech.edu (outgoing-mail.its.caltech.edu [131.215.239.19]) by olra.theworths.org (Postfix) with ESMTP id 75601431FAF for ; Tue, 19 Jun 2012 11:52:08 -0700 (PDT) Received: from earth-doxen.imss.caltech.edu (localhost [127.0.0.1]) by earth-doxen-postvirus (Postfix) with ESMTP id 1A87E66E00FE; Tue, 19 Jun 2012 11:52:08 -0700 (PDT) X-Spam-Scanned: at Caltech-IMSS on earth-doxen by amavisd-new Received: from finestructure.net (gwave-111.ligo.caltech.edu [131.215.114.111]) (Authenticated sender: jrollins) by earth-doxen-submit (Postfix) with ESMTP id 7FA9166E008D; Tue, 19 Jun 2012 11:52:04 -0700 (PDT) Received: by finestructure.net (Postfix, from userid 1000) id 49D8293D; Tue, 19 Jun 2012 11:52:04 -0700 (PDT) From: Jameson Graef Rollins To: Mark Walters , notmuch@notmuchmail.org Subject: Re: Notmuch Pick In-Reply-To: <87mx3zok3d.fsf@qmul.ac.uk> References: <87395ump0d.fsf@qmul.ac.uk> <87zk80ilt9.fsf@servo.finestructure.net> <87mx3zok3d.fsf@qmul.ac.uk> User-Agent: Notmuch/0.13.2+53~g1567997 (http://notmuchmail.org) Emacs/23.4.1 (x86_64-pc-linux-gnu) Date: Tue, 19 Jun 2012 11:52:01 -0700 Message-ID: <87zk7z2kz2.fsf@servo.finestructure.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Jun 2012 18:52:09 -0000 --=-=-= Content-Transfer-Encoding: quoted-printable On Tue, Jun 19 2012, Mark Walters wrote: > As you say several tests do need fixing: I > thought I would leave that until people said they were basically happy > with the change.=20 Yeah, but failing tests are basically a non-starter for me. You can maybe get away with not adding new tests during development, but I'm unlikely to even try out the patches if a bunch of the existing tests are failing. > I am not sure about sanity tests for pick: how would that work while it > lives in contrib? Obviously it would need some tests before coming in to > proper mainline. Since you're not actually submitting these patches yet, does it really need to live in contrib? The goal is for it to ultimately *not* live in contrib, so I would say just go ahead and make them apply in the main source. An example of a sanity test I'm talking about is simply display the pick thread for something and test that it looks like expected. Sort of a zeroth order thing. There are other emacs tests that do the same sort of thing so it shouldn't be too hard to add. >> Would it also be useful to make this same change for the search out, for >> consistency? I notice the search output now uses newlines between all >> fields, which should help for asynchronous processing, but it might be >> nice to put newline separators between the initial and final brackets as >> well. > > Right.=20 I would say go ahead and send to the list any patches that come up in development that are generally useful. Especially if they're small like this they are likely to applied before this series, so it will be smaller and easier to read when the times comes. >> df97df62b70b884a1cd367360ed6ff7eda0e8af6 >> cli: add --headers_only option to notmuch-show.c >> >> Your comment in this patch is very interesting: >> >> This is used by notmuch-pick.el (although not needed) because it giv= es a >> speed-up of at least a factor of a two; moreover it reduces the memo= ry >> usage in emacs hugely. >> >> The only difference between the regular show json output and the >> --headers-only output, as far as I can tell, is the presence of the >> content of text/plain parts if they exist in the message. We previously >> had a discussion about the show output not including any part bodies at >> all, but we decided that the inclusion of text/plain bodies shouldn't >> affect anything, so why not include them. If they actually do, then I >> argue we should just move to having show json not include any body parts >> at all by default, and just have them be retrieved individually like we >> do currently for non-text/plain parts. This would make things cleaner, >> and would get rid of the need to have this extra option, which really >> doesn't produce a significantly different output. > > For one use of pick (displaying the structure of a single thread) this > is not important (and the asynchronous stuff is irrelevant too). For > another use of displaying the thread structure of a whole "folder" of > messages it is important. For example the output of=20 > > notmuch show tag:notmuch > > is about 70MB on my system, whereas with the --headers-only option this > drops to about 7MB. Note Emacs uses substantially more than this much > memory to actually process the JSON, and on a low-powered laptop this > is enough to cause a swap-storm. > > Your suggestion of just not including the text/plain is nice for > notmuch-pick, and is very simple (a single line change). However, it > does seem to slow the emacs show mode down noticeably on large threads > (something like 2s to 4s on a thread with 180 messages) so I worry that > this change might annoy people. What do you think? Threads that long are already a bit of a pain to deal with. But I think this reveals a weakness in the way we're displaying threads more than anything. It seems silly to me that we would retrieve all 180 messages From=20such a thread, and format all of them, only to then hide all but the one that the reader is seeing. I think this "pick" series illustrates this nicely. Maybe it would be better to refactor the current emacs show mode to only retrieve parts for messages that are being displayed. I'm not sure how difficult that would be, though. I would also like to have access to more of the message headers in show mode. For instance, in emacs I want to access more headers with notmuch-message-headers than are currently available. Having json always include all headers might be a bit of a performance hit for some messages with lots of headers, but it would allow callers access to headers they don't currently have access to at all. So I think it would be nice and clean if the default json output included all the message headers and the message structure, and then part contents could be retrieved individually. I imagine show mode could be restructured such that it could use this efficiently. jamie. PS. where did the name "pick" come from? It doesn't seem to fit with the functionality to me. --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQIcBAEBCAAGBQJP4MpSAAoJEO00zqvie6q8HWIP/3zuS2RWqZVQyvq7SPhwCtFB 9zmWoYvHtTwmeGqgBCAbfVKZVzczSSVW7JIltDL5hT91IDnBY5VT5vifz0fecBz+ 1U9H0DT5fdU84WB8kzGgvzptHVEbO0QMWodiPRbFjfBk1WovB7EVN1OPfBQE27lw x7okLDEKg0wAXLdFnXC1swoPCxSYVv2Zr+joJwLBmyLBW85Hm+cEmP/ZrMAVrmBe zbl9phzD7XbSudU5UE0sg1AjKfcYbZpeqmgKqdXn38e+0JHQCZsWptBfu/8SDz09 9A6+dYSIVKJ8Gv25JkdI9gIKHgh8exc05DH+8KgDEa+L4srpbHpG3DfvedS1WpBy n4xQchF1DXcTk89G228pP0EaNLHR6eF82TqJ2vIfe+csTU0VnaxFsTbXS6Skup1F UvTedyC005el3Ej7quLI43JQa39CktsVT8sEUiqdi9HQpcdmC/PiU/hHdloLjEvS ym1qy45Vfi/hKmoLKVCERmkMVn+i0lkW76m7FVX1oDdM5LjqIYWRDF2JdhP5/whW VT2YCDbwdejKMbB906fZk9tMpvi/ui477gCOYMwf056kJ0iftsj/Jxolc08aHTs5 uyRd+5XWdUjCMdGb6NuuxSpoG1B7k7M5iZmSBaKmyDKR+MMPuGUzHBykbehTU4EF R3hVLYoSH5if5SkBFJv5 =5XeH -----END PGP SIGNATURE----- --=-=-=--