From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id 840DC431FC3 for ; Fri, 9 Nov 2012 13:26:02 -0800 (PST) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none] autolearn=disabled Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yveKxeW6y-Bj for ; Fri, 9 Nov 2012 13:26:00 -0800 (PST) Received: from guru.guru-group.fi (guru.guru-group.fi [46.183.73.34]) by olra.theworths.org (Postfix) with ESMTP id 0ECE7431FAE for ; Fri, 9 Nov 2012 13:26:00 -0800 (PST) Received: from guru.guru-group.fi (localhost [IPv6:::1]) by guru.guru-group.fi (Postfix) with ESMTP id CA202100094; Fri, 9 Nov 2012 23:25:58 +0200 (EET) From: Tomi Ollila To: Mark Walters , notmuch@notmuchmail.org Subject: Re: [PATCH 0/3] Outline fix for emacs tagging race In-Reply-To: <1352487491-31512-1-git-send-email-markwalters1009@gmail.com> References: <1352487491-31512-1-git-send-email-markwalters1009@gmail.com> User-Agent: Notmuch/0.14+84~g8a199bf (http://notmuchmail.org) Emacs/24.2.1 (x86_64-unknown-linux-gnu) X-Face: HhBM'cA~ MIME-Version: 1.0 Content-Type: text/plain X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Nov 2012 21:26:03 -0000 On Fri, Nov 09 2012, Mark Walters wrote: > > This patch series implements the above mentioned solution and seems to > work except for one problem. > > Since emacs now tags each message in a thread in the search buffer it > is easy to ask it to tag a lot of messages. This could be done > individually which would be ridiculously slow so instead they are all > done in one batch. But now it is relatively easy to take notmuch over > the threshold for ARG_MAX. > > In [3] Tomi did some experiments and found on a standard Debian system > with getconf ARG_MAX =131072 that command lines with 10000 200 byte > arguments worked. I am a little puzzled by that as I get the same > results and I getconf ARG_MAX gives 2097152 for me. > > More importantly though, when trying to execute a command from emacs I > am finding that 131072 is the limit on the command length in bytes and > we can hit this with something around 1500 messages (see end for a > very hacky emacs test script). This is probably more than we can > expect in a single thread so tagging from show is probably safe but it > does cause a problem when tagging from search. That's because your test script below constructs one very loooong string (and used just one arg). Like you said, [3] used 10000 200-char args. This test script is more like what the perl(1) script did in [3] (progn (setq arglist '("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa")) (setq n 10000) (while (> n 0) (setq n (1- n)) (setq arglist (cons "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" arglist))) (apply 'call-process "/bin/echo" nil t nil arglist)) (except that the args here are just 80 chars but you should get the point :) I did not check your patches yet.. buf if you concatenate there you could try to provide those as separate args... ... then we can experiment how notmuch (and xapian) copes with thosew IIRC notmuch will concatenate one long string out of those and do xapian stuff using that (but then we're not limited by command line arguments restrictions). > > Best wishes > > Mark Tomi > > [3] id:m2liody7av.fsf@guru.guru-group.fi > TEST SCRIPT > (progn > (setq bigstring > "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa") > (setq n 1310) > (setq big nil) > (while (> n 0) > (setq n (1- n)) > (setq big (concat big (format "%s" n) " " bigstring))) > (call-process "echo" nil t nil big))