From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id 566D1429E28 for ; Thu, 26 May 2011 13:18:55 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: -0.699 X-Spam-Level: X-Spam-Status: No, score=-0.699 tagged_above=-999 required=5 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id l1k6gRNVKev9 for ; Thu, 26 May 2011 13:18:54 -0700 (PDT) Received: from mail-qw0-f53.google.com (mail-qw0-f53.google.com [209.85.216.53]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by olra.theworths.org (Postfix) with ESMTPS id 6EDFB431FB6 for ; Thu, 26 May 2011 13:18:54 -0700 (PDT) Received: by qwb7 with SMTP id 7so723465qwb.26 for ; Thu, 26 May 2011 13:18:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=GEFCyn1hdkFdnm9bJLZnOPjUk7Km+3iLvPG5ntw8ZdU=; b=hnUXrYC75xY8g79i4Jb5IauqrYAN0X1YMXJtI7oZLwQk1CXachC7I0Kr729r0eaHyZ 4b2liaSTK7sOgyd8ARTxHCurpSKr+FlBoIqjHSTFJ4jOwsxMbYu5YSL+iGhWTeCLyeOZ PmWFIzcFv26HpA5P134lCaz6zvDes2zM2tdg8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=eWithQF5A6v2jJCJU5f5qtp8L5Fv4G7lZQwQwim+dM4OOuuaD2Gu3AGpnH13Ql7Ysn CbwatW2PGHzUcHY4OTR5duQ08ABN0igl7EsJC28A9SUEErNdLzE63izd3ZawZanMLAI6 xdLW7fa5DC+JFYEY+F/nQ23zmgt/x5f11H21o= MIME-Version: 1.0 Received: by 10.229.142.11 with SMTP id o11mr1003970qcu.46.1306441133374; Thu, 26 May 2011 13:18:53 -0700 (PDT) Sender: amdragon@gmail.com Received: by 10.229.188.68 with HTTP; Thu, 26 May 2011 13:18:53 -0700 (PDT) In-Reply-To: <877h9d9y5m.fsf@yoom.home.cworth.org> References: <1306397849-sup-3304@brick> <877h9d9y5m.fsf@yoom.home.cworth.org> Date: Thu, 26 May 2011 16:18:53 -0400 X-Google-Sender-Auth: f8JApCG5-0jsiDTITBVtmmMXEbU Message-ID: Subject: Re: one-time-iterators From: Austin Clements To: Carl Worth , Patrick Totzke Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: notmuch X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 May 2011 20:18:55 -0000 On May 26, 2011 1:20 PM, "Carl Worth" wrote: > > The question: How do you solve this in the emacs code? > > do you store all tids of a query? > > The emacs code does not use the notmuch library interface like your > python bindings do. Instead, it uses the notmuch command-line tool, (and > buffers up the text output by it). The support for asynchronous > operations in the emacs interface means that it's likely possible > someone could run into a similar problem: > > =A0 =A0 =A0 =A01. Start a search returning a *lot* of results > > =A0 =A0 =A0 =A02. When the first results come in, make some tag changes > > =A0 =A0 =A0 =A03. See if the original search aborts > > I may have even had this happen to me before, but if I did I've never > actually noticed it. I don't know what a good answer might be for this > problem. I proposed a solution to this problem a while ago (id:"AANLkTi=3DKOx8aTJipkiArFVjEHE6zt_JypoASMiiAWBZ6@mail.gmail.com"), though I haven't tried implementing it yet. Though, Patrick, that solution doesn't address your problem.=A0 On the other hand, it's not clear to me what concurrent access semantics you're actually expecting.=A0 I suspect you don't want the remaining iteration to reflect the changes, since your changes could equally well have affected earlier iteration results.=A0 But if you want a consistent view of your query results, something's going to have to materialize that iterator, and it might as well be you (or Xapian would need more sophisticated concurrency control than it has).=A0 But this shouldn't be expensive because all you need to materialize are the document ids; you shouldn't need to eagerly fetch the per-thread information. Have you tried simply calling list() on your thread iterator to see how expensive it is? My bet is that it's quite cheap, both memory-wise and CPU-wise.