From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id SNFlK5o6KGNJQAEAbAwnHQ (envelope-from ) for ; Mon, 19 Sep 2022 11:47:06 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id YEA+K5o6KGMpNAAAauVa8A (envelope-from ) for ; Mon, 19 Sep 2022 11:47:06 +0200 Received: from mail.notmuchmail.org (yantan.tethera.net [135.181.149.255]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id F0EE638853 for ; Mon, 19 Sep 2022 11:47:05 +0200 (CEST) Received: from yantan.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id 5141D5F365; Mon, 19 Sep 2022 09:47:03 +0000 (UTC) Received: from fethera.tethera.net (fethera.tethera.net [198.245.60.197]) by mail.notmuchmail.org (Postfix) with ESMTP id 0C74F5E552 for ; Mon, 19 Sep 2022 09:47:01 +0000 (UTC) Received: by fethera.tethera.net (Postfix, from userid 1001) id AA0BB5FBC0; Mon, 19 Sep 2022 05:46:59 -0400 (EDT) Received: (nullmailer pid 3269744 invoked by uid 1000); Mon, 19 Sep 2022 09:46:58 -0000 From: David Bremner To: Jose A Ortega Ruiz , notmuch@notmuchmail.org Subject: Re: Notmuch emacs client is slow when opening inbox. In-Reply-To: <87r108p0xm.fsf@mail.jao.io> References: <875yhmfglb.fsf@gmail.com> <871qs9riqd.fsf@mail.jao.io> <87zgew6tpv.fsf@tethera.net> <87r108p0xm.fsf@mail.jao.io> Date: Mon, 19 Sep 2022 06:46:58 -0300 Message-ID: <87wn9z7l2l.fsf@tethera.net> MIME-Version: 1.0 Message-ID-Hash: GZIZ3XG2V5OS25CL4CMYUMAFJPYH5L7P X-Message-ID-Hash: GZIZ3XG2V5OS25CL4CMYUMAFJPYH5L7P X-MailFrom: david@tethera.net X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.3 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: DE ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1663580826; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-owner:list-unsubscribe:list-subscribe:list-post; bh=Qdhz09Je/n2bCS9IY8y4kxUVKip38meFTHG15gKZ6OA=; b=pQ51y3bOSFyHuWsl+e+NyIWozwNRrTfANHU41HScESw2z/63eIrDMbfj/cM23Q4ODFiq5X 5Q5JiJJgBIky5h2QWN3Ie/9fOpIURC/yJrHpaoQAcT2XJ93Tj7op9bBV01gytL/7dFyLBA JkfKjPzlMENjdbLHpTgN6GKxxZj+Qy7MqUa8Il73yizgLhVcjgNdYX1b+ZjKsAVtaBB/CC AJE/JB1ZjtPvVccgBzJIiPjQeLJKub0ecZb7IwgkPrECAA2NOFmBPUwuIont3+y7nlimPo tv1CKzMP2VLlQUf7+IHWQrcO4nB6es46Q7/gjNE2nxpAfDpRfo6tDzJHSsHlZw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1663580826; a=rsa-sha256; cv=none; b=LknONuhmGfONh5Yqfq4InzS8S74y9SUV9u8ChUbnpIo48t1Zmlu4814d+NYh42Qd2yWC33 gJo8CEy5slQC2gNPh5Jl+BwRUETOwPhG9GNJ2fqMx4yN91J78BdF8OaoUdSbP0sC5Zkkf6 H21uwLOmGGEzL1JDUqaFBq1wEb6RkCc2UPTwYuLhxpnZzDATRksGUVJ8G4anx1VmTm3Jnt 1yByajiP83B73omglHWQgZ39dzXY7TVeGFac1oluZtFExHiEv1n2n2MYvp6LNVUDXShiK7 Rq8I50BU6Sw/7R11Y7O5Ef4gvyggQNFKI+UMhUh8pmSWRDoQ+QGhBt5exEceIQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 135.181.149.255 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Spam-Score: -1.46 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 135.181.149.255 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: F0EE638853 X-Spam-Score: -1.46 X-Migadu-Scanner: scn1.migadu.com X-TUID: bxeu/zizV3D/ Jose A Ortega Ruiz writes: > On Sun, Sep 18 2022, David Bremner wrote: > >> Jose A Ortega Ruiz writes: >>> >>> interesting! if we were to have this, i think i'd prefer pagination >>> rather than increasing the limit and keeping the old ones, and make it >>> work for tree searches too. and perhaps the limit (page size) could be >>> a query parameter, so that one could have different page sizes for >>> different queries. if that makes sense (and i'm not missing a better >>> way already in place), i might try to implement it when i have a bit of >>> time. >> >> Before going to far with pagination, maybe get Jani to recap the >> difficulties he saw at the time that led him to propose the simpler >> approach. >> >> I can think of a few issues, but I don't really know if they are >> blockers. >> - the result set is going to vary dynamically as messages are added and >> removed from the database >> >> - As far as I can tell, there isn't really an easy way to jump to page >> n of the results. > > fwiw, my idea here was to always run the full, non-paginated query, and > populate its result buffer just as now, but show, instead of that > buffer, an indirect buffer pointing to it, narrowed to the page at hand. [snip] > not as efficient as a limit/offset query per page, but i think the > bottleneck here is in displaying results rather than running the query > in most cases, isn't it? Ah, indeed that sounds like a different approach than was previously mooted. I guess it might be a simple-ish way of making the display more lazy, which I agree is most likely the bottleneck most people are seeing.