From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id 2IWGIIsml15aTQAA0tVLHw (envelope-from ) for ; Wed, 15 Apr 2020 15:21:47 +0000 Received: from aspmx2.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id EObJMI4ml16bPgAA1q6Kng (envelope-from ) for ; Wed, 15 Apr 2020 15:21:50 +0000 Received: from arlo.cworth.org (arlo.cworth.org [50.126.95.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (No client certificate requested) by aspmx2.migadu.com (Postfix) with ESMTPS id 7131E6813E1 for ; Wed, 15 Apr 2020 15:21:46 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 453C16DE0E8C; Wed, 15 Apr 2020 08:21:42 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XcYXQ0wWgz7V; Wed, 15 Apr 2020 08:21:41 -0700 (PDT) Received: from arlo.cworth.org (localhost [IPv6:::1]) by arlo.cworth.org (Postfix) with ESMTP id 3597C6DE09ED; Wed, 15 Apr 2020 08:21:41 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 022026DE09ED for ; Wed, 15 Apr 2020 08:15:13 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nPukgI5QQh6X for ; Wed, 15 Apr 2020 08:15:08 -0700 (PDT) X-Greylist: delayed 305 seconds by postgrey-1.36 at arlo; Wed, 15 Apr 2020 08:15:08 PDT Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) by arlo.cworth.org (Postfix) with ESMTPS id 5F62A6DE0944 for ; Wed, 15 Apr 2020 08:15:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1586963707; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=fyJ9RpjHI7mfZT4X53LUxipXQpmV5J0bcM7ll+albMk=; b=L2ALqFq7JWFi2MA4/D9TDiDCYUpwJ49g5cA5Q8BTsPA32FQzzR5H1a23l/XG32XgdG+AAf X4OHCtbhSpa0DLSKxrIE6byqxGMchDvE0XZOcZr8hIpLdH65GuHVV/qRyzvQ4FpeGjwis3 I0FyDk4Aa8kfh6TgYpWznTzcVbbDnZU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-480-wEeAHVIBO4KZZi4lb-i3kg-1; Wed, 15 Apr 2020 11:08:04 -0400 X-MC-Unique: wEeAHVIBO4KZZi4lb-i3kg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9C16F1060DB1 for ; Wed, 15 Apr 2020 15:08:03 +0000 (UTC) Received: from redhat.com (ovpn-116-138.rdu2.redhat.com [10.10.116.138]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5346D10016E8 for ; Wed, 15 Apr 2020 15:08:03 +0000 (UTC) Date: Wed, 15 Apr 2020 11:08:01 -0400 From: Don Zickus To: notmuch@notmuchmail.org Subject: performance problems with notmuch new Message-ID: <20200415150801.h2mazyo37sspvech@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline X-Mailman-Approved-At: Wed, 15 Apr 2020 08:21:39 -0700 X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: notmuch-bounces@notmuchmail.org Sender: "notmuch" X-Scanner: scn0 X-Spam-Score: 1.59 Authentication-Results: aspmx2.migadu.com; dkim=fail (body hash did not verify) header.d=redhat.com header.s=mimecast20190719 header.b=L2ALqFq7; dmarc=fail reason="SPF not aligned (relaxed)" header.from=redhat.com (policy=none); spf=pass (aspmx2.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 50.126.95.6 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Scan-Result: default: False [1.59 / 13.00]; MID_RHS_MATCH_FROM(0.00)[]; GENERIC_REPUTATION(0.00)[-0.45354006960445]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; SPF_REPUTATION_HAM(0.00)[-0.45208012254759]; IP_REPUTATION_HAM(0.00)[asn: 27017(-0.18), country: US(-0.01), ip: 50.126.95.6(-0.45)]; R_DKIM_REJECT(1.00)[redhat.com:s=mimecast20190719]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[notmuch@notmuchmail.org]; HAS_LIST_UNSUB(-0.01)[]; RCPT_COUNT_ONE(0.00)[1]; MX_INVALID(1.00)[]; R_SPF_ALLOW(-0.20)[+a]; DKIM_TRACE(0.00)[redhat.com:-]; FORGED_SENDER_MAILLIST(0.00)[]; MAILLIST(-0.20)[mailman]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:27017, ipnet:50.126.64.0/18, country:US]; FROM_NEQ_ENVFROM(0.00)[dzickus@redhat.com,notmuch-bounces@notmuchmail.org]; RCVD_COUNT_SEVEN(0.00)[10]; DMARC_POLICY_SOFTFAIL(0.10)[redhat.com : SPF not aligned (relaxed),none] X-TUID: 7LA4Y/YRG8GZ Hi, I have noticed my 'notmuch new' command seems awfully slow, maybe 10-20 emails / second on large batches. It goes quick for the first few hundred (maybe close to 100/second), then quickly slows down to about 10/second after processing the first 500 or so. I am guessing that isn't an expected behaviour. So I am trying to figure out a good way to analyze and debug this? This could be a problem with my fedora distro or laptop. I just don't know where to look. Tips for debugging this? I ran the notmuch performance/time-test, but after 15 minutes of waiting for the initial notmuch new to finish, I gave up and aborted. I am using 'glass' for my xapian storage if that helps. Help? Cheers, Don