From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id QMJvJGIWj2R2NAEASxT56A (envelope-from ) for ; Sun, 18 Jun 2023 16:36:18 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id UKcDI2IWj2RVCgEAG6o9tA (envelope-from ) for ; Sun, 18 Jun 2023 16:36:18 +0200 Received: from mail.notmuchmail.org (yantan.tethera.net [135.181.149.255]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id E675239EB9 for ; Sun, 18 Jun 2023 16:36:17 +0200 (CEST) Received: from yantan.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id 36BA55E01C; Sun, 18 Jun 2023 14:36:15 +0000 (UTC) Received: from mail.bitfolk.com (mail.bitfolk.com [IPv6:2001:ba8:1f1:f019::25]) by mail.notmuchmail.org (Postfix) with ESMTPS id 476975DC0A for ; Sun, 18 Jun 2023 14:36:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=strugglers.net; s=alpha; h=In-Reply-To:Content-Type:MIME-Version:References :Message-ID:Subject:To:From:Date:Content-Transfer-Encoding:Sender:Reply-To:Cc :Content-ID:Content-Description:Resent-To; bh=3kb6X6+LWLUAWRn2AfK0Wbd5NUWdSHFv6EEcIBty6vM=; b=nDi2whZunSRW09KFtOi9z8cYFn 4D1YNl9LUH95vCImgyM0LSfeFgoh3u8wqYMDsC9XddCV3TnAOM527/dFM1MFa156jsetARH5K6flw NwwqLBHCr0cbcx4xgXtodvLR4Yi9Bv6zevCscxpXdy2a3eGKnC48+iZlzbKBZNhPtBd5ZVHr5gqAm 5s6/yYtudu4HCG8SXU8QXmTKs5gCM0bQy+gzMU+iJTy9VJg2E7Il7u3wp1rpijnvblq4vACjV4lkG R/Sd1NyGfLmHVoJFCP2ccq1fVV9Bt22/N91UKmgdy53he2XGB/9LFye9uBq0gBGjlJKQlw5T0Xgo3 mKgdHQ+Q==; Received: from andy by mail.bitfolk.com with local (Exim 4.94.2) (envelope-from ) id 1qAtVe-0007Nu-J5 for notmuch@notmuchmail.org; Sun, 18 Jun 2023 14:36:06 +0000 Date: Sun, 18 Jun 2023 14:36:06 +0000 From: Andy Smith To: notmuch@notmuchmail.org Subject: Re: "notmuch compact" questions Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: OpenPGP: id=BF15490B; url=http://strugglers.net/~andy/pubkey.asc X-URL: http://strugglers.net/wiki/User:Andy X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: andy@strugglers.net X-SA-Exim-Scanned: No (on mail.bitfolk.com); SAEximRunCond expanded to false Message-ID-Hash: UQCO2B2SV65YD6GODJINW3MWLMDQDCX4 X-Message-ID-Hash: UQCO2B2SV65YD6GODJINW3MWLMDQDCX4 X-MailFrom: andy@strugglers.net X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.3 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Country: DE X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1687098978; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-owner:list-unsubscribe:list-subscribe:list-post:dkim-signature: openpgp:openpgp; bh=NE962hsZBiIqdLnB2QxGlufIZjnR9SJhdy+lT7dazjo=; b=V2WB2+SO24deVOKyFyrF3ZIaPxe3vam8X3mVxLyHujYWOEn9dokvMWn/P0nb9GQY4G0Yd9 L2ts36QUjxZWh1N7Gt1G94neGONekJ76qx5VqMQp0z4bER6fR3crmW4WfUDkenT1r3uWzk A5QxaMBnW5IJoXXHOVThb1c0DxKahug/nQjUK9DtOFy9lfhOlKmdydlgvvNKbDHSZBaHcl /HaW1MeGTRmTIU8gVV97n7MOUer+QgpkOsZr/kiZ9JOwp18DtlqtREmTFdOOVCVm68EXuQ xZscW/IWE6FDVo82u7cAI09n4d0di7nbw2YKRfSSMUKb1qSa4THXdXP21hBOVA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=strugglers.net header.s=alpha header.b=nDi2whZu; dmarc=fail reason="SPF not aligned (relaxed)" header.from=strugglers.net (policy=none); spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 135.181.149.255 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org ARC-Seal: i=1; s=key1; d=yhetil.org; t=1687098978; a=rsa-sha256; cv=none; b=L7INZTJNozbsRGTpJUsLYVvIp2dG1qFuyNkhcbA9m9+kkxviKnI1MwFZDfLXiCJ5J7GEvQ DVamK7VrVfuhVCmM36O39IkOUsGe91BcvV3cKxyn/In4/S9+qDSZpiinX+KYKm6yoFND2S 1gXHeUEXG/Da1i4vSOFNzfmc9cf+VH0tBvPRS5M+tHJNAeDkOaHBmvCr/KHFzTmUe0OfIe IaMPFAPlRj1cE5mY1GbtJvfivIECnmKEpceHSOreHKoLb8ZLvlqUam7PU2lXHVIXHFRndr 2SGvtG94UlDiwXSFYizWu/fz3osP04Ycn0tdPRJZiCqvRjDldu93DhU3m4/5Gg== X-Migadu-Scanner: scn1.migadu.com X-Migadu-Spam-Score: -0.04 Authentication-Results: aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=strugglers.net header.s=alpha header.b=nDi2whZu; dmarc=fail reason="SPF not aligned (relaxed)" header.from=strugglers.net (policy=none); spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 135.181.149.255 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: E675239EB9 X-Spam-Score: -0.04 X-TUID: grGG2toyiXWT Hi Michael, On Sun, Jun 18, 2023 at 01:36:30PM +0200, Michael J Gruber wrote: > On my comparatively small db, gains are typically substantial > (postlist, position), even though I compact from time to time. Thanks for your help. The savings were indeed a lot more than I expected (and the compact didn't take anywhere near as long as I thought it would, either): $ sudo compsize ~/.notmuch Processed 7 files, 5159972 regular extents (5484004 refs), 2 inline. Type Perc Disk Usage Uncompressed Referenced TOTAL 60% 51G 85G 56G none 100% 18G 18G 16G zstd 49% 32G 66G 39G $ notmuch compact Compacting database... compacting table postlist Reduced by 62% 8838736K (14138192K -> 5299456K) compacting table docdata Reduced by 50% 1168K (2304K -> 1136K) compacting table termlist Reduced by 51% 6816008K (13112728K -> 6296720K) compacting table position Reduced by 54% 17223112K (31731384K -> 14508272K) compacting table spelling doesn't exist compacting table synonym doesn't exist Done. $ sudo compsize ~/.notmuch Processed 7 files, 186246 regular extents (187018 refs), 2 inline. Type Perc Disk Usage Uncompressed Referenced TOTAL 56% 14G 25G 25G none 100% 3.7G 3.7G 3.7G zstd 48% 10G 21G 21G I don't know why so much space should have been saved, though it may be something to do with the fact that I indexed everything and then decided to move a million or so messages around into a different folder structure. Thanks, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting