From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 7AEAB6DE01E4 for ; Wed, 5 Sep 2018 11:06:03 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org X-Spam-Flag: NO X-Spam-Score: 0.039 X-Spam-Level: X-Spam-Status: No, score=0.039 tagged_above=-999 required=5 tests=[AWL=0.039, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=disabled Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aiCD-Pws-4bQ for ; Wed, 5 Sep 2018 11:06:02 -0700 (PDT) Received: from mail-lj1-f179.google.com (mail-lj1-f179.google.com [209.85.208.179]) by arlo.cworth.org (Postfix) with ESMTPS id 0C0A36DE01D9 for ; Wed, 5 Sep 2018 11:06:01 -0700 (PDT) Received: by mail-lj1-f179.google.com with SMTP id p10-v6so7044395ljg.2 for ; Wed, 05 Sep 2018 11:06:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nikula-org.20150623.gappssmtp.com; s=20150623; h=from:to:subject:in-reply-to:references:date:message-id:mime-version; bh=AsM+39WL3GcfYto/MlCEz/4K6VA0gIiERD1fsi9Sag4=; b=gQ48XgA8KavlcQ1jCbHgKaWaA3qsHCCNFT39u2MpjWu20INNO6WJ9tlP8MjeiAwsmD cuaSONnAnqu/fxymjc4LLq41CswOaQvOiqfxbJ5AsUlHKEszG+SemEHMlRT10eY+BD21 CA5pnGrfvWTpz/0Z2bSMbxaSPYrTuv/2NYnrsUdDzKuheVftqUnUiNovLGRSRO718C0u wxcEQw8zjzUbEJArSvmf+bSlkKaV39f4SL7Bou6TTgsyxVafJ7ycEn+f7DE7sXSxn9Iu jHxwuBvmEyeeo0FXda2EKxuGM0u2O1CG17LxLiUt7w7G2pC4NGNxnWJlelz4DDjq/CdE H+Fw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:in-reply-to:references:date :message-id:mime-version; bh=AsM+39WL3GcfYto/MlCEz/4K6VA0gIiERD1fsi9Sag4=; b=qq8OXWMV5TawnsT945B3IHNTqOUeTlf5WRv6jnENJGo99IkN6PRlOFz7drRsZlO1RG kYrLX8jb+kyHI1h01T9VUv7YpPRNZ1TxkVRez/Ev1bHa6scQOGXbCtVh/qfkrFrIC3ut TCJsioOdPSh9g49MT8todsVvW/C0Jej3K+WM5C0nrydwsmSED1CTHkkA55jLn4fOWQ1A E+xss+V6VoQ1w0HB/jiMq0MWkNt3BUbrmMp5FfiRKgkvEc/K72F4VyCUYQEUlIJF9FIS xpNcaa4cuwgg9dHopWGSSUSQ7HQnLQqEHFBtLIAGeGKUxv8G//Bj6o4weUdDuitStruJ 8FdA== X-Gm-Message-State: APzg51DZRxpnYY2VytP1L8OJAWyN1cwaQQNg4FJwQLGjQVgWzW3I874E qOXS2m9JhAlNAsF3lz9LR74EQw== X-Google-Smtp-Source: ANB0Vdb332ywBir9o05LpPmNWMNrVRxUJNH0rokdooAix+AQ4xdzDR/rL2Bz05QRLgTP5zi2RSodPw== X-Received: by 2002:a2e:350b:: with SMTP id z11-v6mr24815113ljz.55.1536170758949; Wed, 05 Sep 2018 11:05:58 -0700 (PDT) Received: from localhost (82-203-184-217.bb.dnainternet.fi. [82.203.184.217]) by smtp.gmail.com with ESMTPSA id 13-v6sm420818lfy.87.2018.09.05.11.05.56 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 05 Sep 2018 11:05:57 -0700 (PDT) From: Jani Nikula To: mueen@nawaz.org, notmuch@notmuchmail.org Subject: Re: Notmuch DB Problems In-Reply-To: <47409d4ed692a336458371102bcbcbd86ab4a067@webmail.nawaz.org> References: <47409d4ed692a336458371102bcbcbd86ab4a067@webmail.nawaz.org> Date: Wed, 05 Sep 2018 21:05:53 +0300 Message-ID: <874lf3es5a.fsf@nikula.org> MIME-Version: 1.0 Content-Type: text/plain X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2018 18:06:03 -0000 On Tue, 04 Sep 2018, mueen@nawaz.org wrote: > Hi, > > A few days ago I noticed notmuch new was no longer working (I have it > as a cron job so it took a while to figure it out). > > It just freezes. I do have a Python hook, and it was freezing on the > line that opens the database. > > I tried a notmuch dump. Same problem - freezes > > Based on some earlier threads, I tried a notmuch compact. Same problem > - freezes. > > All these freezes seem to use no memory/CPU. > > Interestingly, queries work fine - from both the command line and the > Emacs interface. So I can read old stuff just fine. But all the > commands above cause a freeze. > > Currently using notmuch-0.24.2. I tried notmuch-0.27 - same problem. > > Results of a xapian check: > > docdata: > blocksize=8K items=6 firstunused=3 revision=6442 levels=0 root=0 > B-tree checked okay > docdata table structure checked OK > > termlist: > blocksize=8K items=178562 firstunused=53441 revision=6442 levels=2 > root=46086 > /usr/bin/xapian-check: DatabaseError: 1 unused block(s) missing from > the free list, first is 0 > > What are my options? Unfortunately the last dump I have is many months > old, so I'm a bit wary of deleting the database and rebuilding. Given > that the show and search commands work, I was wondering if I can write > a script to get all the message/thread ID's for all the tags and store > them, and then rebuild the database and use that stored information to > retag all my messages (all without using the dump command)? It might be interesting to see an strace log to possibly get an idea where it gets stuck. Is the filesystem writable and working okay? If search and show work, I'm guessing it gets stuck in trying to open the database writable. One hackish idea is to patch notmuch dump to open the database in read-only mode, and dump the tags. See below. The dump command opens the database writable to prevent changes while dumping. (Arguably this could be a command line option for cases like yours.) BR, Jani. diff --git a/notmuch-dump.c b/notmuch-dump.c index ef2f02dfeb5c..d06dbcf50224 100644 --- a/notmuch-dump.c +++ b/notmuch-dump.c @@ -364,7 +364,7 @@ notmuch_dump_command (notmuch_config_t *config, int argc, char *argv[]) int ret; if (notmuch_database_open (notmuch_config_get_database_path (config), - NOTMUCH_DATABASE_MODE_READ_WRITE, ¬much)) + NOTMUCH_DATABASE_MODE_READ_ONLY, ¬much)) return EXIT_FAILURE; notmuch_exit_if_unmatched_db_uuid (notmuch);