From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 6B3FA6DE0183 for ; Thu, 28 Dec 2017 14:54:51 -0800 (PST) X-Virus-Scanned: Debian amavisd-new at cworth.org X-Spam-Flag: NO X-Spam-Score: 0.438 X-Spam-Level: X-Spam-Status: No, score=0.438 tagged_above=-999 required=5 tests=[AWL=-0.195, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_NEUTRAL=0.652, UNPARSEABLE_RELAY=0.001] autolearn=disabled Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LsQL8uteGK2e for ; Thu, 28 Dec 2017 14:54:50 -0800 (PST) Received: from mail-wm0-f52.google.com (mail-wm0-f52.google.com [74.125.82.52]) by arlo.cworth.org (Postfix) with ESMTPS id 5A0986DE014D for ; Thu, 28 Dec 2017 14:54:50 -0800 (PST) Received: by mail-wm0-f52.google.com with SMTP id f21so2637227wmh.1 for ; Thu, 28 Dec 2017 14:54:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dme-org.20150623.gappssmtp.com; s=20150623; h=to:subject:from:date:message-id:mime-version :content-transfer-encoding; bh=qojLxv50Vm6QbENs2DAz1BpgRLiBd/2gLLzHrlQn30w=; b=TK4pby5IdNA+6DRSz7ox84MLqQ6+AAxVWVKb3uG2UbQLcThzD64yh7LE0MEqSfEFTL bUz0hvDAtpCgPMG84jUppHHMnt7aTpzj7XuzDypEYEYfA+eKjMlZrhmPjWGFj5cEXXr5 xZAR2MMJr1GX/nYSrr/cc/qeMA3AQNbDFeVWu1QbyY5x+oVJRSftdkSYZrWZpnqhQdJX a0stAruHL5Bfh9ah1s6KHZc0u/aBfP2sN5ox0EBGEvHDTkXGk5XMEvqo7Pe6xWZiAsJn fbUo5qU8GVP6pzruZaLnwpeOeGQDIu4X1MJ0x3OpAUObTT9huVOBSiheNYYFWXLvm36O PxFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:subject:from:date:message-id:mime-version :content-transfer-encoding; bh=qojLxv50Vm6QbENs2DAz1BpgRLiBd/2gLLzHrlQn30w=; b=V3Dehc3ynWf+0cdSJ7LmIzrDPpGY+OtsT/GKC3cLVNID31c9t5c7QdMgVgDAkE0DYV lJOxcvyIpC0hx7KvbOhQftlG5ropHYAAgji0geSgzM8TYTNi7ln7kVsYFPzqKtvVO5UG afQq7OqeYtr7HiutLC2m4SN8SnF4KosCvOzl318HgMCjEq0d0FDFDSoom62rcXPxJgY8 RWWUXL3o1IrKBvJq11p/AQUhFq8S9sTrStSj9GoULBptq4fQYZ3jCnSk2srh9PNIvTRW PQtCe1aGXRY1KU1dNuw+UpXGx8c+//N2KjM2SfuoA4idPLxzFwMvMcgVzu2Sqr+cU3hw gmdQ== X-Gm-Message-State: AKGB3mJMxxrA80yb3bbz8RyItw/ov41lqtK88JJ5l8tFm804VXSNBroo PPIZNKlrQqlUMXBAdIrfkZ/xHW27fLc= X-Google-Smtp-Source: ACJfBosz6Ndf/K74b5qcdCIcbLEfV2nLfccKlRbvQCuwn9LGVFpUiCTmxHBQ1DiB9Xkuf3jiYHnH4Q== X-Received: by 10.28.138.75 with SMTP id m72mr28914190wmd.97.1514501687999; Thu, 28 Dec 2017 14:54:47 -0800 (PST) Received: from disaster-area.hh.sledj.net (disaster-area.hh.sledj.net. [81.149.164.25]) by smtp.gmail.com with ESMTPSA id i33sm29162573wri.90.2017.12.28.14.54.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 28 Dec 2017 14:54:46 -0800 (PST) Received: from localhost (disaster-area.hh.sledj.net [local]) by disaster-area.hh.sledj.net (OpenSMTPD) with ESMTPA id 5e4de548; Thu, 28 Dec 2017 22:54:45 +0000 (UTC) To: notmuch@notmuchmail.org Subject: Xapian exception leading to database corruption X-HGTTG: zarniwoop From: David Edmondson Date: Thu, 28 Dec 2017 22:54:45 +0000 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Dec 2017 22:54:51 -0000 Using current git notmuch on Debian testing a rebuild from scratch of my database fails: > agrajag-testing ~/s/notmuch % ./notmuch new > Found 605510 total files (that's not much mail).=20=20=20=20 > add_file: A Xapian exception occurred 28m 37s remaining). > A Xapian exception occurred adding message: Unexpected end of posting lis= t for 'G0000000000014364'. > Processed 137296 total files in 8h 4m 45s (4 files/sec.). > Added 135950 new messages to the database. > Note: A fatal error was encountered: A Xapian exception occurred > agrajag-testing ~/s/notmuch %=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20 Checking the database at this point seems fine: > agrajag-testing ~/s/notmuch % xapian-check ~/Maildir/.notmuch/xapian=20 > docdata: > blocksize=3D8K items=3D0 firstunused=3D1 revision=3D1 levels=3D0 root=3D(= faked) > void B-tree checked okay > docdata table structure checked OK >=20 > termlist: > blocksize=3D8K items=3D0 firstunused=3D2 revision=3D1 levels=3D0 root=3D(= faked) > void B-tree checked okay > termlist table structure checked OK >=20 > postlist: > blocksize=3D8K items=3D2 firstunused=3D1 revision=3D1 levels=3D0 root=3D0 > B-tree checked okay > postlist table structure checked OK >=20 > position: > blocksize=3D8K items=3D0 firstunused=3D1 revision=3D1 levels=3D0 root=3D(= faked) > void B-tree checked okay > position table structure checked OK >=20 > spelling: > Lazily created, and not yet used. >=20 > synonym: > Lazily created, and not yet used. >=20 > No errors found > agrajag-testing ~/s/notmuch %=20 If I run =E2=80=9Cnotmuch new=E2=80=9D again then the exception repeats, ex= cept the database is listed as corrupt (this is actually from an earlier cycle of testing, but it is reproducible): > agrajag-testing ~/s/notmuch % ./notmuch new > add_file: A Xapian exception occurred > A Xapian exception occurred adding message: Unexpected end of posting lis= t for 'G0000000000013f27'. > Processed 16 total files in 59s (0 files/sec.). > Added 15 new messages to the database. > Note: A fatal error was encountered: A Xapian exception occurred > agrajag-testing ~/s/notmuch % xapian-check ~/Maildir/.notmuch/xapian=20=20 > docdata: > blocksize=3D8K items=3D15 firstunused=3D3 revision=3D9 levels=3D0 root=3D2 > B-tree checked okay > docdata table structure checked OK >=20 > termlist: > blocksize=3D8K items=3D301848 firstunused=3D68658 revision=3D9 levels=3D2= root=3D14019 > xapian-check: DatabaseError: 1 unused block(s) missing from the free list= , first is 0 > agrajag-testing ~/s/notmuch %=20 I see that bremner reported something like this in #xapian, but not any resolution. Any suggestions? Is it possible to force a new chert database to be created rather than glass? (Mostly I'd like to get back to work!) dme. --=20 But uh oh, I love her because, she moves in her own way.