From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 9863A6DE1290 for ; Mon, 21 Nov 2016 12:16:30 -0800 (PST) X-Virus-Scanned: Debian amavisd-new at cworth.org X-Spam-Flag: NO X-Spam-Score: 0.31 X-Spam-Level: X-Spam-Status: No, score=0.31 tagged_above=-999 required=5 tests=[AWL=0.330, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01] autolearn=disabled Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ICE6gSInSxpG for ; Mon, 21 Nov 2016 12:16:29 -0800 (PST) Received: from mail-wm0-f67.google.com (mail-wm0-f67.google.com [74.125.82.67]) by arlo.cworth.org (Postfix) with ESMTPS id 7E0B16DE128E for ; Mon, 21 Nov 2016 12:16:29 -0800 (PST) Received: by mail-wm0-f67.google.com with SMTP id u144so289474wmu.0 for ; Mon, 21 Nov 2016 12:16:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nikula-org.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=Y7EGIE0X5QaRkA2q8fKDBxy/NvtNK1mvCcPHRbNfWLk=; b=T8/Os4zHbhbzFdcwGPdkdIH75NpHKWWLJ/gyJIiIEeHfzMBWY3svpjnAkM5TrwxTpY APuGHtlPkmuZqaC9gkYX7KxiRXXx+Wi3C7QQhBqQ5cMOAhgicbznxZn8dumVOiQ+wPma nqlu+0wuly9RE2nYkHGIi6cPZ5xzG4nfW+05HxORAF9RnX532r5rWJ03o/c3B4FmJrXT cUIi8pOMQidaLARt1T0HG2yZmys7l1RJ1CqIJUSdCrSzl1RFP5CXPGX1M7XXjNmkBW7R qXqcT5Rdkj4+6NALFSgdvhzQq1bImb0zlrffikd8Xs59FLnzNdLwXgrfcDHJ3svAIlZu RQoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=Y7EGIE0X5QaRkA2q8fKDBxy/NvtNK1mvCcPHRbNfWLk=; b=enr76ad3Jv/KmJco6CEweOIp761fsCGrbN0Kfbl5OXzfdeLbuzO0BrhNMbmT/N+/CC uKxYJTala6wrsFkF380Su4sL1YJ7ylihZsbw6YpbDp3OrbylTRSpWiiaO4NPrLAYax3B zt5ywDzlrKHXbSOhRNgZy9xGIW72qZfbUWIHfEjW4XCzZGL7E7XOQ1PXetiYoWPfSxkw O3/v72d90VLr5oqPTyFueJtfle7u6rRCVYuRnXAJ7z1spXZ03X5+IqcCtS4l+NUycSe/ cvn0gKYuTNUpNbs+3zjyidGvRmI0OG8d+6ib3F+Ew04yCEe4kPDyxJevZEmNAHkSRR/5 nQ7A== X-Gm-Message-State: AKaTC03jhP4L13vDhAIpqm8g+oByhW3nYPcICy6k8YXfUe0aclch7Z0Y1BkD/wrChLFHmw== X-Received: by 10.25.21.205 with SMTP id 74mr3685609lfv.138.1479759387634; Mon, 21 Nov 2016 12:16:27 -0800 (PST) Received: from localhost (dsl-hkibrasgw2-58c368-70.dhcp.inet.fi. [88.195.104.70]) by smtp.gmail.com with ESMTPSA id h129sm5651038lfh.7.2016.11.21.12.16.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 21 Nov 2016 12:16:26 -0800 (PST) From: Jani Nikula To: David Bremner , Jani Nikula , Paul Wise , notmuch@notmuchmail.org Subject: [PATCH v2] cli: consider files vanishing during notmuch new non-fatal Date: Mon, 21 Nov 2016 22:14:33 +0200 Message-Id: <1479759273-18010-1-git-send-email-jani@nikula.org> X-Mailer: git-send-email 2.1.4 In-Reply-To: <87shqrwsye.fsf@tethera.net> References: <87shqrwsye.fsf@tethera.net> X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Nov 2016 20:16:30 -0000 If some software other than notmuch new renames or removes files during the notmuch new scan (specifically after scandir but before indexing the file), keep going instead of bailing out. Failing to index the file is just a race condition between notmuch and the other software; the rename could happen after the notmuch new scan anyway. It's not fatal, and we'll catch the renamed files on the next scan. Add a new exit code for when files vanished, so the caller has a chance to detect the race and re-run notmuch new to recover. Reported by Paul Wise at http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=843127 --- v2: use EX_TEMPFAIL status code --- notmuch-client.h | 11 +++++++++++ notmuch-new.c | 15 ++++++++++++--- 2 files changed, 23 insertions(+), 3 deletions(-) diff --git a/notmuch-client.h b/notmuch-client.h index 9ce2aef17431..793f32ecc55a 100644 --- a/notmuch-client.h +++ b/notmuch-client.h @@ -25,6 +25,7 @@ #define _GNU_SOURCE /* for getline */ #endif #include +#include #include "compat.h" @@ -114,6 +115,16 @@ chomp_newline (char *str) str[strlen(str)-1] = '\0'; } +/* Exit status code indicating temporary failure; user is invited to + * retry. + * + * For example, file(s) in the mail store were removed or renamed + * after notmuch new scanned the directories but before indexing the + * file(s). If the file was renamed, the indexing might not be + * complete, and the user is advised to re-run notmuch new. + */ +#define NOTMUCH_EXIT_TEMPFAIL EX_TEMPFAIL + /* Exit status code indicating the requested format version is too old * (support for that version has been dropped). CLI code should use * notmuch_exit_if_unsupported_format rather than directly exiting diff --git a/notmuch-new.c b/notmuch-new.c index c55dea7bc1b7..cc680b412a45 100644 --- a/notmuch-new.c +++ b/notmuch-new.c @@ -53,6 +53,7 @@ typedef struct { int total_files; int processed_files; int added_messages, removed_messages, renamed_messages; + int vanished_files; struct timeval tv_start; _filename_list_t *removed_files; @@ -280,11 +281,13 @@ add_file (notmuch_database_t *notmuch, const char *filename, case NOTMUCH_STATUS_FILE_NOT_EMAIL: fprintf (stderr, "Note: Ignoring non-mail file: %s\n", filename); break; - /* Fatal issues. Don't process anymore. */ case NOTMUCH_STATUS_FILE_ERROR: + /* Someone renamed/removed the file between scandir and now. */ + state->vanished_files++; fprintf (stderr, "Unexpected error with file %s\n", filename); (void) print_status_database ("add_file", notmuch, status); - goto DONE; + break; + /* Fatal issues. Don't process anymore. */ case NOTMUCH_STATUS_READ_ONLY_DATABASE: case NOTMUCH_STATUS_XAPIAN_EXCEPTION: case NOTMUCH_STATUS_OUT_OF_MEMORY: @@ -1151,5 +1154,11 @@ notmuch_new_command (notmuch_config_t *config, int argc, char *argv[]) if (!no_hooks && !ret && !interrupted) ret = notmuch_run_hook (db_path, "post-new"); - return ret || interrupted ? EXIT_FAILURE : EXIT_SUCCESS; + if (ret || interrupted) + return EXIT_FAILURE; + + if (add_files_state.vanished_files) + return NOTMUCH_EXIT_TEMPFAIL; + + return EXIT_SUCCESS; } -- 2.1.4