From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 2717A6DE0159 for ; Sat, 5 Nov 2016 05:58:59 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org X-Spam-Flag: NO X-Spam-Score: -0.017 X-Spam-Level: X-Spam-Status: No, score=-0.017 tagged_above=-999 required=5 tests=[AWL=0.703, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01] autolearn=disabled Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id v70DM5I7mX0X for ; Sat, 5 Nov 2016 05:58:58 -0700 (PDT) Received: from mail-lf0-f46.google.com (mail-lf0-f46.google.com [209.85.215.46]) by arlo.cworth.org (Postfix) with ESMTPS id 745D36DE00D3 for ; Sat, 5 Nov 2016 05:58:57 -0700 (PDT) Received: by mail-lf0-f46.google.com with SMTP id c13so82766271lfg.0 for ; Sat, 05 Nov 2016 05:58:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nikula-org.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=B4e2evuU6k1TNh3Y6d9EC/I3El5KA0d1WTOuQ8URNls=; b=XbIPzVLwWtq3AmJt2PDyb0mlxDr0LYwhuN3PlYrvVgS7kmaBYZ8p1ppvYu74WThezk JHvfFyLzlfO3B8eB6zbkkh1/Oc34BBVJGEQhMgEg27BG+1cPygqmi73I3OCaL25MfHCc I8mK+iW6T+v5m8qb4DhgcUx8tyFRe8xvahT7C5XhHJgQZ0clukebc/nJh4+BjFATkisV ZnqOMZJtpFZmjy5BRwYKoHB8sHuoplG81ATyzovfF7Hx8hkQB2GTM2ibvTPyZKk635jm 9E4Y0/F4pziYTivMA0W4WRXkmnwpyE4SLAfE0MeaV9SU2rPgLs/sYBReJUbx7qv3PIop 7jzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=B4e2evuU6k1TNh3Y6d9EC/I3El5KA0d1WTOuQ8URNls=; b=C4fEly+sUzqf5BpLsMDZKGv/QC2t2uvZwZfPP6FoSaKU1qD4R8Z9C7xiSFSyYb6Yl6 x/pd/mI82hCT1LmX+hheLMu2OP1N+IVWI3ZfYdr02pbOT0NFq4I0WYksnnk0DTKTItRz WEEP98J7JQoBHmVsyGOkYADKQcJJhKyWAscsRsvZEuk5FFLn8CD+OyOfrUfGzYCgV4oa osn6nREGV0YtSnry0hG3HC8TGnOr1GZ58b0BKYECxHkBUlzTlR0nOrm2Tfa7Jk/xbcpv 9m4r2JiEJBna3REhYEH1Uz0kQBNw67GXEDj4eM8O1MufBVAhEUTQAqalWJmhC69tp38x dTIw== X-Gm-Message-State: ABUngvdkfgYPkqDmmlCTvvG6bEmZEJeX6KQCjCLBX04F20/Q15ci8CwtaBPvM+sG0VYudw== X-Received: by 10.25.228.88 with SMTP id b85mr55030lfh.100.1478350735245; Sat, 05 Nov 2016 05:58:55 -0700 (PDT) Received: from localhost (dsl-hkibrasgw2-58c368-70.dhcp.inet.fi. [88.195.104.70]) by smtp.gmail.com with ESMTPSA id z76sm3240421lff.35.2016.11.05.05.58.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 05 Nov 2016 05:58:54 -0700 (PDT) From: Jani Nikula To: Paul Wise , David Bremner , notmuch@notmuchmail.org Subject: [PATCH] cli: consider files vanishing during notmuch new non-fatal Date: Sat, 5 Nov 2016 14:57:01 +0200 Message-Id: <1478350621-17137-1-git-send-email-jani@nikula.org> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1478312104.1979.8.camel@debian.org> References: <1478312104.1979.8.camel@debian.org> X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Nov 2016 12:58:59 -0000 If some software other than notmuch new renames or removes files during the notmuch new scan (specifically after scandir but before indexing the file), keep going instead of bailing out. Failing to index the file is just a race condition between notmuch and the other software; the rename could happen after the notmuch new scan anyway. It's not fatal, and we'll catch the renamed files on the next scan. Add a new exit code for when files vanished, so the caller has a chance to detect the race and re-run notmuch new to recover. Reported by Paul Wise at http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=843127 --- Having notmuch new re-run (parts of) the scan automatically seems a rather more involved change. So does inotify support. This simple change both finishes the scan and lets the user recover, IMO a reasonable first step. Please suggest a better alternative to "vanish" in code... --- notmuch-client.h | 8 ++++++++ notmuch-new.c | 15 ++++++++++++--- 2 files changed, 20 insertions(+), 3 deletions(-) diff --git a/notmuch-client.h b/notmuch-client.h index 9ce2aef17431..d2057e26c5cd 100644 --- a/notmuch-client.h +++ b/notmuch-client.h @@ -114,6 +114,14 @@ chomp_newline (char *str) str[strlen(str)-1] = '\0'; } +/* Exit status code indicating that file(s) in the mail store were + * removed or renamed after notmuch new scanned the directories but + * before indexing the file(s). If the file was renamed, the indexing + * might not be complete, and the user is advised to re-run notmuch + * new. + */ +#define NOTMUCH_EXIT_VANISHED_FILES 10 + /* Exit status code indicating the requested format version is too old * (support for that version has been dropped). CLI code should use * notmuch_exit_if_unsupported_format rather than directly exiting diff --git a/notmuch-new.c b/notmuch-new.c index c55dea7bc1b7..e694a6adcee1 100644 --- a/notmuch-new.c +++ b/notmuch-new.c @@ -53,6 +53,7 @@ typedef struct { int total_files; int processed_files; int added_messages, removed_messages, renamed_messages; + int vanished_files; struct timeval tv_start; _filename_list_t *removed_files; @@ -280,11 +281,13 @@ add_file (notmuch_database_t *notmuch, const char *filename, case NOTMUCH_STATUS_FILE_NOT_EMAIL: fprintf (stderr, "Note: Ignoring non-mail file: %s\n", filename); break; - /* Fatal issues. Don't process anymore. */ case NOTMUCH_STATUS_FILE_ERROR: + /* Someone renamed/removed the file between scandir and now. */ + state->vanished_files++; fprintf (stderr, "Unexpected error with file %s\n", filename); (void) print_status_database ("add_file", notmuch, status); - goto DONE; + break; + /* Fatal issues. Don't process anymore. */ case NOTMUCH_STATUS_READ_ONLY_DATABASE: case NOTMUCH_STATUS_XAPIAN_EXCEPTION: case NOTMUCH_STATUS_OUT_OF_MEMORY: @@ -1151,5 +1154,11 @@ notmuch_new_command (notmuch_config_t *config, int argc, char *argv[]) if (!no_hooks && !ret && !interrupted) ret = notmuch_run_hook (db_path, "post-new"); - return ret || interrupted ? EXIT_FAILURE : EXIT_SUCCESS; + if (ret || interrupted) + return EXIT_FAILURE; + + if (add_files_state.vanished_files) + return NOTMUCH_EXIT_VANISHED_FILES; + + return EXIT_SUCCESS; } -- 2.1.4