[PATCH 1/2] test: add known broken test for diagnostics from over long filenames.

[PATCH 1/2] test: add known broken test for diagnostics from over long filenames.

[David Bremner]

Previously we tested over long directory names, add similar testing
for over long filenames.
---
 test/T050-new.sh | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/test/T050-new.sh b/test/T050-new.sh
index cb67889c..74890928 100755
--- a/test/T050-new.sh
+++ b/test/T050-new.sh
@@ -383,6 +383,21 @@ No new mail. Removed 1 message.
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest "Long file names have reasonable diagnostics"
+test_subtest_known_broken
+printf -v name 'f%.0s' {1..234}
+generate_message "[filename]=$name"
+notmuch new 2>&1 | notmuch_dir_sanitize >OUTPUT
+rm ${MAIL_DIR}/${name}
+cat <<EOF > EXPECTED
+Unexpected error with file MAIL_DIR/$name
+add_file: Something went wrong trying to read or write a file
+filename too long for file-direntry term: MAIL_DIR/$name
+Processed 1 file in almost no time.
+No new mail.
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
 test_begin_subtest "Xapian exception: read only files"
 chmod u-w ${MAIL_DIR}/.notmuch/xapian/*.*
 output=$(NOTMUCH_NEW --debug 2>&1 | sed 's/: .*$//' )
-- 
2.35.2

[PATCH 2/2] lib: add better diagnostics for over long filenames.

[David Bremner]

Previously we just crashed with an internal error. With this change,
the caller can handle it better, although the error code could be
further improved.
---
 lib/message.cc   | 15 +++++++++++----
 test/T050-new.sh |  1 -
 2 files changed, 11 insertions(+), 5 deletions(-)

diff --git a/lib/message.cc b/lib/message.cc
index 1c87f8c0..0e6ea809 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -941,6 +941,7 @@ _notmuch_message_add_filename (notmuch_message_t *message,
 {
     const char *relative, *directory;
     notmuch_status_t status;
+    notmuch_private_status_t private_status;
     void *local = talloc_new (message);
     char *direntry;
 
@@ -964,10 +965,16 @@ _notmuch_message_add_filename (notmuch_message_t *message,
 
     /* New file-direntry allows navigating to this message with
      * notmuch_directory_get_child_files() . */
-    status = COERCE_STATUS (_notmuch_message_add_term (message, "file-direntry", direntry),
-			    "adding file-direntry term");
-    if (status)
-	return status;
+    private_status =_notmuch_message_add_term (message, "file-direntry", direntry);
+    switch (private_status) {
+    case NOTMUCH_PRIVATE_STATUS_SUCCESS:
+	break;
+    case NOTMUCH_PRIVATE_STATUS_TERM_TOO_LONG:
+	_notmuch_database_log(message->notmuch, "filename too long for file-direntry term: %s\n", filename);
+	return NOTMUCH_STATUS_FILE_ERROR;
+    default:
+	return COERCE_STATUS (private_status, "adding file-direntry term");
+    }
 
     status = _notmuch_message_add_folder_terms (message, directory);
     if (status)
diff --git a/test/T050-new.sh b/test/T050-new.sh
index 74890928..bf086609 100755
--- a/test/T050-new.sh
+++ b/test/T050-new.sh
@@ -384,7 +384,6 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "Long file names have reasonable diagnostics"
-test_subtest_known_broken
 printf -v name 'f%.0s' {1..234}
 generate_message "[filename]=$name"
 notmuch new 2>&1 | notmuch_dir_sanitize >OUTPUT
-- 
2.35.2

Re: [PATCH 2/2] lib: add better diagnostics for over long filenames.

[Jose A Ortega Ruiz]

On Sun, Oct 02 2022, David Bremner wrote:

> Previously we just crashed with an internal error. With this change,
> the caller can handle it better, although the error code could be
> further improved.

This works better for me, yes.  A possible improvement is that, with
this patch, one's only warned once about offending files, which are in
subsequent calls to notmuch new simply ignored.  I think it'd be better
if we could have a behaviour similar to that for "non-email" files in
the maildir, for which you get a warning every time you run new.

At any rate, many thanks David for the quick response!

Cheers,
jao
-- 
Contentment is, after all, simply refined indolence.
  -Thomas Chandler Haliburton, author, judge, and politician (1796-1865)

v2 of long filename diagnostics

[David Bremner]

This one use a more reasonable error code. As far as showing up every
time, it does if I use --full-scan. That is the same (for me) as
non-email files.

[PATCH v2 1/2] test: add known broken test for diagnostics from over long filenames.

[David Bremner]

Previously we tested over long directory names, add similar testing
for over long filenames.
---
 test/T050-new.sh | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/test/T050-new.sh b/test/T050-new.sh
index cb67889c..f0a56c96 100755
--- a/test/T050-new.sh
+++ b/test/T050-new.sh
@@ -383,6 +383,21 @@ No new mail. Removed 1 message.
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest "Long file names have reasonable diagnostics"
+test_subtest_known_broken
+printf -v name 'f%.0s' {1..234}
+generate_message "[filename]=$name"
+notmuch new 2>&1 | notmuch_dir_sanitize >OUTPUT
+rm ${MAIL_DIR}/${name}
+cat <<EOF > EXPECTED
+Note: Ignoring non-indexable path: MAIL_DIR/$name
+add_file: Path supplied is illegal for this function
+filename too long for file-direntry term: MAIL_DIR/$name
+Processed 1 file in almost no time.
+No new mail.
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
 test_begin_subtest "Xapian exception: read only files"
 chmod u-w ${MAIL_DIR}/.notmuch/xapian/*.*
 output=$(NOTMUCH_NEW --debug 2>&1 | sed 's/: .*$//' )
-- 
2.35.2

[PATCH v2 2/2] lib: add better diagnostics for over long filenames.

[David Bremner]

Previously we just crashed with an internal error. With this change,
the caller can handle it better. Update notmuch-new so that it doesn't
crash with "unknown error code" because of this change.
---
 lib/message.cc   | 16 ++++++++++++----
 lib/notmuch.h    |  2 ++
 notmuch-new.c    |  4 ++++
 test/T050-new.sh |  1 -
 4 files changed, 18 insertions(+), 5 deletions(-)

diff --git a/lib/message.cc b/lib/message.cc
index 5ccca95a..1b1a071a 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -941,6 +941,7 @@ _notmuch_message_add_filename (notmuch_message_t *message,
 {
     const char *relative, *directory;
     notmuch_status_t status;
+    notmuch_private_status_t private_status;
     void *local = talloc_new (message);
     char *direntry;
 
@@ -964,10 +965,17 @@ _notmuch_message_add_filename (notmuch_message_t *message,
 
     /* New file-direntry allows navigating to this message with
      * notmuch_directory_get_child_files() . */
-    status = COERCE_STATUS (_notmuch_message_add_term (message, "file-direntry", direntry),
-			    "adding file-direntry term");
-    if (status)
-	return status;
+    private_status = _notmuch_message_add_term (message, "file-direntry", direntry);
+    switch (private_status) {
+    case NOTMUCH_PRIVATE_STATUS_SUCCESS:
+	break;
+    case NOTMUCH_PRIVATE_STATUS_TERM_TOO_LONG:
+	_notmuch_database_log (message->notmuch, "filename too long for file-direntry term: %s\n",
+			       filename);
+	return NOTMUCH_STATUS_PATH_ERROR;
+    default:
+	return COERCE_STATUS (private_status, "adding file-direntry term");
+    }
 
     status = _notmuch_message_add_folder_terms (message, directory);
     if (status)
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 0b0540b1..ce375c04 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -293,6 +293,8 @@ typedef struct _notmuch_indexopts notmuch_indexopts_t;
  *
  * NOTMUCH_STATUS_OUT_OF_MEMORY: Out of memory.
  *
+ * NOTMUCH_STATUS_PATH_ERROR: filename is too long
+ *
  * NOTMUCH_STATUS_FILE_ERROR: An error occurred trying to create the
  *	database file (such as permission denied, or file not found,
  *	etc.), or the database already exists.
diff --git a/notmuch-new.c b/notmuch-new.c
index 346e6469..4a53e3eb 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -413,6 +413,10 @@ add_file (notmuch_database_t *notmuch, const char *filename,
     case NOTMUCH_STATUS_FILE_NOT_EMAIL:
 	fprintf (stderr, "Note: Ignoring non-mail file: %s\n", filename);
 	break;
+    case NOTMUCH_STATUS_PATH_ERROR:
+	fprintf (stderr, "Note: Ignoring non-indexable path: %s\n", filename);
+	(void) print_status_database ("add_file", notmuch, status);
+	break;
     case NOTMUCH_STATUS_FILE_ERROR:
 	/* Someone renamed/removed the file between scandir and now. */
 	state->vanished_files++;
diff --git a/test/T050-new.sh b/test/T050-new.sh
index f0a56c96..3167d338 100755
--- a/test/T050-new.sh
+++ b/test/T050-new.sh
@@ -384,7 +384,6 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "Long file names have reasonable diagnostics"
-test_subtest_known_broken
 printf -v name 'f%.0s' {1..234}
 generate_message "[filename]=$name"
 notmuch new 2>&1 | notmuch_dir_sanitize >OUTPUT
-- 
2.35.2

Re: v2 of long filename diagnostics

[David Bremner]

David Bremner <david@tethera.net> writes:

> This one use a more reasonable error code. As far as showing up every
> time, it does if I use --full-scan. That is the same (for me) as
> non-email files.

series applied to master.

d