[Kevin McCarthy] Bug#966100: notmuch-mutt: symlinking now fails for indexed mailboxes with a space in the name

[Kevin McCarthy] Bug#966100: notmuch-mutt: symlinking now fails for indexed mailboxes with a space in the name

[David Bremner]

[-- Attachment #1: Type: message/rfc822, Size: 10219 bytes --]

From: Kevin McCarthy <kevin@8t8.us>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: Bug#966100: notmuch-mutt: symlinking now fails for indexed mailboxes with a space in the name
Date: Wed, 22 Jul 2020 19:06:27 -0700
Message-ID: <159546998738.232977.10166967333633103899.reportbug@afu.lan>

Package: notmuch-mutt
Version: 0.30-1
Severity: important
Tags: upstream

Dear Maintainer,

After recently updating from stable to testing, notmuch-mutt search
and thread commands stopped including results from my "Sent Items"
maildir folder.  There are broken links in the results folder, and
symlink error messages in the terminal window after exiting mutt.

I tracked this down to upstream commit
<https://git.notmuchmail.org/git?p=notmuch;a=commitdiff;h=1fcf068e331b9b79e14f79c8b126711fc3d72cbb>
but unfortunately my shell-fu is not good enough to understand exactly
why the new version isn't working.

If I manually swap revert the diff then my Sent Items results start
working again.  So somehow the backslash-escaping works for xargs but
not for the shell while/read loop.

Here's the part of the commit above causing the breakage.

- - - - - - - - -
diff --git a/contrib/notmuch-mutt/notmuch-mutt b/contrib/notmuch-mutt/notmuch-mutt
index 0e46a8c1b95e76163eed68694aa5a1a973c8b371..d33223bdd88e776acf79ae8b4ba2703ff1e6cc58 100755 (executable)
--- a/contrib/notmuch-mutt/notmuch-mutt
+++ b/contrib/notmuch-mutt/notmuch-mutt
@@ -50,7 +50,7 @@ sub search($$$) {
     empty_maildir($maildir);
     system("notmuch search --output=files $dup_option $query"
           . " | sed -e 's: :\\\\ :g'"
-          . " | xargs -r -I searchoutput ln -s searchoutput $maildir/cur/");
+          . " | while IFS= read -r searchoutput; do ln -s \$searchoutput $maildir/cur/; done");
 }
 
 sub prompt($$) {
- - - - - - - - -


-- System Information:
Debian Release: bullseye/sid
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 5.7.0-1-amd64 (SMP w/16 CPU threads)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages notmuch-mutt depends on:
ii  libmail-box-perl           3.008-1
ii  libmailtools-perl          2.21-1
ii  libstring-shellquote-perl  1.04-1
ii  libterm-readline-gnu-perl  1.36-2+b1
ii  notmuch                    0.30-1
ii  perl                       5.30.3-4

Versions of packages notmuch-mutt recommends:
pn  mutt  <none>

notmuch-mutt suggests no packages.

-- no debconf information

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [Kevin McCarthy] Bug#966100: notmuch-mutt: symlinking now fails for indexed mailboxes with a space in the name

[Stefano Zacchiroli]

Heya, thanks for the heads up. I don't think I was involved in that
specific commit though, so I'm shamelessly punting to the author (added
to Cc:) :-)

(FWIW, I don't see either at first sight why it would break anything...)

Cheers

On Thu, Jul 23, 2020 at 06:50:29AM -0300, David Bremner wrote:
> Date: Wed, 22 Jul 2020 19:06:27 -0700
> From: Kevin McCarthy <kevin@8t8.us>
> To: Debian Bug Tracking System <submit@bugs.debian.org>
> Subject: Bug#966100: notmuch-mutt: symlinking now fails for indexed
>  mailboxes with a space in the name
> Reply-To: Kevin McCarthy <kevin@8t8.us>, 966100@bugs.debian.org
> Resent-From: Kevin McCarthy <kevin@8t8.us>
> Resent-To: debian-bugs-dist@lists.debian.org
> Resent-Date: Thu, 23 Jul 2020 02:21:01 +0000
> Message-ID: <159546998738.232977.10166967333633103899.reportbug@afu.lan>
> 
> Package: notmuch-mutt
> Version: 0.30-1
> Severity: important
> Tags: upstream
> 
> Dear Maintainer,
> 
> After recently updating from stable to testing, notmuch-mutt search
> and thread commands stopped including results from my "Sent Items"
> maildir folder.  There are broken links in the results folder, and
> symlink error messages in the terminal window after exiting mutt.
> 
> I tracked this down to upstream commit
> <https://git.notmuchmail.org/git?p=notmuch;a=commitdiff;h=1fcf068e331b9b79e14f79c8b126711fc3d72cbb>
> but unfortunately my shell-fu is not good enough to understand exactly
> why the new version isn't working.
> 
> If I manually swap revert the diff then my Sent Items results start
> working again.  So somehow the backslash-escaping works for xargs but
> not for the shell while/read loop.
> 
> Here's the part of the commit above causing the breakage.
> 
> - - - - - - - - -
> diff --git a/contrib/notmuch-mutt/notmuch-mutt b/contrib/notmuch-mutt/notmuch-mutt
> index 0e46a8c1b95e76163eed68694aa5a1a973c8b371..d33223bdd88e776acf79ae8b4ba2703ff1e6cc58 100755 (executable)
> --- a/contrib/notmuch-mutt/notmuch-mutt
> +++ b/contrib/notmuch-mutt/notmuch-mutt
> @@ -50,7 +50,7 @@ sub search($$$) {
>      empty_maildir($maildir);
>      system("notmuch search --output=files $dup_option $query"
>            . " | sed -e 's: :\\\\ :g'"
> -          . " | xargs -r -I searchoutput ln -s searchoutput $maildir/cur/");
> +          . " | while IFS= read -r searchoutput; do ln -s \$searchoutput $maildir/cur/; done");
>  }
>  
>  sub prompt($$) {
> - - - - - - - - -
> 
> 
> -- System Information:
> Debian Release: bullseye/sid
>   APT prefers testing
>   APT policy: (500, 'testing')
> Architecture: amd64 (x86_64)
> 
> Kernel: Linux 5.7.0-1-amd64 (SMP w/16 CPU threads)
> Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
> Shell: /bin/sh linked to /bin/dash
> Init: systemd (via /run/systemd/system)
> LSM: AppArmor: enabled
> 
> Versions of packages notmuch-mutt depends on:
> ii  libmail-box-perl           3.008-1
> ii  libmailtools-perl          2.21-1
> ii  libstring-shellquote-perl  1.04-1
> ii  libterm-readline-gnu-perl  1.36-2+b1
> ii  notmuch                    0.30-1
> ii  perl                       5.30.3-4
> 
> Versions of packages notmuch-mutt recommends:
> pn  mutt  <none>
> 
> notmuch-mutt suggests no packages.
> 
> -- no debconf information
> 

-- 
Stefano Zacchiroli . zack@upsilon.cc . upsilon.cc/zack . . o . . . o . o
Computer Science Professor . CTO Software Heritage . . . . . o . . . o o
Former Debian Project Leader & OSI Board Director  . . . o o o . . . o .
« the first rule of tautology club is the first rule of tautology club »

Re: [Kevin McCarthy] Bug#966100: notmuch-mutt: symlinking now fails for indexed mailboxes with a space in the name

[Greg Anders]

 From 92221d856e663450c1f55b9ec9a6aa69b4d27b3d Mon Sep 17 00:00:00 2001
From: Greg Anders <pub@gpanders.com>
Date: Sun, 26 Jul 2020 14:57:56 -0600
Subject: [PATCH] notmuch-mutt: quote variables to prevent word splitting

This fixes a bug where mail under directories with spaces in their names
would not be found.
---
  contrib/notmuch-mutt/notmuch-mutt | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/contrib/notmuch-mutt/notmuch-mutt b/contrib/notmuch-mutt/notmuch-mutt
index d33223bd..8ab1849f 100755
--- a/contrib/notmuch-mutt/notmuch-mutt
+++ b/contrib/notmuch-mutt/notmuch-mutt
@@ -50,7 +50,7 @@ sub search($$$) {
      empty_maildir($maildir);
      system("notmuch search --output=files $dup_option $query"
  	   . " | sed -e 's: :\\\\ :g'"
-	   . " | while IFS= read -r searchoutput; do ln -s \$searchoutput $maildir/cur/; done");
+	   . " | while IFS= read -r searchoutput; do ln -s \"\$searchoutput\" $maildir/cur/; done");
  }
  
  sub prompt($$) {
-- 
2.27.0

Re: [Kevin McCarthy] Bug#966100: notmuch-mutt: symlinking now fails for indexed mailboxes with a space in the name

[David Bremner]

Greg Anders <greg@gpanders.com> writes:

>  From 92221d856e663450c1f55b9ec9a6aa69b4d27b3d Mon Sep 17 00:00:00 2001
> From: Greg Anders <pub@gpanders.com>
> Date: Sun, 26 Jul 2020 14:57:56 -0600
> Subject: [PATCH] notmuch-mutt: quote variables to prevent word splitting
>
> This fixes a bug where mail under directories with spaces in their names
> would not be found.

I have the impression that Tomi's patch is a more thorough solution to
the same problem (although I didn't test it).

  https://nmbug.notmuchmail.org/nmweb/search/id%3A20200727193833.4654-1-tomi.ollila%40iki.fi

I would appreciate it if one of you could confirm or deny

Re: [Kevin McCarthy] Bug#966100: notmuch-mutt: symlinking now fails for indexed mailboxes with a space in the name

[Greg Anders]

On Sat Aug 1, 2020 at 2:22 PM MDT, David Bremner wrote:
> Greg Anders <greg@gpanders.com> writes:
>
> >  From 92221d856e663450c1f55b9ec9a6aa69b4d27b3d Mon Sep 17 00:00:00 2001
> > From: Greg Anders <pub@gpanders.com>
> > Date: Sun, 26 Jul 2020 14:57:56 -0600
> > Subject: [PATCH] notmuch-mutt: quote variables to prevent word splitting
> >
> > This fixes a bug where mail under directories with spaces in their names
> > would not be found.
>
> I have the impression that Tomi's patch is a more thorough solution to
> the same problem (although I didn't test it).
>

I completely agree. I think rewriting the process in Perl instead of 
shelling out is the better way to go.

Re: [Kevin McCarthy] Bug#966100: notmuch-mutt: symlinking now fails for indexed mailboxes with a space in the name

[Tomi Ollila]

On Thu, Jul 23 2020, David Bremner wrote:

> From: Kevin McCarthy <kevin@8t8.us>
> Subject: Bug#966100: notmuch-mutt: symlinking now fails for indexed mailboxes with a space in the name
> To: Debian Bug Tracking System <submit@bugs.debian.org>
> Date: Wed, 22 Jul 2020 19:06:27 -0700
>
> Package: notmuch-mutt
> Version: 0.30-1
> Severity: important
> Tags: upstream
>
> Dear Maintainer,

Hi Kevin, others.

>
> After recently updating from stable to testing, notmuch-mutt search
> and thread commands stopped including results from my "Sent Items"
> maildir folder.  There are broken links in the results folder, and
> symlink error messages in the terminal window after exiting mutt.
>
> I tracked this down to upstream commit
> <https://git.notmuchmail.org/git?p=notmuch;a=commitdiff;h=1fcf068e331b9b79e14f79c8b126711fc3d72cbb>
> but unfortunately my shell-fu is not good enough to understand exactly
> why the new version isn't working.
>
> If I manually swap revert the diff then my Sent Items results start
> working again.  So somehow the backslash-escaping works for xargs but
> not for the shell while/read loop.
>
> Here's the part of the commit above causing the breakage.
>
> - - - - - - - - -
> diff --git a/contrib/notmuch-mutt/notmuch-mutt b/contrib/notmuch-mutt/notmuch-mutt
> index 0e46a8c1b95e76163eed68694aa5a1a973c8b371..d33223bdd88e776acf79ae8b4ba2703ff1e6cc58 100755 (executable)
> --- a/contrib/notmuch-mutt/notmuch-mutt
> +++ b/contrib/notmuch-mutt/notmuch-mutt
> @@ -50,7 +50,7 @@ sub search($$$) {
>      empty_maildir($maildir);
>      system("notmuch search --output=files $dup_option $query"
>            . " | sed -e 's: :\\\\ :g'"
> -          . " | xargs -r -I searchoutput ln -s searchoutput $maildir/cur/");
> +          . " | while IFS= read -r searchoutput; do ln -s \$searchoutput $maildir/cur/; done");
>  }

This could work, cannot test (compiles OK w/ perl -c when I outcommented
packages I don't have.

--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--

diff --git a/contrib/notmuch-mutt/notmuch-mutt b/contrib/notmuch-mutt/notmuch-mutt
index d33223bd..b6c2d7b5 100755
--- a/contrib/notmuch-mutt/notmuch-mutt
+++ b/contrib/notmuch-mutt/notmuch-mutt
@@ -12,6 +12,7 @@ use strict;
 use warnings;
 
 use File::Path;
+use File::Basename;
 use Getopt::Long qw(:config no_getopt_compat);
 use Mail::Header;
 use Mail::Box::Maildir;
@@ -41,16 +42,19 @@ sub search($$$) {
     my ($maildir, $remove_dups, $query) = @_;
     my $dup_option = "";
 
-    $query = shell_quote($query);
-
+    my @args = qw/notmuch search --output=files/;
     if ($remove_dups) {
-      $dup_option = "--duplicate=1";
+	push @args, "--duplicate=1";
     }
+    push @args, $query;
 
     empty_maildir($maildir);
-    system("notmuch search --output=files $dup_option $query"
-	   . " | sed -e 's: :\\\\ :g'"
-	   . " | while IFS= read -r searchoutput; do ln -s \$searchoutput $maildir/cur/; done");
+    open my $p, '-|', @args or die "Running @args failed: $!\n";
+    while (<$p>) {
+	chomp;
+	my $bn = basename $_;
+	symlink $_, "$maildir/cur/$bn";  # fixme: check (report?) error
+    }
 }
 
 sub prompt($$) {

--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--


>  
>  sub prompt($$) {
> - - - - - - - - -
>
>
> -- System Information:
> Debian Release: bullseye/sid
>   APT prefers testing
>   APT policy: (500, 'testing')
> Architecture: amd64 (x86_64)
>
> Kernel: Linux 5.7.0-1-amd64 (SMP w/16 CPU threads)
> Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
> Shell: /bin/sh linked to /bin/dash
> Init: systemd (via /run/systemd/system)
> LSM: AppArmor: enabled
>
> Versions of packages notmuch-mutt depends on:
> ii  libmail-box-perl           3.008-1
> ii  libmailtools-perl          2.21-1
> ii  libstring-shellquote-perl  1.04-1
> ii  libterm-readline-gnu-perl  1.36-2+b1
> ii  notmuch                    0.30-1
> ii  perl                       5.30.3-4
>
> Versions of packages notmuch-mutt recommends:
> pn  mutt  <none>
>
> notmuch-mutt suggests no packages.
>
> -- no debconf information

Re: [Kevin McCarthy] Bug#966100: notmuch-mutt: symlinking now fails for indexed mailboxes with a space in the name

[Kevin J. McCarthy]

[-- Attachment #1.1: Type: text/plain, Size: 324 bytes --]

On Thu, Jul 23, 2020 at 06:58:42PM +0300, Tomi Ollila wrote:
>This could work, cannot test (compiles OK w/ perl -c when I outcommented
>packages I don't have.

I'm not subscribed to the mailing list, so my reply there will probably 
bounce.

A quick test of your patch seemed to work for me.

Thank you!

-Kevin

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]