bug#31737: 26.1.50; Gnus does not use mailutils movemail

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Robert Pluim]

This is based on message <87po162oze.fsf@nwalsh.com> to ding@gnus.org,
where someone noted that emacs-26 no longer installs movemail,
breaking his Gnus setup [1].

In `mail-source-movemail' in lisp/gnus/mail-source.el, Gnus only looks
in exec-directory for movemail, even though configure looked in PATH
to find mailutils movemail and decide not to install Emacs'
movemail. (rmail doesnʼt have this issue, it does runtime detection)

Gnus should probably look in at least exec-path for movemail (after
exec-directory). One way to do this would be to change the default
value of `mail-source-movemail-program to be' (executable-find
"movemail"), but that has the risk of silently switching emacs to
using mailutils' movemail instead of the emacs one.

Also, mailutils' movemail has a '--emacs' flag which rmail uses. Itʼs not
immediately obvious to me if gnus should do the same or not.

Robert

Footnotes: 
[1]  I guess people no longer use movemail that much, otherwise this
would have been noticed sooner

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Colin Baxter]

>>>>> "Robert" == Robert Pluim <rpluim@gmail.com> writes:

    Robert> This is based on message <87po162oze.fsf@nwalsh.com> to
    Robert> ding@gnus.org, where someone noted that emacs-26 no longer
    Robert> installs movemail, breaking his Gnus setup [1].

    Robert> In `mail-source-movemail' in lisp/gnus/mail-source.el, Gnus
    Robert> only looks in exec-directory for movemail, even though
    Robert> configure looked in PATH to find mailutils movemail and
    Robert> decide not to install Emacs' movemail. (rmail doesnʼt have
    Robert> this issue, it does runtime detection)

    Robert> Gnus should probably look in at least exec-path for movemail
    Robert> (after exec-directory). One way to do this would be to
    Robert> change the default value of `mail-source-movemail-program to
    Robert> be' (executable-find "movemail"), but that has the risk of
    Robert> silently switching emacs to using mailutils' movemail
    Robert> instead of the emacs one.

    Robert> Also, mailutils' movemail has a '--emacs' flag which rmail
    Robert> uses. Itʼs not immediately obvious to me if gnus should do
    Robert> the same or not.

I don't know whether my input will be relevant, but here goes.

I use gnus and have had no trouble with movemail. In emacs-26.1 I
compiled using ./configure --with-mailutils and in my ~/.emacs I point
the mail-source-movemail-program to "/usr/bin/movemail". Everything
works fine.

Best wishes

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Robert Pluim]

Colin Baxter <m43cap@yandex.com> writes:

>>>>>> "Robert" == Robert Pluim <rpluim@gmail.com> writes:
>
>     Robert> Gnus should probably look in at least exec-path for movemail
>     Robert> (after exec-directory). One way to do this would be to
>     Robert> change the default value of `mail-source-movemail-program to
>     Robert> be' (executable-find "movemail"), but that has the risk of
>     Robert> silently switching emacs to using mailutils' movemail
>     Robert> instead of the emacs one.
>
>     Robert> Also, mailutils' movemail has a '--emacs' flag which rmail
>     Robert> uses. Itʼs not immediately obvious to me if gnus should do
>     Robert> the same or not.
>
> I don't know whether my input will be relevant, but here goes.
>

I think itʼs very relevant.

> I use gnus and have had no trouble with movemail. In emacs-26.1 I
> compiled using ./configure --with-mailutils and in my ~/.emacs I point
> the mail-source-movemail-program to "/usr/bin/movemail". Everything
> works fine.

So it looks like the --emacs flag is not necessary, which means we
could just change the default of mail-source-movemail-progam. [1]

Robert

Footnotes: 
[1]  Eli, 26.2? This is an emacs-26 regression

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Date: Wed, 06 Jun 2018 21:55:20 +0200
> Cc: 31737@debbugs.gnu.org
> 
> > I use gnus and have had no trouble with movemail. In emacs-26.1 I
> > compiled using ./configure --with-mailutils and in my ~/.emacs I point
> > the mail-source-movemail-program to "/usr/bin/movemail". Everything
> > works fine.
> 
> So it looks like the --emacs flag is not necessary, which means we
> could just change the default of mail-source-movemail-progam. [1]

Sorry, I don't think I follow: mail-source-movemail-progam is a
variable in Gnus, and its default is nil.

And I don't see how you concluded that the --emacs flag is
unnecessary: the Mailutils manual says it causes movemail to "output
information used by Emacs rmail interface", i.e. this is unrelated to
Gnus.

> Footnotes: 
> [1]  Eli, 26.2? This is an emacs-26 regression

What exactly was the regression, and which change do you propose for
the branch?

Thanks.

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Robert Pluim]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Robert Pluim <rpluim@gmail.com>
>> Date: Wed, 06 Jun 2018 21:55:20 +0200
>> Cc: 31737@debbugs.gnu.org
>> 
>> > I use gnus and have had no trouble with movemail. In emacs-26.1 I
>> > compiled using ./configure --with-mailutils and in my ~/.emacs I point
>> > the mail-source-movemail-program to "/usr/bin/movemail". Everything
>> > works fine.
>> 
>> So it looks like the --emacs flag is not necessary, which means we
>> could just change the default of mail-source-movemail-progam. [1]
>
> Sorry, I don't think I follow: mail-source-movemail-progam is a
> variable in Gnus, and its default is nil.

Yes, and in that case Gnus looks for movemail in 'exec-directory'
only, where emacs no longer installs it if it found Mailutils
movemail at configure time.

> And I don't see how you concluded that the --emacs flag is
> unnecessary: the Mailutils manual says it causes movemail to "output
> information used by Emacs rmail interface", i.e. this is unrelated to
> Gnus.

It wasn't a firm conclusion, it was a tentative one based on one
report. Thereʼs no real documentation on the '--emacs' flag that I
could find.

>> Footnotes: 
>> [1]  Eli, 26.2? This is an emacs-26 regression
>
> What exactly was the regression, and which change do you propose for
> the branch?

Gnus can no longer use movemail, even though at configure time it was
found.

The simplest fix is below, but it will cause people who currently have
Mailutils movemail installed, but an emacs built to not use it, to
start using it, so it makes me uncomfortable. Perhaps gnus should
check in 'exec-directory' first if mail-source-movemail-program is
nil, and then fall back to 'executable-find'? Or perhaps we should
just forget about 'our' movemail and require people to install
Mailutils.

diff --git i/lisp/gnus/mail-source.el w/lisp/gnus/mail-source.el
index d2850f4cee..1e328c9b43 100644
--- i/lisp/gnus/mail-source.el
+++ w/lisp/gnus/mail-source.el
@@ -301,7 +301,7 @@ mail-source-idle-time-delay
   :group 'mail-source
   :type 'number)
 
-(defcustom mail-source-movemail-program nil
+(defcustom mail-source-movemail-program (executable-find "movemail")
   "If non-nil, name of program for fetching new mail."
   :version "22.1"
   :group 'mail-source

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: m43cap@yandex.com,  31737@debbugs.gnu.org
> Date: Fri, 08 Jun 2018 18:01:14 +0200
> 
> > Sorry, I don't think I follow: mail-source-movemail-progam is a
> > variable in Gnus, and its default is nil.
> 
> Yes, and in that case Gnus looks for movemail in 'exec-directory'
> only, where emacs no longer installs it if it found Mailutils
> movemail at configure time.

But then the fix should be where mail-source-movemail-progam is used,
not in its default value, right?  Gnus should use executable-find to
look for the program named by mail-source-movemail-progam.

> >> [1]  Eli, 26.2? This is an emacs-26 regression
> >
> > What exactly was the regression, and which change do you propose for
> > the branch?
> 
> Gnus can no longer use movemail, even though at configure time it was
> found.

And why is that an emacs-26 regression?  the value of that variable
hasn't changed since 2004, AFAICS.

> The simplest fix is below, but it will cause people who currently have
> Mailutils movemail installed, but an emacs built to not use it, to
> start using it, so it makes me uncomfortable. Perhaps gnus should
> check in 'exec-directory' first if mail-source-movemail-program is
> nil, and then fall back to 'executable-find'?

I see no need for any special handling by Emacs here: the user could
customize the value to point to "our" movemail as an absolute file
name, in which case executable-find will pick it up.  We just need to
change Gnus to use executable-find where the variable is used.

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Robert Pluim]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Robert Pluim <rpluim@gmail.com>
>> Cc: m43cap@yandex.com,  31737@debbugs.gnu.org
>> Date: Fri, 08 Jun 2018 18:01:14 +0200
>> 
>> > Sorry, I don't think I follow: mail-source-movemail-progam is a
>> > variable in Gnus, and its default is nil.
>> 
>> Yes, and in that case Gnus looks for movemail in 'exec-directory'
>> only, where emacs no longer installs it if it found Mailutils
>> movemail at configure time.
>
> But then the fix should be where mail-source-movemail-progam is used,
> not in its default value, right?  Gnus should use executable-find to
> look for the program named by mail-source-movemail-progam.
>

It blindly calls call-process at the moment, rather than using
executable-find, but thatʼs a separate issue.

>> >> [1]  Eli, 26.2? This is an emacs-26 regression
>> >
>> > What exactly was the regression, and which change do you propose for
>> > the branch?
>> 
>> Gnus can no longer use movemail, even though at configure time it was
>> found.
>
> And why is that an emacs-26 regression?  the value of that variable
> hasn't changed since 2004, AFAICS.

Pre-emacs 26 gnus used the 'builtin' movemail. That movemail is no
longer installed, because configure found the Mailutils one. But gnus
doesnʼt use the Mailutils one. So the end result from the user's
perspective is "emacs disabled its own movemail because I have
Mailutils installed, but then doesnʼt use the Mailutils movemail"

We can call it a regression or not, but itʼs an unexpected outcome.

>> The simplest fix is below, but it will cause people who currently have
>> Mailutils movemail installed, but an emacs built to not use it, to
>> start using it, so it makes me uncomfortable. Perhaps gnus should
>> check in 'exec-directory' first if mail-source-movemail-program is
>> nil, and then fall back to 'executable-find'?
>
> I see no need for any special handling by Emacs here: the user could
> customize the value to point to "our" movemail as an absolute file
> name, in which case executable-find will pick it up.  We just need to
> change Gnus to use executable-find where the variable is used.

In emacs-26 there is no "our" movemail to point at when Mailutils were
detected, and asking the user to customize a variable to get back to
where they were before feels wrong as well.

Robert

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: m43cap@yandex.com,  31737@debbugs.gnu.org
> Date: Fri, 08 Jun 2018 20:47:53 +0200
> 
> > But then the fix should be where mail-source-movemail-progam is used,
> > not in its default value, right?  Gnus should use executable-find to
> > look for the program named by mail-source-movemail-progam.
> >
> 
> It blindly calls call-process at the moment, rather than using
> executable-find, but thatʼs a separate issue.

But call-process looks up the program along exec-path, and exec-path
puts exec-directory at its end.  Right?  So if GNU Mailutils are
installed and on PATH, Gnus should find its movemail first, right?  Or
am I missing something?

IOW, if Gnus calls call-process, then it already uses executable-find,
albeit internally.

> >> The simplest fix is below, but it will cause people who currently have
> >> Mailutils movemail installed, but an emacs built to not use it, to
> >> start using it, so it makes me uncomfortable. Perhaps gnus should
> >> check in 'exec-directory' first if mail-source-movemail-program is
> >> nil, and then fall back to 'executable-find'?
> >
> > I see no need for any special handling by Emacs here: the user could
> > customize the value to point to "our" movemail as an absolute file
> > name, in which case executable-find will pick it up.  We just need to
> > change Gnus to use executable-find where the variable is used.
> 
> In emacs-26 there is no "our" movemail to point at when Mailutils were
> detected, and asking the user to customize a variable to get back to
> where they were before feels wrong as well.

We are mis-communicating, I think.  The situation you described in
which you feel uncomfortable was that both Mailutils and "our"
movemail are installed.  If that's not the situation, then what makes
you uncomfortable about a case where only one movemail is available
and Gnus uses it?

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Robert Pluim]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Robert Pluim <rpluim@gmail.com>
>> Cc: m43cap@yandex.com,  31737@debbugs.gnu.org
>> Date: Fri, 08 Jun 2018 20:47:53 +0200
>> 
>> > But then the fix should be where mail-source-movemail-progam is used,
>> > not in its default value, right?  Gnus should use executable-find to
>> > look for the program named by mail-source-movemail-progam.
>> >
>> 
>> It blindly calls call-process at the moment, rather than using
>> executable-find, but thatʼs a separate issue.
>
> But call-process looks up the program along exec-path, and exec-path
> puts exec-directory at its end.  Right?  So if GNU Mailutils are
> installed and on PATH, Gnus should find its movemail first, right?  Or
> am I missing something?
>
> IOW, if Gnus calls call-process, then it already uses executable-find,
> albeit internally.

As noted on emacs-devel, gnus passes (expand-file-name "movemail"
exec-directory) to call-process, so it doesnʼt search in $PATH.

I think the current suggested fix is this:

 		       (append
 			(list
 			 (or mail-source-movemail-program
-			     (expand-file-name "movemail" exec-directory))
+			     "movemail"
 			 nil errors nil from to)))))
 	      (when (file-exists-p to)
 		(set-file-modes to mail-source-default-file-modes))

That will work for people who have Mailutils installed in $PATH, and
for those who donʼt it should use the exec-directory one.

The only issue is that people who install Mailutils after they've
built emacs will now be switched to using the Mailutils movemail, with
no indication that itʼs happened.

Robert

PS The possible requirement for passing '--emacs' to movemail is still
not clear to me. Rmail cares whether itʼs using Mailutils, but I donʼt
know if Gnus does.

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: m43cap@yandex.com,  31737@debbugs.gnu.org
> Date: Mon, 11 Jun 2018 10:27:13 +0200
> 
> I think the current suggested fix is this:
> 
>  		       (append
>  			(list
>  			 (or mail-source-movemail-program
> -			     (expand-file-name "movemail" exec-directory))
> +			     "movemail"
>  			 nil errors nil from to)))))
>  	      (when (file-exists-p to)
>  		(set-file-modes to mail-source-default-file-modes))
> 
> That will work for people who have Mailutils installed in $PATH, and
> for those who donʼt it should use the exec-directory one.
> 
> The only issue is that people who install Mailutils after they've
> built emacs will now be switched to using the Mailutils movemail, with
> no indication that itʼs happened.

Which is what we want, I think, given the message output by
'configure'.

So I think we should install the above.

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Robert Pluim]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Robert Pluim <rpluim@gmail.com>
>> Cc: m43cap@yandex.com,  31737@debbugs.gnu.org
>> Date: Mon, 11 Jun 2018 10:27:13 +0200
>> 
>> I think the current suggested fix is this:
>> 
>>  		       (append
>>  			(list
>>  			 (or mail-source-movemail-program
>> -			     (expand-file-name "movemail" exec-directory))
>> +			     "movemail"
>>  			 nil errors nil from to)))))
>>  	      (when (file-exists-p to)
>>  		(set-file-modes to mail-source-default-file-modes))
>> 
>> That will work for people who have Mailutils installed in $PATH, and
>> for those who donʼt it should use the exec-directory one.
>> 
>> The only issue is that people who install Mailutils after they've
>> built emacs will now be switched to using the Mailutils movemail, with
>> no indication that itʼs happened.
>
> Which is what we want, I think, given the message output by
> 'configure'.
>

OK

> So I think we should install the above.

Are we sure enough that '--emacs' is not needed?

Robert

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: m43cap@yandex.com,  31737@debbugs.gnu.org
> Date: Mon, 11 Jun 2018 23:47:12 +0200
> 
> Are we sure enough that '--emacs' is not needed?

I don't know.  I don't use Gnus, and I don't really know what that
option in Mailutils does.  If someone here can explain what it does,
we can make the decision; otherwise, we will leave Gnus to its
devices.

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Colin Baxter]

>>>>> "Eli" == Eli Zaretskii <eliz@gnu.org> writes:

    >> From: Robert Pluim <rpluim@gmail.com> Cc: m43cap@yandex.com,
    >> 31737@debbugs.gnu.org Date: Mon, 11 Jun 2018 23:47:12 +0200
    >> 
    >> Are we sure enough that '--emacs' is not needed?

    Eli> I don't know.  I don't use Gnus, and I don't really know what
    Eli> that option in Mailutils does.  If someone here can explain
    Eli> what it does, we can make the decision; otherwise, we will
    Eli> leave Gnus to its devices.

According to the movemail man page, the --emacs option is only used by
rmail not gnus. My email setup uses both rmail and gnus and I've never
used that option.

Best wishes,

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Eli Zaretskii]

> From: Colin Baxter <m43cap@yandex.com>
> Cc: Robert Pluim <rpluim@gmail.com>,  31737@debbugs.gnu.org
> Date: Tue, 12 Jun 2018 07:12:09 +0100
> 
> >>>>> "Eli" == Eli Zaretskii <eliz@gnu.org> writes:
> 
>     >> From: Robert Pluim <rpluim@gmail.com> Cc: m43cap@yandex.com,
>     >> 31737@debbugs.gnu.org Date: Mon, 11 Jun 2018 23:47:12 +0200
>     >> 
>     >> Are we sure enough that '--emacs' is not needed?
> 
>     Eli> I don't know.  I don't use Gnus, and I don't really know what
>     Eli> that option in Mailutils does.  If someone here can explain
>     Eli> what it does, we can make the decision; otherwise, we will
>     Eli> leave Gnus to its devices.
> 
> According to the movemail man page, the --emacs option is only used by
> rmail not gnus. My email setup uses both rmail and gnus and I've never
> used that option.

The man page isn't useful in this regard, it obviously assumes that no
user needs to understand what that option does.

So I "used the source", and it turns out that this option simply
countermands some configuration settings that users could set up which
will then get in the way of movemail's invocation by Emacs.

As to whether Gnus should also use that option -- I think it should,
because the code in mail-source.el which invokes movemail is an almost
exact copy of its sibling in rmail.el.  It's quite possible that Gnus
simply uses some old version of what's in Rmail.  (In particular,
rmail.el already does what I suggested to do in order to invoke
movemail from mailutils in preference to the version which comes with
Emacs.)

So I'm CC'ing Lars in the hope that he will decide what to do with
these two issues.

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> As to whether Gnus should also use that option -- I think it should,
> because the code in mail-source.el which invokes movemail is an almost
> exact copy of its sibling in rmail.el.  It's quite possible that Gnus
> simply uses some old version of what's in Rmail.  (In particular,
> rmail.el already does what I suggested to do in order to invoke
> movemail from mailutils in preference to the version which comes with
> Emacs.)
>
> So I'm CC'ing Lars in the hope that he will decide what to do with
> these two issues.

I don't think that mail-source needs that --emacs switch -- it never
has, I think?  But from what you say, it probably doesn't hurt...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Robert Pluim]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> As to whether Gnus should also use that option -- I think it should,
>> because the code in mail-source.el which invokes movemail is an almost
>> exact copy of its sibling in rmail.el.  It's quite possible that Gnus
>> simply uses some old version of what's in Rmail.  (In particular,
>> rmail.el already does what I suggested to do in order to invoke
>> movemail from mailutils in preference to the version which comes with
>> Emacs.)
>>
>> So I'm CC'ing Lars in the hope that he will decide what to do with
>> these two issues.
>
> I don't think that mail-source needs that --emacs switch -- it never
> has, I think? But from what you say, it probably doesn't hurt...

mail-source will only ever have used Mailutils movemail for those
people who had it installed *and* were aware that
mail-source-movemail-program needed specifying, so I suspect the
sample weʼre talking about is small.

Eli, were you thinking that mail-source should duplicate the movemail
detection logic in rmail? Or should it just assume Mailutils movemail
and pass '--emacs'?

Robert

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  Colin Baxter <m43cap@yandex.com>,  31737@debbugs.gnu.org
> Date: Wed, 13 Jun 2018 16:08:53 +0200
> 
> Eli, were you thinking that mail-source should duplicate the movemail
> detection logic in rmail? Or should it just assume Mailutils movemail
> and pass '--emacs'?

The former.

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Robert Pluim]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Robert Pluim <rpluim@gmail.com>
>> Cc: Eli Zaretskii <eliz@gnu.org>,  Colin Baxter <m43cap@yandex.com>,  31737@debbugs.gnu.org
>> Date: Wed, 13 Jun 2018 16:08:53 +0200
>> 
>> Eli, were you thinking that mail-source should duplicate the movemail
>> detection logic in rmail? Or should it just assume Mailutils movemail
>> and pass '--emacs'?
>
> The former.

Something like this? Testing definitely required by someone who uses
mailutils movemail for Gnus. Not sure this is emacs-26 material
anymore.

diff --git i/lisp/gnus/mail-source.el w/lisp/gnus/mail-source.el
index d2850f4cee..99dc40c843 100644
--- i/lisp/gnus/mail-source.el
+++ w/lisp/gnus/mail-source.el
@@ -301,7 +301,7 @@ mail-source-idle-time-delay
   :group 'mail-source
   :type 'number)
 
-(defcustom mail-source-movemail-program nil
+(defcustom mail-source-movemail-program "movemail"
   "If non-nil, name of program for fetching new mail."
   :version "22.1"
   :group 'mail-source
@@ -653,6 +653,12 @@ mail-source-delete-crash-box
 	     mail-source-delete-incoming
 	     mail-source-delete-old-incoming-confirm)))))))
 
+(defun mail-source-movemail-variant-p (variants)
+  "Return t if the current movemail variant is any of VARIANTS.
+See `rmail-movemail-variant-p' for details"
+  (let ((rmail-movemail-program mail-source-movemail-program))
+    (rmail-movemail-variant-p variant)))
+
 (defun mail-source-movemail (from to)
   "Move FROM to TO using movemail."
   (if (not (file-writable-p to))
@@ -682,13 +688,20 @@ mail-source-movemail
 	      (setq errors (generate-new-buffer " *mail source loss*"))
 	      (let ((default-directory "/"))
 		(setq result
+                      ;; call-proces looks in exec-path, which
+                      ;; contains exec-directory, so will find
+                      ;; Mailutils movemail if it exists, else it will
+                      ;; find "our" movemail in exec-directory.
+                      ;; Bug#31737
 		      (apply
 		       'call-process
 		       (append
 			(list
-			 (or mail-source-movemail-program
-			     (expand-file-name "movemail" exec-directory))
-			 nil errors nil from to)))))
+			 mail-source-movemail-program
+			 nil errors nil)
+                        (when (mail-source-movemail-variant-p 'mailutils)
+                          (list "--emacs"))
+                        (list from to)))))
 	      (when (file-exists-p to)
 		(set-file-modes to mail-source-default-file-modes))
 	      (if (and (or (not (buffer-modified-p errors))

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Robert Pluim]

Robert Pluim <rpluim@gmail.com> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Robert Pluim <rpluim@gmail.com>
>>> Cc: Eli Zaretskii <eliz@gnu.org>,  Colin Baxter <m43cap@yandex.com>,  31737@debbugs.gnu.org
>>> Date: Wed, 13 Jun 2018 16:08:53 +0200
>>> 
>>> Eli, were you thinking that mail-source should duplicate the movemail
>>> detection logic in rmail? Or should it just assume Mailutils movemail
>>> and pass '--emacs'?
>>
>> The former.
>
> Something like this? Testing definitely required by someone who uses
> mailutils movemail for Gnus. Not sure this is emacs-26 material
> anymore.

*sigh*. With the typo fixed now.

diff --git i/lisp/gnus/mail-source.el w/lisp/gnus/mail-source.el
index d2850f4cee..37bcb716f9 100644
--- i/lisp/gnus/mail-source.el
+++ w/lisp/gnus/mail-source.el
@@ -301,7 +301,7 @@ mail-source-idle-time-delay
   :group 'mail-source
   :type 'number)
 
-(defcustom mail-source-movemail-program nil
+(defcustom mail-source-movemail-program "movemail"
   "If non-nil, name of program for fetching new mail."
   :version "22.1"
   :group 'mail-source
@@ -653,6 +653,12 @@ mail-source-delete-crash-box
 	     mail-source-delete-incoming
 	     mail-source-delete-old-incoming-confirm)))))))
 
+(defun mail-source-movemail-variant-p (variant)
+  "Return t if the current movemail variant is VARIANT.
+See `rmail-movemail-variant-p' for details"
+  (let ((rmail-movemail-program mail-source-movemail-program))
+    (rmail-movemail-variant-p variant)))
+
 (defun mail-source-movemail (from to)
   "Move FROM to TO using movemail."
   (if (not (file-writable-p to))
@@ -682,13 +688,20 @@ mail-source-movemail
 	      (setq errors (generate-new-buffer " *mail source loss*"))
 	      (let ((default-directory "/"))
 		(setq result
+                      ;; call-proces looks in exec-path, which
+                      ;; contains exec-directory, so will find
+                      ;; Mailutils movemail if it exists, else it will
+                      ;; find "our" movemail in exec-directory.
+                      ;; Bug#31737
 		      (apply
 		       'call-process
 		       (append
 			(list
-			 (or mail-source-movemail-program
-			     (expand-file-name "movemail" exec-directory))
-			 nil errors nil from to)))))
+			 mail-source-movemail-program
+			 nil errors nil)
+                        (when (mail-source-movemail-variant-p 'mailutils)
+                          (list "--emacs"))
+                        (list from to)))))
 	      (when (file-exists-p to)
 		(set-file-modes to mail-source-default-file-modes))
 	      (if (and (or (not (buffer-modified-p errors))

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: m43cap@yandex.com,  larsi@gnus.org,  31737@debbugs.gnu.org
> Date: Thu, 14 Jun 2018 14:14:28 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Robert Pluim <rpluim@gmail.com>
> >> Cc: Eli Zaretskii <eliz@gnu.org>,  Colin Baxter <m43cap@yandex.com>,  31737@debbugs.gnu.org
> >> Date: Wed, 13 Jun 2018 16:08:53 +0200
> >> 
> >> Eli, were you thinking that mail-source should duplicate the movemail
> >> detection logic in rmail? Or should it just assume Mailutils movemail
> >> and pass '--emacs'?
> >
> > The former.
> 
> Something like this? Testing definitely required by someone who uses
> mailutils movemail for Gnus. Not sure this is emacs-26 material
> anymore.

We could install on emacs-26 something much simpler: just call
"movemail", without assuming it's in exec-directory, if
mail-source-movemail-program is nil, and don't use the "--emacs"
option.

Thanks.

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Robert Pluim]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Robert Pluim <rpluim@gmail.com>
>> Cc: m43cap@yandex.com,  larsi@gnus.org,  31737@debbugs.gnu.org
>> Date: Thu, 14 Jun 2018 14:14:28 +0200
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> From: Robert Pluim <rpluim@gmail.com>
>> >> Cc: Eli Zaretskii <eliz@gnu.org>,  Colin Baxter <m43cap@yandex.com>,  31737@debbugs.gnu.org
>> >> Date: Wed, 13 Jun 2018 16:08:53 +0200
>> >> 
>> >> Eli, were you thinking that mail-source should duplicate the movemail
>> >> detection logic in rmail? Or should it just assume Mailutils movemail
>> >> and pass '--emacs'?
>> >
>> > The former.
>> 
>> Something like this? Testing definitely required by someone who uses
>> mailutils movemail for Gnus. Not sure this is emacs-26 material
>> anymore.
>
> We could install on emacs-26 something much simpler: just call
> "movemail", without assuming it's in exec-directory, if
> mail-source-movemail-program is nil, and don't use the "--emacs"
> option.

Even simpler (Iʼm tempted to put ""Don't merge to master." in the ChangeLog).

2018-06-14  Robert Pluim  <rpluim@gmail.com>

	* lisp/gnus/mail-source.el (mail-source-movemail-program): Change
	default to "movemail".
	(mail-source-movemail): Pass just mail-source-movemail-program to
	call-process instead of fully specifying it relative to
	exec-directory.  Ensures that we will find Mailutils movemail if
	it is installed.  (Bug#31737)

diff --git i/etc/NEWS w/etc/NEWS
index 938644215f..0f67ca572a 100644
--- i/etc/NEWS
+++ w/etc/NEWS
@@ -46,6 +46,14 @@ often cause crashes.  Set it to nil if you really need those fonts.
 \f
 * Changes in Specialized Modes and Packages in Emacs 26.2
 
+** Gnus
+
+---
+*** Mailutils movemail will now be used if found at runtime.
+The default value of mail-source-movemail-program is now "movemail".
+This ensures that the movemail program from GNU Mailutils will be used
+if found in 'exec-path', even if it was not found at build time.
+
 ** Shell mode
 
 ---
diff --git i/lisp/gnus/mail-source.el w/lisp/gnus/mail-source.el
index d2850f4cee..22d3eb374f 100644
--- i/lisp/gnus/mail-source.el
+++ w/lisp/gnus/mail-source.el
@@ -301,9 +301,9 @@ mail-source-idle-time-delay
   :group 'mail-source
   :type 'number)
 
-(defcustom mail-source-movemail-program nil
+(defcustom mail-source-movemail-program "movemail"
   "If non-nil, name of program for fetching new mail."
-  :version "22.1"
+  :version "26.2"
   :group 'mail-source
   :type '(choice (const nil) string))
 
@@ -682,12 +682,16 @@ mail-source-movemail
 	      (setq errors (generate-new-buffer " *mail source loss*"))
 	      (let ((default-directory "/"))
 		(setq result
+                      ;; call-proces looks in exec-path, which
+                      ;; contains exec-directory, so will find
+                      ;; Mailutils movemail if it exists, else it will
+                      ;; find "our" movemail in exec-directory.
+                      ;; Bug#31737
 		      (apply
 		       'call-process
 		       (append
 			(list
-			 (or mail-source-movemail-program
-			     (expand-file-name "movemail" exec-directory))
+			 mail-source-movemail-program
 			 nil errors nil from to)))))
 	      (when (file-exists-p to)
 		(set-file-modes to mail-source-default-file-modes))

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: m43cap@yandex.com,  larsi@gnus.org,  31737@debbugs.gnu.org
> Date: Thu, 14 Jun 2018 16:37:59 +0200
> 
> > We could install on emacs-26 something much simpler: just call
> > "movemail", without assuming it's in exec-directory, if
> > mail-source-movemail-program is nil, and don't use the "--emacs"
> > option.
> 
> Even simpler

LGTM, thanks.

> (Iʼm tempted to put ""Don't merge to master." in the ChangeLog).

Yes, please.

> +** Gnus
> +
> +---
> +*** Mailutils movemail will now be used if found at runtime.
> +The default value of mail-source-movemail-program is now "movemail".
> +This ensures that the movemail program from GNU Mailutils will be used
> +if found in 'exec-path', even if it was not found at build time.

I'd add a sentence here saying that if the first "movemail" on PATH is
not what the user wants, she should customize
mail-source-movemail-program to point to the absolute file name of the
program she does want.

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Paul Eggert]

I just proposed a simple patch for the movemail $PATH problem here:

https://lists.gnu.org/r/emacs-devel/2018-06/msg00441.html

But now I see that Robert proposed a slightly-more-complicated patch here:

https://debbugs.gnu.org/31737#63

I think either will work, and that one of the two should be applied to 
the emacs-26 and master branches.

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Eli Zaretskii]

> Cc: Eli Zaretskii <eliz@gnu.org>, Robert Pluim <rpluim@gmail.com>,
>  m43cap@yandex.com, larsi@gnus.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Thu, 14 Jun 2018 15:54:52 -0700
> 
> I just proposed a simple patch for the movemail $PATH problem here:
> 
> https://lists.gnu.org/r/emacs-devel/2018-06/msg00441.html
> 
> But now I see that Robert proposed a slightly-more-complicated patch here:
> 
> https://debbugs.gnu.org/31737#63
> 
> I think either will work, and that one of the two should be applied to 
> the emacs-26 and master branches.

They are both okay, but the patch proposed by Robert was there first,
and includes a NEWS entry.  So I think that one should go in.

Thanks.

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Paul Eggert]

Eli Zaretskii wrote:
> the patch proposed by Robert was there first,
> and includes a NEWS entry.  So I think that one should go in.

OK, I installed that in the emacs-26 branch and am marking this bug as done.

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Colin Baxter]

>>>>> "Paul" == Paul Eggert <eggert@cs.ucla.edu> writes:

    Paul> Eli Zaretskii wrote:
    >> the patch proposed by Robert was there first, and includes a NEWS
    >> entry.  So I think that one should go in.

    Paul> OK, I installed that in the emacs-26 branch and am marking
    Paul> this bug as done.

I've tested this with my setup on a newly pulled emacs-26 branch. It
seems ok; with no difference seen from my previous arrangement, where I
had explicitly (setq mail-source-movemail-program "/usr/bin/movemail").

Thank you.

Best wishes,

-- 
Colin Baxter
m43cap@yandex.com

bug#31737: 26.1.50; Gnus does not use mailutils movemail

[Robert Pluim]

Colin Baxter <m43cap@yandex.com> writes:

>>>>>> "Paul" == Paul Eggert <eggert@cs.ucla.edu> writes:
>
>     Paul> Eli Zaretskii wrote:
>     >> the patch proposed by Robert was there first, and includes a NEWS
>     >> entry.  So I think that one should go in.
>
>     Paul> OK, I installed that in the emacs-26 branch and am marking
>     Paul> this bug as done.
>
> I've tested this with my setup on a newly pulled emacs-26 branch. It
> seems ok; with no difference seen from my previous arrangement, where I
> had explicitly (setq mail-source-movemail-program "/usr/bin/movemail").

Thanks for testing. Iʼm glad we donʼt appear to have broken anything.

Robert