bug#36937: 26.1; Sending mails doesn't set From: correctly

bug#36937: 26.1; Sending mails doesn't set From: correctly

[Jose A. Ortega Ruiz]

This problem started happening in builds from master around 4 days ago:

- start emacs -Q
- M-x mail (or any other form of sending email)
- Set a From: value different from user@localhostname
- Send mail using local sendmail

The mail is sent with a from header seen by postfix (i don't know about
other MTAs) as user@localhost, instead of the From: i set in the
headers.  By user@localhost i mean the unix username running emacs and
the hostname as returned by hostname(1).  It seems emacs is just
ignoring the value of From:, or setting it in a way postfix doesn't
understand.

I know almost for sure it's an emacs problem because emacs 26 or an
earlier version of emacs-snapshot is working without problems.

Thanks!


In GNU Emacs 27.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
 of 2019-08-03, unofficial emacs-snapshot build: http://emacs.secretsauce.net built on lilbastard
Windowing system distributor 'The X.Org Foundation', version 11.0.12004000
System Description: Debian GNU/Linux bullseye/sid

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Sending...
Save this mail sending choice? (y or n) n
Sending...done

Configured using:
 'configure --build x86_64-linux-gnu --prefix=/usr
 --sharedstatedir=/var/lib --libexecdir=/usr/lib
 --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --with-pop=yes
 --enable-locallisppath=/etc/emacs-snapshot:/etc/emacs:/usr/local/share/emacs/27.0.50/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/27.0.50/site-lisp:/usr/share/emacs/site-lisp
 --build x86_64-linux-gnu --prefix=/usr --sharedstatedir=/var/lib
 --libexecdir=/usr/lib --localstatedir=/var/lib
 --infodir=/usr/share/info --mandir=/usr/share/man --with-pop=yes
 --enable-locallisppath=/etc/emacs-snapshot:/etc/emacs:/usr/local/share/emacs/27.0.50/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/27.0.50/site-lisp:/usr/share/emacs/site-lisp
 --with-x=yes --with-x-toolkit=lucid --with-toolkit-scroll-bars
 --without-gconf --without-gsettings 'CFLAGS=-g -O2
 -fdebug-prefix-map=/build/emacs-snapshot-eSXVQs/emacs-snapshot-20190803+emacs-26.1-6779-g5ae1191e5b0=. -fstack-protector-strong
 -Wformat -Werror=format-security -Wall -fno-omit-frame-pointer'
 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' LDFLAGS=-Wl,-z,relro'

Configured features:
XAW3D XPM JPEG TIFF GIF PNG RSVG SOUND GPM DBUS GLIB NOTIFY INOTIFY ACL
LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS LUCID X11 XDBE XIM THREADS PDUMPER LCMS2 GMP

Important settings:
  value of $LANG: en_GB.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs text-property-search time-date
subr-x seq byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs
cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode
lisp-mode prog-mode register page menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify lcms2 dynamic-setting
font-render-setting x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 44987 9529)
 (symbols 48 6203 1)
 (strings 32 16184 1920)
 (string-bytes 1 513447)
 (vectors 16 9265)
 (vector-slots 8 129612 10712)
 (floats 8 24 26)
 (intervals 56 271 0)
 (buffers 992 13))

bug#36937: 26.1; Sending mails doesn't set From: correctly

[Sven Joachim]

notfound 36937 26.1
found 36937 27.0.50
thanks

On 2019-08-05 19:00 +0100, Jose A. Ortega Ruiz wrote:

> This problem started happening in builds from master around 4 days ago:
>
> - start emacs -Q
> - M-x mail (or any other form of sending email)
> - Set a From: value different from user@localhostname
> - Send mail using local sendmail
>
> The mail is sent with a from header seen by postfix (i don't know about
> other MTAs) as user@localhost, instead of the From: i set in the
> headers.  By user@localhost i mean the unix username running emacs and
> the hostname as returned by hostname(1).  It seems emacs is just
> ignoring the value of From:, or setting it in a way postfix doesn't
> understand.
>
> I know almost for sure it's an emacs problem because emacs 26 or an
> earlier version of emacs-snapshot is working without problems.

Confirmed here, although it did not prevent me personally from sending
mail, since I had already set up postfix to rewrite such
user@localhostname addresses back.

Reverting commit 3a59cc840693 (" * lisp/gnus/message.el: Reduce
redundancy with send-mail-function") fixed the problem.  It looks like
this commit changed some important options and looks NEWS-worthy.

Stefan?

Cheers,
       Sven

bug#36937: 26.1; Sending mails doesn't set From: correctly

[Sven Joachim]

On 2019-08-05 21:46 +0200, Sven Joachim wrote:

> On 2019-08-05 19:00 +0100, Jose A. Ortega Ruiz wrote:
>
>> This problem started happening in builds from master around 4 days ago:
>>
>> - start emacs -Q
>> - M-x mail (or any other form of sending email)
>> - Set a From: value different from user@localhostname
>> - Send mail using local sendmail
>>
>> The mail is sent with a from header seen by postfix (i don't know about
>> other MTAs) as user@localhost, instead of the From: i set in the
>> headers.  By user@localhost i mean the unix username running emacs and
>> the hostname as returned by hostname(1).  It seems emacs is just
>> ignoring the value of From:, or setting it in a way postfix doesn't
>> understand.
>>
>> I know almost for sure it's an emacs problem because emacs 26 or an
>> earlier version of emacs-snapshot is working without problems.
>
> Confirmed here, although it did not prevent me personally from sending
> mail, since I had already set up postfix to rewrite such
> user@localhostname addresses back.
>
> Reverting commit 3a59cc840693 (" * lisp/gnus/message.el: Reduce
> redundancy with send-mail-function") fixed the problem.  It looks like
> this commit changed some important options and looks NEWS-worthy.

Most notably, it changed the default of message-sendmail-f-is-evil,
which is now t.  Setting message-sendmail-f-is-evil back to nil restores
the previous behavior.

> Stefan?

Cheers,
       Sven

bug#36937: 26.1; Sending mails doesn't set From: correctly

[Stefan Monnier]

Sven Joachim <svenjoac@gmx.de> writes:

> notfound 36937 26.1
> found 36937 27.0.50
> thanks
>
> On 2019-08-05 19:00 +0100, Jose A. Ortega Ruiz wrote:
>
>> This problem started happening in builds from master around 4 days ago:
>>
>> - start emacs -Q
>> - M-x mail (or any other form of sending email)
>> - Set a From: value different from user@localhostname
>> - Send mail using local sendmail
>>
>> The mail is sent with a from header seen by postfix (i don't know about
>> other MTAs) as user@localhost, instead of the From: i set in the
>> headers.  By user@localhost i mean the unix username running emacs and
>> the hostname as returned by hostname(1).  It seems emacs is just
>> ignoring the value of From:, or setting it in a way postfix doesn't
>> understand.
>>
>> I know almost for sure it's an emacs problem because emacs 26 or an
>> earlier version of emacs-snapshot is working without problems.
>
> Confirmed here, although it did not prevent me personally from sending
> mail, since I had already set up postfix to rewrite such
> user@localhostname addresses back.
>
> Reverting commit 3a59cc840693 (" * lisp/gnus/message.el: Reduce
> redundancy with send-mail-function") fixed the problem.  It looks like
> this commit changed some important options and looks NEWS-worthy.

It is not supposed to change anything important, so this is just a bug.

The weird thing is that `M-x mail` doesn't use message-mode, so it
should (in theory) be unaffected by my patch which only touches
message.el.  Practice proves the theory wrong, tho.


        Stefan

bug#36937: 26.1; Sending mails doesn't set From: correctly

[Stefan Monnier]

>>> This problem started happening in builds from master around 4 days ago:
>>>
>>> - start emacs -Q
>>> - M-x mail (or any other form of sending email)
>>> - Set a From: value different from user@localhostname
>>> - Send mail using local sendmail
>>>
>>> The mail is sent with a from header seen by postfix (i don't know about
>>> other MTAs) as user@localhost, instead of the From: i set in the
>>> headers.  By user@localhost i mean the unix username running emacs and
>>> the hostname as returned by hostname(1).  It seems emacs is just
>>> ignoring the value of From:, or setting it in a way postfix doesn't
>>> understand.
>>>
>>> I know almost for sure it's an emacs problem because emacs 26 or an
>>> earlier version of emacs-snapshot is working without problems.
>>
>> Confirmed here, although it did not prevent me personally from sending
>> mail, since I had already set up postfix to rewrite such
>> user@localhostname addresses back.
>>
>> Reverting commit 3a59cc840693 (" * lisp/gnus/message.el: Reduce
>> redundancy with send-mail-function") fixed the problem.  It looks like
>> this commit changed some important options and looks NEWS-worthy.
>
> Most notably, it changed the default of message-sendmail-f-is-evil,
> which is now t.

Indeed, I aligned it with mail-specify-envelope-from.  
I guess I should revert this part.  But I'm still perplexed: `M-x mail`
shouldn't pay any attention to message-sendmail-f-is-evil when sending
the message.


        Stefan

bug#36937: 26.1; Sending mails doesn't set From: correctly

[Sven Joachim]

On 2019-08-06 03:32 -0400, Stefan Monnier wrote:

>>>> This problem started happening in builds from master around 4 days ago:
>>>>
>>>> - start emacs -Q
>>>> - M-x mail (or any other form of sending email)
>>>> - Set a From: value different from user@localhostname
>>>> - Send mail using local sendmail
>>>>
>>>> The mail is sent with a from header seen by postfix (i don't know about
>>>> other MTAs) as user@localhost, instead of the From: i set in the
>>>> headers.  By user@localhost i mean the unix username running emacs and
>>>> the hostname as returned by hostname(1).  It seems emacs is just
>>>> ignoring the value of From:, or setting it in a way postfix doesn't
>>>> understand.
>>>>
>>>> I know almost for sure it's an emacs problem because emacs 26 or an
>>>> earlier version of emacs-snapshot is working without problems.
>>>
>>> Confirmed here, although it did not prevent me personally from sending
>>> mail, since I had already set up postfix to rewrite such
>>> user@localhostname addresses back.
>>>
>>> Reverting commit 3a59cc840693 (" * lisp/gnus/message.el: Reduce
>>> redundancy with send-mail-function") fixed the problem.  It looks like
>>> this commit changed some important options and looks NEWS-worthy.
>>
>> Most notably, it changed the default of message-sendmail-f-is-evil,
>> which is now t.
>
> Indeed, I aligned it with mail-specify-envelope-from.
> I guess I should revert this part.  But I'm still perplexed: `M-x mail`
> shouldn't pay any attention to message-sendmail-f-is-evil when sending
> the message.

Oh, I think it actually does not.  I almost always use Gnus, or
otherwise C-x m aka compose-mail which uses message-mode.  With
M-x mail, message-sendmail-f-is-evil does not seem to have an effect,
postfix always sees the user@localhost address then.  And this is even
the case in the emacs-26 branch.

Jose, did you actually use "M-x mail", or did you use something else
which uses message-mode to compose and send the mail?

Cheers,
       Sven

bug#36937: 26.1; Sending mails doesn't set From: correctly

[Jose A. Ortega Ruiz]

On Tue, Aug 06 2019, Jose A. Ortega Ruiz wrote:

> On Tue, Aug 06 2019, Sven Joachim wrote:
>
> [...]
>
>> Oh, I think it actually does not.  I almost always use Gnus, or
>> otherwise C-x m aka compose-mail which uses message-mode.  With
>> M-x mail, message-sendmail-f-is-evil does not seem to have an effect,
>> postfix always sees the user@localhost address then.  And this is even
>> the case in the emacs-26 branch.
>>
>> Jose, did you actually use "M-x mail", or did you use something else
>> which uses message-mode to compose and send the mail?
>
> I was using M-x mail.  I observed it first with message-mode, though
> (normally invoked from Gnus), but i'm almost sure it's happened with
> mail too.  Here in emacs-26, i've got message-sendmail-f-is-evil nil,
> and things are working properly (in that postfix is seeing the From:
> address... i use that to multiplex over several smtp servers).

Sorry, what i said before was not true.

In emacs-27, M-x mail is working just fine (i.e., the correct From:
address is seen by postfix).  The problem only happens when using
message-mode (either via Gnus or simply via M-x message-mail) to send
the email.  Setting message-sendmail-f-is-evil back to nil fixes it for
me too.  And setting it to t in emacs-26 reproduces the (to me) bad
behaviour too.

Cheers,
jao

bug#36937: 26.1; Sending mails doesn't set From: correctly

[Stefan Monnier]

> In emacs-27, M-x mail is working just fine (i.e., the correct From:
> address is seen by postfix).  The problem only happens when using
> message-mode (either via Gnus or simply via M-x message-mail) to send
> the email.  Setting message-sendmail-f-is-evil back to nil fixes it for
> me too.  And setting it to t in emacs-26 reproduces the (to me) bad
> behaviour too.

Thank you for testing.  I've reverted this part of the change, so
message-sendmail-f-is-evil is always nil, as before.


        Stefan

bug#36937: 26.1; Sending mails doesn't set From: correctly

[Jose A. Ortega Ruiz]

On Tue, Aug 06 2019, Stefan Monnier wrote:

> Thank you for testing.  I've reverted this part of the change, so
> message-sendmail-f-is-evil is always nil, as before.

Thanks a lot, guys.  It's always a pleasure to report emacs bugs,
because of responsiveness and swiftness such as yours :)

jao