bug#35682: 27.0.50; Weird failure to authenticate in smtpmail

bug#35682: 27.0.50; Weird failure to authenticate in smtpmail

[Stefan Monnier]

Package: Emacs
Version: 27.0.50


Recently my SMTP server was updated and I started to get authentication
errors when sending email.  More specifically, when sending from
message-mode, I get the following error:

    Debugger entered--Lisp error: (error "Sending failed: 453 4.7.1 <monnier@iro.umontreal.ca>: Sender address rejected: not logged in")
      signal(error ("Sending failed: 453 4.7.1 <monnier@iro.umontreal.ca>: Sender address rejected: not logged in"))
      error("Sending failed: 453 4.7.1 <monnier@iro.umontreal.ca>: Sender address rejected: not logged in")
      smtpmail-send-it()
      message-smtpmail-send-it()
      message-multi-smtp-send-mail()
      message-send-mail(nil)
      message-send-via-mail(nil)
      message-send(nil)
      message-send-and-exit(nil)
      funcall-interactively(message-send-and-exit nil)
      call-interactively(message-send-and-exit nil nil)
      command-execute(message-send-and-exit)

My smtpmail-related settings are:

    (setq send-mail-function #'smtpmail-send-it)
    (setq smtpmail-smtp-service 587)
    (setq smtpmail-smtp-server <mailserver>)
    (setq smtpmail-smtp-user "monnier@iro.umontreal.ca")

and until recently this worked fine, prompting me for a password and
authenticating with it.  But now, I don't get prompted for a password
any more.

I can work around the problem with the patch below, but I have no idea
what other undesired effects it should have.  Apparently the code
catches the error 530 and treats it as a request to perform
authentication, so maybe the origin of the problem is that the mail
server now returns 453 instead of 530?  Should Emacs handle 453 like it
handles 530?  Should I report the problem to my SMTP admin instead?


        Stefan



In GNU Emacs 27.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
 of 2019-05-08 built on pastel
Repository revision: e8754776a6d9cef4b28c3f0ad8320669656500d9
Repository branch: work
Windowing system distributor 'The X.Org Foundation', version 11.0.11902000
System Description: Debian GNU/Linux 9 (stretch)

Recent messages:
Saving file /home/monnier/src/emacs/elpa/packages/haskell-tng/test/haskell-tng-syntax-test.el...
Wrote /home/monnier/src/emacs/elpa/packages/haskell-tng/test/haskell-tng-syntax-test.el
Saving file /home/monnier/src/emacs/elpa/packages/haskell-tng/test/haskell-tng-syntax-test.el...
Wrote /home/monnier/src/emacs/elpa/packages/haskell-tng/test/haskell-tng-syntax-test.el
Mark saved where search started
Finding changes in /home/monnier/src/elisp/haskell-tng.el/...done
Warning: hide-sublevels is obsolete!
Mark set [3 times]
Saved text until "eftest haskell-tng-syntax-file-tests ()
"

Configured using:
 'configure -C --enable-checking --with-modules --enable-check-lisp-object-type
 'CFLAGS=-Wall -g3 -Og -Wno-pointer-sign'
 PKG_CONFIG_PATH=/home/monnier/lib/pkgconfig'

Configured features:
XAW3D XPM JPEG TIFF GIF PNG SOUND GPM DBUS NOTIFY INOTIFY GNUTLS LIBXML2
FREETYPE XFT ZLIB TOOLKIT_SCROLL_BARS LUCID X11 XDBE XIM MODULES THREADS
PDUMPER GMP

Important settings:
  value of $LANG: fr_CH.UTF-8
  locale-coding-system: utf-8-unix

Major mode: InactiveMinibuffer

Minor modes in effect:
  shell-dirtrack-mode: t
  electric-pair-mode: t
  global-reveal-mode: t
  reveal-mode: t
  auto-insert-mode: t
  savehist-mode: t
  minibuffer-electric-default-mode: t
  global-compact-docstrings-mode: t
  url-handler-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  global-prettify-symbols-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  line-number-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
/home/monnier/src/emacs/elpa/packages/adaptive-wrap/adaptive-wrap hides /home/monnier/src/emacs/elpa/packages/taskpaper-mode/adaptive-wrap
/home/monnier/src/emacs/elpa/packages/svg/svg hides /home/monnier/src/emacs/work/lisp/svg
/home/monnier/src/emacs/elpa/packages/ada-mode/ada-prj hides /home/monnier/src/emacs/work/lisp/progmodes/ada-prj
/home/monnier/src/emacs/elpa/packages/ada-mode/ada-stmt hides /home/monnier/src/emacs/work/lisp/progmodes/ada-stmt
/home/monnier/src/emacs/elpa/packages/ada-mode/ada-mode hides /home/monnier/src/emacs/work/lisp/progmodes/ada-mode
/home/monnier/src/emacs/elpa/packages/ada-mode/ada-xref hides /home/monnier/src/emacs/work/lisp/progmodes/ada-xref
/home/monnier/src/emacs/elpa/packages/hyperbole/set hides /home/monnier/src/emacs/work/lisp/emacs-lisp/set
/home/monnier/src/emacs/elpa/packages/crisp/crisp hides /home/monnier/src/emacs/work/lisp/obsolete/crisp
/home/monnier/src/emacs/elpa/packages/landmark/landmark hides /home/monnier/src/emacs/work/lisp/obsolete/landmark

Features:
(shadow sort mail-extr emacsbug sendmail log-view vc-bzr vc-src vc-sccs
vc-svn vc-cvs vc-rcs vc-dir ewoc shell pcomplete grep log-edit message
rmc puny dired dired-loaddefs format-spec rfc822 mml mml-sec gnus-util
rmail rmail-loaddefs text-property-search mm-decode mm-bodies mm-encode
mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr
mailabbrev mail-utils gmm-utils mailheader pcvs-util pcase add-log
smerge-mode vc vc-dispatcher cl-print debug backtrace executable
copyright dabbrev eieio-opt speedbar imenu sb-image ezimage dframe
find-func completion cl-extra advice help-fns autoload radix-tree
misearch multi-isearch vc-git diff-mode filecache epa-file epa derived
epg server time-date flymake-proc flymake compile comint ansi-color ring
warnings noutline outline easy-mmode flyspell ispell checkdoc lisp-mnt
thingatpt help-mode load-dir elec-pair reveal autoinsert savehist
minibuf-eldef disp-table compact-docstrings cl-seq inline kotl-autoloads
realgud-recursive-autoloads finder-inf url-auth info vm-autoloads
mule-util lmc-autoloads proof-site proof-autoloads package easymenu
epg-config url-handlers url-parse auth-source eieio eieio-core cl-macs
gv eieio-loaddefs password-cache json subr-x map url-vars seq byte-opt
bytecomp byte-compile cconv cl-loaddefs cl-lib tooltip eldoc electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow isearch timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame simple minibuffer cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help abbrev obarray cl-preloaded nadvice loaddefs button faces cus-face
macroexp files text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget hashtable-print-readable backquote threads
dbusbind inotify dynamic-setting font-render-setting x-toolkit
x multi-tty make-network-process emacs)

Memory information:
((conses 16 285720 39081)
 (symbols 48 15867 1) (strings 32 68170 6261) (string-bytes 1 2205149)
 (vectors 16 65536)
 (vector-slots 8 1453965 117536) (floats 8 137 228) (intervals 56 3877 887)
 (buffers 992 62))

bug#35682: 27.0.50; Weird failure to authenticate in smtpmail

[Robert Pluim]

>>>>> On Fri, 10 May 2019 19:26:42 -0400, Stefan Monnier <monnier@iro.umontreal.ca> said:

    Stefan> Package: Emacs Version: 27.0.50


    Stefan> Recently my SMTP server was updated and I started to get
    Stefan> authentication errors when sending email.  More
    Stefan> specifically, when sending from message-mode, I get the
    Stefan> following error:

    Stefan>     Debugger entered--Lisp error: (error "Sending failed:
    Stefan> 453 4.7.1 <monnier@iro.umontreal.ca>: Sender address
    Stefan> rejected: not logged in") signal(error ("Sending failed:
    Stefan> 453 4.7.1 <monnier@iro.umontreal.ca>: Sender address
    Stefan> rejected: not logged in")) error("Sending failed: 453
    Stefan> 4.7.1 <monnier@iro.umontreal.ca>: Sender address rejected:
    Stefan> not logged in") smtpmail-send-it()
    Stefan> message-smtpmail-send-it() message-multi-smtp-send-mail()
    Stefan> message-send-mail(nil) message-send-via-mail(nil)
    Stefan> message-send(nil) message-send-and-exit(nil)
    Stefan> funcall-interactively(message-send-and-exit nil)
    Stefan> call-interactively(message-send-and-exit nil nil)
    Stefan> command-execute(message-send-and-exit)

    Stefan> My smtpmail-related settings are:

    Stefan>     (setq send-mail-function #'smtpmail-send-it) (setq
    Stefan> smtpmail-smtp-service 587) (setq smtpmail-smtp-server
    Stefan> <mailserver>) (setq smtpmail-smtp-user
    Stefan> "monnier@iro.umontreal.ca")

Iʼd encourage 465 rather than 587, but that might not work for you.

    Stefan> and until recently this worked fine, prompting me for a
    Stefan> password and authenticating with it.  But now, I don't get
    Stefan> prompted for a password any more.

    Stefan> I can work around the problem with the patch below, but I
    
ENOPATCH (but I can imagine what it looks like)

    Stefan> have no idea what other undesired effects it should have.
    Stefan> Apparently the code catches the error 530 and treats it as
    Stefan> a request to perform authentication, so maybe the origin
    Stefan> of the problem is that the mail server now returns 453
    Stefan> instead of 530?  Should Emacs handle 453 like it handles
    Stefan> 530?  Should I report the problem to my SMTP admin
    Stefan> instead?

Talk to your SMTP admin, they've messed something up. 453 is basically
'go away, Iʼm not accepting messages'. 530 is 'go away, you haven't
authenticated yourself'.

Iʼm assuming you donʼt have an authinfo entry for this user? If I
remember correctly, when you do have such an entry (including a
passwordless one) smtpmail.el will proactively authenticate, rather
than wait for a rejection. (see smtpmail-try-auth-methods)

Robert

bug#35682: 27.0.50; Weird failure to authenticate in smtpmail

[Stefan Monnier]

>     Stefan>     (setq send-mail-function #'smtpmail-send-it) (setq
>     Stefan> smtpmail-smtp-service 587) (setq smtpmail-smtp-server
>     Stefan> <mailserver>) (setq smtpmail-smtp-user
>     Stefan> "monnier@iro.umontreal.ca")
> Iʼd encourage 465 rather than 587, but that might not work for you.

Indeed, 465 doesn't work here.

>     Stefan> I can work around the problem with the patch below, but I
> ENOPATCH (but I can imagine what it looks like)

Duh.  See below.

> Talk to your SMTP admin, they've messed something up. 453 is basically
> 'go away, Iʼm not accepting messages'. 530 is 'go away, you haven't
> authenticated yourself'.

Yet the text they return says pretty much what you describe of 530.

> Iʼm assuming you donʼt have an authinfo entry for this user?

Indeed.

> If I remember correctly, when you do have such an entry (including
> a passwordless one) smtpmail.el will proactively authenticate, rather
> than wait for a rejection. (see smtpmail-try-auth-methods)

Shouldn't smtpmail.el also proactively authenticate when
smtpmail-smtp-user is set?


        Stefan


diff --git a/lisp/mail/smtpmail.el b/lisp/mail/smtpmail.el
index a79e3db52d..8bb8cb8fad 100644
--- a/lisp/mail/smtpmail.el
+++ b/lisp/mail/smtpmail.el
@@ -336,7 +336,11 @@ smtpmail-send-it
 	      (if (not (null smtpmail-recipient-address-list))
 		  (when (setq result
 			      (smtpmail-via-smtp
-			       smtpmail-recipient-address-list tembuf))
+			       smtpmail-recipient-address-list tembuf
+                               ;; FIXME: It looks like I need this setting
+                               ;; otherwise my SMTP server signals a 453 error
+                               ;; because I'm not authenticated.
+                               t))      ;ask-for-password
 		    (error "Sending failed: %s" result))
 		(error "Sending failed; no recipients"))
 	    (let* ((file-data

bug#35682: 27.0.50; Weird failure to authenticate in smtpmail

[Robert Pluim]

>>>>> On Sat, 11 May 2019 09:12:40 -0400, Stefan Monnier <monnier@iro.umontreal.ca> said:
    >> Talk to your SMTP admin, they've messed something up. 453 is
    >> basically 'go away, Iʼm not accepting messages'. 530 is 'go
    >> away, you haven't authenticated yourself'.

    Stefan> Yet the text they return says pretty much what you
    Stefan> describe of 530.

Yes, but 

1. That text has no semantics in SMTP, itʼs only for humans
2. The extended error code they send is 4.7.1, which is 
   a) Not meant to be used with 453
   b) Means "you are not allowed to send to this specific destination"
   
They should be sending 530, like SMTP servers have forever (and then
smtpmail won't care about the specific extended error).

    >> Iʼm assuming you donʼt have an authinfo entry for this user?

    Stefan> Indeed.

    >> If I remember correctly, when you do have such an entry
    >> (including a passwordless one) smtpmail.el will proactively
    >> authenticate, rather than wait for a rejection. (see
    >> smtpmail-try-auth-methods)

    Stefan> Shouldn't smtpmail.el also proactively authenticate when
    Stefan> smtpmail-smtp-user is set?

Not unless you want to change longstanding default behaviour, Iʼm sure
people will object (although I personally think itʼs a good
change). Is there a reason you canʼt put the entry in authinfo
(without a password)?

Robert

bug#35682: 27.0.50; Weird failure to authenticate in smtpmail

[Noam Postavsky]

forcemerge 26359 35682
quit

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> +			       smtpmail-recipient-address-list tembuf
> +                               ;; FIXME: It looks like I need this setting
> +                               ;; otherwise my SMTP server signals a 453 error
> +                               ;; because I'm not authenticated.
> +                               t))      ;ask-for-password

This can now be controlled with smtpmail-servers-requiring-authorization.