bug#66602: 29.1; smtpmail-send-it fails to authenticate using auth-sources

bug#66602: 29.1; smtpmail-send-it fails to authenticate using auth-sources

[Jens Östlund via Bug reports for GNU Emacs, the Swiss army knife of text editors]

When sending an email with 'smtpmail-send-it' the function 'smtpmail-try-auth-methods' fails to find the matching entry. Using 'edebug' I've found that the reason is this piece of code where the 'port' is converted to a string (smtpmail.el.gz:533-536):

'(setq port
	(if port
	    (format "%s" port)
	  "smtp"))'

When the 'port' is later passed to 'auth-source-search' on line 541 it fails to find any matching entries since ':port' is matched as a number, not a string. If I remove the above piece of code, so that 'port' is never turned into a string, sending the email works as expected. Maybe the best solution is to make 'auth-source-search' match both numeric and string values, at least for the ':port' argument.

SMTP session trace:

'220 mailrelay6 ESMTP
250-mailrelay6
250-PIPELINING
250-CHUNKING
250-SIZE 104857600
250-AUTH LOGIN PLAIN
250-ENHANCEDSTATUSCODES
250 8BITMIME
MAIL FROM:<xxxxxxx@xxxxxx.xx> SIZE=322
550 5.7.1 [M1] Authentication required (431ff6c7-6d37-11ee-accc-23968f462fbe)
QUIT
221 2.0.0 Bye

Process smtpmail connection broken by remote peer’

Best regards,

Jens Östlund

In GNU Emacs 29.1 (build 1, aarch64-apple-darwin22.6.0, NS
appkit-2299.70 Version 13.5 (Build 22G74))
Windowing system distributor 'Apple', version 10.3.2299
System Description:  macOS 13.5

Configured using:
'configure
--prefix=/nix/store/42jvqrs42gs59rvmcf996d5x5jrmda8v-emacs-29.1
--disable-build-details --with-modules --disable-ns-self-contained
--with-ns --with-native-compilation --with-tree-sitter'

Configured features:
ACL GLIB GMP GNUTLS JPEG JSON LIBXML2 MODULES NATIVE_COMP NOTIFY KQUEUE
NS PDUMPER PNG RSVG SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER
WEBP ZLIB

Important settings:
  value of $EMACSLOADPATH: /nix/store/2bsnl0w6hlgwj1h73m0nvzsf1awldxk7-emacs-packages-deps/share/emacs/site-lisp:
  value of $EMACSNATIVELOADPATH: /nix/store/2bsnl0w6hlgwj1h73m0nvzsf1awldxk7-emacs-packages-deps/share/emacs/native-lisp:
  locale-coding-system: utf-8-unix

Major mode: mu4e:main

Minor modes in effect:
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  editorconfig-mode: t
  pdf-occur-global-minor-mode: t
  mu4e-search-minor-mode: t
  mu4e-update-minor-mode: t
  mu4e-context-minor-mode: t
  mu4e-modeline-mode: t
  erc-track-mode: t
  erc-ring-mode: t
  erc-netsplit-mode: t
  erc-menu-mode: t
  erc-match-mode: t
  erc-list-mode: t
  erc-autojoin-mode: t
  erc-autoaway-mode: t
  erc-hl-nicks-mode: t
  erc-button-mode: t
  erc-fill-mode: t
  erc-stamp-mode: t
  erc-irccontrols-mode: t
  erc-noncommands-mode: t
  erc-keep-place-mode: t
  erc-move-to-prompt-mode: t
  erc-readonly-mode: t
  erc-scrolltobottom-mode: t
  erc-networks-mode: t
  global-auto-revert-mode: t
  savehist-mode: t
  vertico-mode: t
  corfu-history-mode: t
  corfu-echo-mode: t
  global-corfu-mode: t
  corfu-mode: t
  treemacs-filewatch-mode: t
  treemacs-follow-mode: t
  treemacs-git-mode: t
  treemacs-fringe-indicator-mode: t
  which-key-mode: t
  windmove-mode: t
  yas-global-mode: t
  yas-minor-mode: t
  marginalia-mode: t
  delete-selection-mode: t
  age-encryption-mode: t
  shell-dirtrack-mode: t
  recentf-mode: t
  pixel-scroll-mode: t
  direnv-mode: t
  override-global-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  global-prettify-symbols-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  overwrite-mode: overwrite-mode-binary

Load-path shadows:
/nix/store/n07y9g419wxwbklyzwfh818rw86spk9k-emacs-packages-deps/share/emacs/site-lisp/site-start hides /nix/store/2bsnl0w6hlgwj1h73m0nvzsf1awldxk7-emacs-packages-deps/share/emacs/site-lisp/site-start
/nix/store/n07y9g419wxwbklyzwfh818rw86spk9k-emacs-packages-deps/share/emacs/site-lisp/site-start hides /nix/store/42jvqrs42gs59rvmcf996d5x5jrmda8v-emacs-29.1/share/emacs/site-lisp/site-start
/Users/iensu/.emacs.d/elpa/transient-20230919.2146/transient hides /nix/store/42jvqrs42gs59rvmcf996d5x5jrmda8v-emacs-29.1/share/emacs/29.1/lisp/transient
/nix/store/n07y9g419wxwbklyzwfh818rw86spk9k-emacs-packages-deps/share/emacs/site-lisp/elpa/xref-1.6.3/xref hides /nix/store/42jvqrs42gs59rvmcf996d5x5jrmda8v-emacs-29.1/share/emacs/29.1/lisp/progmodes/xref
/nix/store/n07y9g419wxwbklyzwfh818rw86spk9k-emacs-packages-deps/share/emacs/site-lisp/elpa/project-0.10.0/project hides /nix/store/42jvqrs42gs59rvmcf996d5x5jrmda8v-emacs-29.1/share/emacs/29.1/lisp/progmodes/project
/Users/iensu/.emacs.d/elpa/seq-2.24/seq hides /nix/store/42jvqrs42gs59rvmcf996d5x5jrmda8v-emacs-29.1/share/emacs/29.1/lisp/emacs-lisp/seq
/nix/store/n07y9g419wxwbklyzwfh818rw86spk9k-emacs-packages-deps/share/emacs/site-lisp/elpa/let-alist-1.0.6/let-alist hides /nix/store/42jvqrs42gs59rvmcf996d5x5jrmda8v-emacs-29.1/share/emacs/29.1/lisp/emacs-lisp/let-alist

Features:
(shadow emacsbug edebug debug backtrace magit-extras treemacs-magit
magit-bookmark magit-submodule magit-blame magit-stash magit-reflog
magit-bisect magit-push magit-pull magit-fetch magit-clone magit-remote
magit-commit magit-sequence magit-notes magit-worktree magit-tag
magit-merge magit-branch magit-reset magit-files magit-refs magit-status
magit magit-repos magit-apply magit-wip magit-log which-func magit-diff
smerge-mode diff git-commit log-edit pcvs-util add-log magit-core
magit-autorevert magit-margin magit-transient magit-process with-editor
magit-mode transient magit-git magit-base magit-section crm
expand-region subword-mode-expansions text-mode-expansions
cc-mode-expansions the-org-mode-expansions js-mode-expansions
html-mode-expansions css-mode-expansions er-basic-expansions
expand-region-core expand-region-custom reveal network-stream mailalias
flyspell-popup flyspell ispell tabify man sort gnus-cite smiley
shr-color mm-archive mail-extr textsec uni-scripts idna-mapping
uni-confusable textsec-check qp epa-file conf-mode cl-print so-long view
misearch multi-isearch dabbrev cape vc-git vc-dispatcher shortdoc
mu4e-speedbar mu4e-contrib eshell esh-cmd esh-ext esh-opt esh-proc
esh-io esh-arg esh-module esh-groups esh-util mu4e-icalendar
gnus-icalendar org-capture icalendar diary-lib diary-loaddefs help-fns
radix-tree tramp-cmds cursor-sensor rainbow-delimiters editorconfig
editorconfig-core editorconfig-core-handle editorconfig-fnmatch hideshow
display-line-numbers cap-words superword subword hl-todo cus-start
checkdoc lisp-mnt flymake-proc flymake highlight-indentation
yaml-ts-mode wat-mode wat-mode-macro wat-mode-font-lock
typescript-ts-mode toml-ts-mode terraform-mode hcl-mode separedit
calc-misc markdown-toc smartparens-markdown markdown-mode edit-indirect
json-ts-mode add-node-modules-path smartparens-javascript js go-ts-mode
fish-mode dockerfile-ts-mode csharp-mode c-ts-common cc-langs sh-script
executable emmet-mode cl css-mode treesit smie smartparens-html
sgml-mode facemenu eww url-queue mm-url pdf-occur ibuf-ext ibuffer
ibuffer-loaddefs tablist tablist-filter semantic/wisent/comp
semantic/wisent semantic/wisent/wisent semantic/util-modes semantic/util
semantic semantic/tag semantic/lex semantic/fw mode-local cedet
pdf-isearch let-alist pdf-misc pdf-tools pdf-view jka-compr pdf-cache
pdf-info tq pdf-util pdf-macs image-mode exif socks nsm elpher mu4e
mu4e-org mu4e-notification notifications mu4e-main mu4e-view gnus-art
mm-uu mml2015 mm-view mml-smime smime gnutls dig gnus-sum gnus-group
gnus-undo gnus-start gnus-dbus dbus gnus-cloud nnimap nnmail mail-source
utf7 nnoo gnus-spec gnus-int gnus-range gnus-win gnus nnheader range
mu4e-headers mu4e-compose mu4e-draft mu4e-actions smtpmail mu4e-search
mu4e-lists mu4e-bookmarks mu4e-mark mu4e-message shr pixel-fill kinsoku
url-file svg xml dom flow-fill mu4e-contacts mu4e-update mu4e-folders
mu4e-context mu4e-query-items mu4e-server mu4e-modeline mu4e-vars
mu4e-helpers mu4e-config mu4e-window message sendmail yank-media puny
rfc822 mml mml-sec epa derived epg epg-config gnus-util mm-decode
mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 mm-util
ietf-drums mail-prsvr mailabbrev mail-utils gmm-utils mailheader
mu4e-obsolete erc-track erc-ring erc-netsplit erc-menu erc-match
erc-list erc-pcomplete erc-join erc-autoaway erc-hl-nicks erc-button
erc-fill erc-stamp erc-goodies erc erc-backend erc-networks erc-common
erc-compat erc-loaddefs org-tree-slide org-timer org-clock org-superstar
ox-gfm ox-texinfo ox-md ox-odt rng-loc rng-uri rng-parse rng-match
rng-dt rng-util rng-pttrn nxml-parse nxml-ns nxml-enc xmltok nxml-util
ox-latex ox-icalendar org-agenda ox-html table ox-ascii ox-publish ox
ob-calc calc-store calc-trail calc-ext calc calc-loaddefs calc-macs ob-C
smartparens-c cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles
cc-align cc-engine cc-vars cc-defs ob-shell org-protocol ob-js org-tempo
tempo org-indent ob-restclient restclient denote-org-dblock embark-org
org-element org-persist xdg org-id org-refile avl-tree generator
smartparens-org org ob ob-tangle ob-ref ob-lob ob-table ob-exp org-macro
org-src ob-comint org-pcomplete org-list org-footnote org-faces
org-entities noutline outline ob-emacs-lisp ob-core ob-eval org-cycle
org-table ol org-fold org-fold-core org-keys oc org-loaddefs cal-menu
calendar cal-loaddefs org-version org-compat org-macs denote
vc-use-package autorevert prettier-js flymake-eslint orderless savehist
vertico corfu-history corfu-echo corfu treemacs treemacs-header-line
treemacs-compatibility treemacs-mode treemacs-bookmarks treemacs-tags
imenu treemacs-interface treemacs-persistence treemacs-filewatch-mode
filenotify treemacs-follow-mode treemacs-rendering treemacs-annotations
treemacs-async treemacs-workspaces treemacs-dom treemacs-visuals
treemacs-fringe-indicator pulse treemacs-faces treemacs-icons
treemacs-scope treemacs-themes treemacs-core-utils pfuture inline
hl-line treemacs-logging treemacs-customization treemacs-macros desktop
frameset modus-vivendi-tinted-theme modus-themes all-the-icons
all-the-icons-faces data-material data-weathericons data-octicons
data-fileicons data-faicons data-alltheicons which-key windmove speedbar
ezimage dframe dumb-jump popup xref yasnippet wgrep-deadgrep wgrep grep
deadgrep spinner dired dired-loaddefs server pinentry pretty-hydra s
hydra lv embark-consult consult embark ffap marginalia compat compat-29
visual-fill-column emojify apropos tar-mode arc-mode archive-mode ht
synosaurus ido multi-vterm project vterm bookmark face-remap compile
text-property-search color term disp-table ehelp find-func vterm-module
term/xterm xterm xterm-color smartparens-config smartparens-text
smartparens loadhist edmacro iedit iedit-lib
mc-hide-unmatched-lines-mode mc-mark-more thingatpt mc-cycle-cursors
multiple-cursors-core advice rect kmacro delsel age tramp-sh tramp
tramp-loaddefs trampver tramp-integration files-x tramp-compat shell
pcomplete comint ansi-osc ansi-color parse-time iso8601 time-date
format-spec rfc6068 recentf tree-widget finder-inf pixel-scroll cua-base
ring comp comp-cstr warnings rx direnv diff-mode dash
exec-path-from-shell cl-extra help-mode cus-edit pp cus-load icons
wid-edit use-package use-package-ensure use-package-delight
use-package-diminish use-package-bind-key bind-key easy-mmode
use-package-core add-node-modules-path-autoloads age-autoloads
all-the-icons-autoloads cape-autoloads corfu-autoloads
deadgrep-autoloads denote-autoloads direnv-autoloads dumb-jump-autoloads
editorconfig-autoloads elpher-autoloads embark-consult-autoloads
consult-autoloads embark-autoloads emmet-mode-autoloads
emojify-autoloads erc-hl-nicks-autoloads exec-path-from-shell-autoloads
expand-region-autoloads fish-mode-autoloads flymake-eslint-autoloads
flymake-shellcheck-autoloads flyspell-popup-autoloads forge-autoloads
closql-autoloads emacsql-autoloads ghub-autoloads
highlight-indentation-autoloads hl-todo-autoloads iedit-autoloads
lsp-ui-autoloads lsp-mode-autoloads f-autoloads marginalia-autoloads
markdown-toc-autoloads markdown-mode-autoloads modus-themes-autoloads
multiple-cursors-autoloads nix-mode-autoloads ob-restclient-autoloads
orderless-autoloads org-superstar-autoloads org-tree-slide-autoloads
ox-gfm-autoloads pinentry-autoloads popup-autoloads
prettier-js-autoloads pretty-hydra-autoloads
rainbow-delimiters-autoloads rainbow-mode-autoloads restclient-autoloads
rjsx-mode-autoloads js2-mode-autoloads rust-mode-autoloads
scss-mode-autoloads separedit-autoloads edit-indirect-autoloads
smartparens-autoloads spinner-autoloads synosaurus-autoloads
terraform-mode-autoloads hcl-mode-autoloads treemacs-magit-autoloads
magit-autoloads pcase magit-section-autoloads git-commit-autoloads
transient-autoloads treemacs-autoloads cfrs-autoloads posframe-autoloads
ht-autoloads hydra-autoloads lv-autoloads pfuture-autoloads
ace-window-autoloads avy-autoloads s-autoloads dash-autoloads
treepy-autoloads vc-use-package-autoloads vertico-autoloads
visual-fill-column-autoloads wat-mode-autoloads web-mode-autoloads
which-key-autoloads with-editor-autoloads compat-autoloads seq-autoloads
xterm-color-autoloads yaml-autoloads yasnippet-autoloads info
mu4e-autoloads multi-vterm-autoloads pdf-tools-autoloads
project-autoloads tablist-autoloads vterm-autoloads package browse-url
url url-proxy url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf url-util mailcap url-handlers url-parse
auth-source cl-seq eieio eieio-core cl-macs password-cache json subr-x
map byte-opt gv bytecomp byte-compile url-vars cl-loaddefs cl-lib rmc
iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
vc-hooks lisp-float-type elisp-mode mwheel term/ns-win ns-win
ucs-normalize mule-util term/common-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads kqueue cocoa ns multi-tty
make-network-process native-compile emacs)

Memory information:
((conses 16 1102029 801206)
(symbols 48 65201 353)
(strings 32 316422 42434)
(string-bytes 1 10559472)
(vectors 16 158095)
(vector-slots 8 5148769 650640)
(floats 8 1326 2766)
(intervals 56 21611 11722)
(buffers 984 58))

bug#66602: 29.1; smtpmail-send-it fails to authenticate using auth-sources

[Eli Zaretskii]

> Date: Wed, 18 Oct 2023 08:17:07 +0200
> From:  Jens Östlund via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> When sending an email with 'smtpmail-send-it' the function 'smtpmail-try-auth-methods' fails to find the matching entry. Using 'edebug' I've found that the reason is this piece of code where the 'port' is converted to a string (smtpmail.el.gz:533-536):
> 
> '(setq port
> 	(if port
> 	    (format "%s" port)
> 	  "smtp"))'
> 
> When the 'port' is later passed to 'auth-source-search' on line 541 it fails to find any matching entries since ':port' is matched as a number, not a string. If I remove the above piece of code, so that 'port' is never turned into a string, sending the email works as expected. Maybe the best solution is to make 'auth-source-search' match both numeric and string values, at least for the ':port' argument.
> 
> SMTP session trace:
> 
> '220 mailrelay6 ESMTP
> 250-mailrelay6
> 250-PIPELINING
> 250-CHUNKING
> 250-SIZE 104857600
> 250-AUTH LOGIN PLAIN
> 250-ENHANCEDSTATUSCODES
> 250 8BITMIME
> MAIL FROM:<xxxxxxx@xxxxxx.xx> SIZE=322
> 550 5.7.1 [M1] Authentication required (431ff6c7-6d37-11ee-accc-23968f462fbe)
> QUIT
> 221 2.0.0 Bye
> 
> Process smtpmail connection broken by remote peer’

Thanks.

I guess I'm missing something, because I use smtpmail with
authentication and a specific port all the time without any trouble.
So maybe show the entry from your ~/.authinfo?

Also, can you point me to the place where you see that port is matched
as a number?  See also bug#55199, where a similar problem was fixed,
and the fix should be in Emacs 29.1.  Maybe you have a stale
auth-sources.elc?

bug#66602: 29.1; smtpmail-send-it fails to authenticate using auth-sources

[Jens Östlund via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Wed, 18 Oct 2023 08:17:07 +0200
>> From:  Jens Östlund via "Bug reports for GNU Emacs,
>>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
>>
>> When sending an email with 'smtpmail-send-it' the function 'smtpmail-try-auth-methods' fails to find the matching entry. Using 'edebug' I've found that the reason is this piece of code where the 'port' is converted to a string (smtpmail.el.gz:533-536):
>>
>> '(setq port
>> 	(if port
>> 	    (format "%s" port)
>> 	  "smtp"))'
>>
>> When the 'port' is later passed to 'auth-source-search' on line 541 it fails to find any matching entries since ':port' is matched as a number, not a string. If I remove the above piece of code, so that 'port' is never turned into a string, sending the email works as expected. Maybe the best solution is to make 'auth-source-search' match both numeric and string values, at least for the ':port' argument.
>>
>> SMTP session trace:
>>
>> '220 mailrelay6 ESMTP
>> 250-mailrelay6
>> 250-PIPELINING
>> 250-CHUNKING
>> 250-SIZE 104857600
>> 250-AUTH LOGIN PLAIN
>> 250-ENHANCEDSTATUSCODES
>> 250 8BITMIME
>> MAIL FROM:<xxxxxxx@xxxxxx.xx> SIZE=322
>> 550 5.7.1 [M1] Authentication required (431ff6c7-6d37-11ee-accc-23968f462fbe)
>> QUIT
>> 221 2.0.0 Bye
>>
>> Process smtpmail connection broken by remote peer’
>
> Thanks.
>
> I guess I'm missing something, because I use smtpmail with
> authentication and a specific port all the time without any trouble.
> So maybe show the entry from your ~/.authinfo?
>
> Also, can you point me to the place where you see that port is matched
> as a number?  See also bug#55199, where a similar problem was fixed,
> and the fix should be in Emacs 29.1.  Maybe you have a stale
> auth-sources.elc?

Indeed seem to have been a stale auth-sources.elc file. I recompiled the file and now everything works as expected. Thank you for your quick response, and sorry for taking up your time!

Cheers,

Jens

bug#66602: 29.1; smtpmail-send-it fails to authenticate using auth-sources

[Eli Zaretskii]

> From: Jens Östlund <jens@iensu.me>
> Cc: 66602@debbugs.gnu.org
> Date: Wed, 18 Oct 2023 14:51:27 +0200
> 
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I guess I'm missing something, because I use smtpmail with
> > authentication and a specific port all the time without any trouble.
> > So maybe show the entry from your ~/.authinfo?
> >
> > Also, can you point me to the place where you see that port is matched
> > as a number?  See also bug#55199, where a similar problem was fixed,
> > and the fix should be in Emacs 29.1.  Maybe you have a stale
> > auth-sources.elc?
> 
> Indeed seem to have been a stale auth-sources.elc file. I recompiled the file and now everything works as expected. Thank you for your quick response, and sorry for taking up your time!

Thanks, I'm therefore closing this bug.