bug#21733: 25.0.50; Umlaut in address makes smtpmail-via-smtp fail

bug#21733: 25.0.50; Umlaut in address makes smtpmail-via-smtp fail

[Stephen Berman]

[-- Attachment #1: Type: text/plain, Size: 5686 bytes --]

I tried to send an email with Gnus to an address containing an umlaut
('ä') in the domain name, and it failed with the error "Sending failed:
501 Syntax error in parameters or arguments".  When I tried sending the
same email to the same address using the GMX webmail interface, it
succeeded.  A copy of the sent email showed an address containing only
ascii characters instead of the address with the umlaut, so I assume the
latter is aliased to the former and GMX automatically found this.  It
would be nice if smtpmail.el could do the same thing instead of erroring
out.  I can reproduce the same error using the fake address
<stephen.berman@gmx.nät> (backtrace attached).

In GNU Emacs 25.0.50.3 (x86_64-suse-linux-gnu, GTK+ Version 3.14.15)
 of 2015-10-16
Repository revision: b0d190fbe2cd4a092fa186b3d674ec89ff000776
Windowing system distributor 'The X.Org Foundation', version 11.0.11601000
System Description:	openSUSE 13.2 (Harlequin) (x86_64)

Configured using:
 'configure 'CFLAGS=-Og -g3''

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND DBUS GCONF GSETTINGS NOTIFY
GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS
GTK3 X11

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Message

Minor modes in effect:
  diff-auto-refine-mode: t
  gnus-message-citation-mode: t
  mml-mode: t
  show-paren-mode: t
  recentf-mode: t
  shell-dirtrack-mode: t
  display-time-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  temp-buffer-resize-mode: t
  column-number-mode: t
  line-number-mode: t
  auto-fill-function: message-do-auto-fill
  abbrev-mode: t

Recent messages:
Back to top level
Mark set
Address ‘stephen.berman@gmx.nät’ (stephen.berman@ =?utf-8?Q?gmx=2En=C3=A4t?=) might be bogus.  Continue? (y or n) y
Sending...
Mark set [2 times]
Sending via mail...
Entering debugger...
Auto-saving...done
Making completion list...
Scanning for dabbrevs...done

Load-path shadows:
None found.

Features:
(shadow emacsbug mm-archive gnus-dup misearch multi-isearch find-dired
grep eieio-opt speedbar sb-image ezimage dframe edebug vc vc-dispatcher
vc-git diff-mode find-func debug mailalias smtpmail sendmail jka-compr
gnus-dired dabbrev nnir qp sort smiley gnus-cite mail-extr gnus-async
gnus-bcklg gnus-ml disp-table gnus-topic cursor-sensor nndraft nnmh
nnfolder gnutls network-stream nsm starttls gnus-agent gnus-srvr
gnus-score score-mode nnvirtual gnus-msg gnus-art mm-uu mml2015 mm-view
mml-smime smime dig mailcap nntp gnus-cache gnus-sum gnus-group
gnus-undo gnus-start gnus-cloud nnimap nnmail mail-source tls utf7 netrc
nnoo parse-time gnus-spec gnus-int gnus-range message dired-x dired
rfc822 mml mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
rfc2047 rfc2045 ietf-drums mailabbrev gmm-utils mailheader gnus-win gnus
gnus-ems nnheader mail-utils edmacro kmacro view cal-china lunar solar
cal-dst cal-bahai cal-islam cal-hebrew holidays hol-loaddefs face-remap
appt tramp-gvfs zeroconf url-util url-parse url-vars dbus xml
tramp-cache tramp-sh tramp tramp-compat auth-source cl-seq eieio
eieio-core cl-macs gnus-util mm-util help-fns mail-prsvr password-cache
tramp-loaddefs trampver srb-light-theme paren recentf tree-widget
wid-edit cus-start cus-load srb-cal+diary+appt todo-mode diary-lib
diary-loaddefs cal-menu calendar cal-loaddefs ido seq byte-opt gv
bytecomp byte-compile cconv cl-extra help-mode ess-toolbar ess-mouse
mouseme browse-url ess-menu ess-swv ess-noweb ess-noweb-font-lock-mode
ess-bugs-l essd-els ess-sas-d ess-sas-l ess-sas-a shell pcomplete
ess-sta-d ess-sta-l cc-vars cc-defs cl-loaddefs cl-lib make-regexp
ess-sp6-d ess-sp3-d ess-julia ess-r-d compile ess-tracebug format-spec
ess-roxy hideshow ess-help ess-developer ess-r-args ess-s-l ess ess-inf
comint ansi-color ring ess-mode ess-noweb-mode ess-utils ess-custom
executable ess-compat ess-site advice preview-latex tex-site auto-loads
w3m-load bbdb-loaddefs srb-recentf srb-mode-line time noutline outline
flotte-karotte srb-misc thingatpt pcase easy-mmode finder-inf package
easymenu epg-config time-date mule-util tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel x-win term/common-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list newcomment
elisp-mode lisp-mode prog-mode register page menu-bar rfn-eshadow timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
frame cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan
thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian
slovak czech european ethiopic indian cyrillic chinese charscript
case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote dbusbind inotify
dynamic-setting system-font-setting font-render-setting move-toolbar gtk
x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 354322 68420)
 (symbols 48 53391 23)
 (miscs 40 364 966)
 (strings 32 97674 12934)
 (string-bytes 1 3264178)
 (vectors 16 40955)
 (vector-slots 8 1499368 196903)
 (floats 8 1015 386)
 (intervals 56 4237 595)
 (buffers 976 52)
 (heap 1024 93673 37196))


[-- Attachment #2: backtrace --]
[-- Type: text/plain, Size: 14142 bytes --]

Debugger entered--Lisp error: (error "Sending failed: 501 Syntax error in parameters or arguments")
  signal(error ("Sending failed: 501 Syntax error in parameters or arguments"))
  error("Sending failed: %s" "501 Syntax error in parameters or arguments")
  (progn (error "Sending failed: %s" result))
  (if (setq result (smtpmail-via-smtp smtpmail-recipient-address-list tembuf)) (progn (error "Sending failed: %s" result)))
  (if (not (null smtpmail-recipient-address-list)) (if (setq result (smtpmail-via-smtp smtpmail-recipient-address-list tembuf)) (progn (error "Sending failed: %s" result))) (error "Sending failed; no recipients"))
  (if (not smtpmail-queue-mail) (if (not (null smtpmail-recipient-address-list)) (if (setq result (smtpmail-via-smtp smtpmail-recipient-address-list tembuf)) (progn (error "Sending failed: %s" result))) (error "Sending failed; no recipients")) (let* ((file-data (expand-file-name (format "%s_%i" (format-time-string "%Y-%m-%d_%H:%M:%S") (setq smtpmail-queue-counter (1+ smtpmail-queue-counter))) smtpmail-queue-dir)) (file-data (convert-standard-filename file-data)) (file-elisp (concat file-data ".el")) (buffer-data (create-file-buffer file-data)) (buffer-elisp (create-file-buffer file-elisp)) (buffer-scratch "*queue-mail*")) (if (file-exists-p smtpmail-queue-dir) nil (make-directory smtpmail-queue-dir t)) (save-current-buffer (set-buffer buffer-data) (erase-buffer) (set-buffer-file-coding-system (coding-system-change-eol-conversion (or smtpmail-code-conv-from (quote undecided)) (quote unix)) nil t) (insert-buffer-substring tembuf) (write-file file-data) (set-buffer buffer-elisp) (erase-buffer) (insert (concat "(setq smtpmail-recipient-address-list '" (prin1-to-string smtpmail-recipient-address-list) ")\n")) (write-file file-elisp) (set-buffer (generate-new-buffer buffer-scratch)) (insert (concat file-data "\n")) (append-to-file (point-min) (point-max) (expand-file-name smtpmail-queue-index-file smtpmail-queue-dir))) (kill-buffer buffer-scratch) (kill-buffer buffer-data) (kill-buffer buffer-elisp)))
  (save-current-buffer (set-buffer tembuf) (erase-buffer) (set-buffer-file-coding-system smtpmail-code-conv-from nil t) (insert-buffer-substring mailbuf) (goto-char (point-max)) (or (= (preceding-char) 10) (insert 10)) (mail-sendmail-undelimit-header) (setq delimline (point-marker)) (if mail-aliases (expand-mail-aliases (point-min) delimline)) (goto-char (point-min)) (while (and (re-search-forward "\n\n\n*" delimline t) (< (point) delimline)) (replace-match "\n")) (let ((case-fold-search t)) (goto-char (point-min)) (if (re-search-forward "^Subject:\\([ 	]*\n\\)+\\b" delimline t) (replace-match "") (if (and (re-search-forward "^Subject:\\([ 	]*\n\\)+" delimline t) (= (match-end 0) delimline)) (replace-match ""))) (goto-char (point-min)) (if (not (re-search-forward "^From:" delimline t)) (let* ((login smtpmail-mail-address) (fullname (user-full-name))) (cond ((eq mail-from-style (quote angles)) (insert "From: " fullname) (let (... ...) (goto-char fullname-start) (if ... ...)) (insert " <" login ">\n")) ((eq mail-from-style (quote parens)) (insert "From: " login " (") (let (...) (insert fullname) (let ... ... ... ... ...)) (insert ")\n")) ((null mail-from-style) (insert "From: " login "\n"))))) (goto-char (point-min)) (if (re-search-forward "^Message-Id:" delimline t) nil (insert "Message-Id: " (message-make-message-id) "\n")) (goto-char (point-min)) (if (re-search-forward "^Date:" delimline t) nil (insert "Date: " (message-make-date) "\n")) (let (charset) (goto-char (point-min)) (and (eq mail-send-nonascii (quote mime)) (not (re-search-forward "^MIME-version:" delimline t)) (progn (skip-chars-forward "\0-\x7f") (/= (point) (point-max))) smtpmail-code-conv-from (setq charset (coding-system-get smtpmail-code-conv-from (quote mime-charset))) (goto-char delimline) (insert "MIME-version: 1.0\n" "Content-type: text/plain; charset=" (symbol-name charset) "\nContent-Transfer-Encoding: 8bit\n"))) (goto-char (1+ delimline)) (if (eval mail-mailer-swallows-blank-line) (newline)) (goto-char (point-min)) (if (re-search-forward "^FCC:" delimline t) (let ((coding-system-for-write (coding-system-change-eol-conversion smtpmail-code-conv-from (quote unix)))) (mail-do-fcc delimline))) (if mail-interactive (save-current-buffer (set-buffer errbuf) (erase-buffer)))) (mail-encode-header (point-min) delimline) (setq smtpmail-address-buffer (generate-new-buffer "*smtp-mail*")) (setq smtpmail-recipient-address-list (smtpmail-deduce-address-list tembuf (point-min) delimline)) (kill-buffer smtpmail-address-buffer) (smtpmail-do-bcc delimline) (if (not smtpmail-queue-mail) (if (not (null smtpmail-recipient-address-list)) (if (setq result (smtpmail-via-smtp smtpmail-recipient-address-list tembuf)) (progn (error "Sending failed: %s" result))) (error "Sending failed; no recipients")) (let* ((file-data (expand-file-name (format "%s_%i" (format-time-string "%Y-%m-%d_%H:%M:%S") (setq smtpmail-queue-counter ...)) smtpmail-queue-dir)) (file-data (convert-standard-filename file-data)) (file-elisp (concat file-data ".el")) (buffer-data (create-file-buffer file-data)) (buffer-elisp (create-file-buffer file-elisp)) (buffer-scratch "*queue-mail*")) (if (file-exists-p smtpmail-queue-dir) nil (make-directory smtpmail-queue-dir t)) (save-current-buffer (set-buffer buffer-data) (erase-buffer) (set-buffer-file-coding-system (coding-system-change-eol-conversion (or smtpmail-code-conv-from (quote undecided)) (quote unix)) nil t) (insert-buffer-substring tembuf) (write-file file-data) (set-buffer buffer-elisp) (erase-buffer) (insert (concat "(setq smtpmail-recipient-address-list '" (prin1-to-string smtpmail-recipient-address-list) ")\n")) (write-file file-elisp) (set-buffer (generate-new-buffer buffer-scratch)) (insert (concat file-data "\n")) (append-to-file (point-min) (point-max) (expand-file-name smtpmail-queue-index-file smtpmail-queue-dir))) (kill-buffer buffer-scratch) (kill-buffer buffer-data) (kill-buffer buffer-elisp))))
  (unwind-protect (save-current-buffer (set-buffer tembuf) (erase-buffer) (set-buffer-file-coding-system smtpmail-code-conv-from nil t) (insert-buffer-substring mailbuf) (goto-char (point-max)) (or (= (preceding-char) 10) (insert 10)) (mail-sendmail-undelimit-header) (setq delimline (point-marker)) (if mail-aliases (expand-mail-aliases (point-min) delimline)) (goto-char (point-min)) (while (and (re-search-forward "\n\n\n*" delimline t) (< (point) delimline)) (replace-match "\n")) (let ((case-fold-search t)) (goto-char (point-min)) (if (re-search-forward "^Subject:\\([ 	]*\n\\)+\\b" delimline t) (replace-match "") (if (and (re-search-forward "^Subject:\\([ 	]*\n\\)+" delimline t) (= (match-end 0) delimline)) (replace-match ""))) (goto-char (point-min)) (if (not (re-search-forward "^From:" delimline t)) (let* ((login smtpmail-mail-address) (fullname (user-full-name))) (cond ((eq mail-from-style ...) (insert "From: " fullname) (let ... ... ...) (insert " <" login ">\n")) ((eq mail-from-style ...) (insert "From: " login " (") (let ... ... ...) (insert ")\n")) ((null mail-from-style) (insert "From: " login "\n"))))) (goto-char (point-min)) (if (re-search-forward "^Message-Id:" delimline t) nil (insert "Message-Id: " (message-make-message-id) "\n")) (goto-char (point-min)) (if (re-search-forward "^Date:" delimline t) nil (insert "Date: " (message-make-date) "\n")) (let (charset) (goto-char (point-min)) (and (eq mail-send-nonascii (quote mime)) (not (re-search-forward "^MIME-version:" delimline t)) (progn (skip-chars-forward "\0-\x7f") (/= (point) (point-max))) smtpmail-code-conv-from (setq charset (coding-system-get smtpmail-code-conv-from (quote mime-charset))) (goto-char delimline) (insert "MIME-version: 1.0\n" "Content-type: text/plain; charset=" (symbol-name charset) "\nContent-Transfer-Encoding: 8bit\n"))) (goto-char (1+ delimline)) (if (eval mail-mailer-swallows-blank-line) (newline)) (goto-char (point-min)) (if (re-search-forward "^FCC:" delimline t) (let ((coding-system-for-write (coding-system-change-eol-conversion smtpmail-code-conv-from ...))) (mail-do-fcc delimline))) (if mail-interactive (save-current-buffer (set-buffer errbuf) (erase-buffer)))) (mail-encode-header (point-min) delimline) (setq smtpmail-address-buffer (generate-new-buffer "*smtp-mail*")) (setq smtpmail-recipient-address-list (smtpmail-deduce-address-list tembuf (point-min) delimline)) (kill-buffer smtpmail-address-buffer) (smtpmail-do-bcc delimline) (if (not smtpmail-queue-mail) (if (not (null smtpmail-recipient-address-list)) (if (setq result (smtpmail-via-smtp smtpmail-recipient-address-list tembuf)) (progn (error "Sending failed: %s" result))) (error "Sending failed; no recipients")) (let* ((file-data (expand-file-name (format "%s_%i" ... ...) smtpmail-queue-dir)) (file-data (convert-standard-filename file-data)) (file-elisp (concat file-data ".el")) (buffer-data (create-file-buffer file-data)) (buffer-elisp (create-file-buffer file-elisp)) (buffer-scratch "*queue-mail*")) (if (file-exists-p smtpmail-queue-dir) nil (make-directory smtpmail-queue-dir t)) (save-current-buffer (set-buffer buffer-data) (erase-buffer) (set-buffer-file-coding-system (coding-system-change-eol-conversion (or smtpmail-code-conv-from ...) (quote unix)) nil t) (insert-buffer-substring tembuf) (write-file file-data) (set-buffer buffer-elisp) (erase-buffer) (insert (concat "(setq smtpmail-recipient-address-list '" (prin1-to-string smtpmail-recipient-address-list) ")\n")) (write-file file-elisp) (set-buffer (generate-new-buffer buffer-scratch)) (insert (concat file-data "\n")) (append-to-file (point-min) (point-max) (expand-file-name smtpmail-queue-index-file smtpmail-queue-dir))) (kill-buffer buffer-scratch) (kill-buffer buffer-data) (kill-buffer buffer-elisp)))) (kill-buffer tembuf) (if (bufferp errbuf) (kill-buffer errbuf)))
  (let ((errbuf (if mail-interactive (generate-new-buffer " smtpmail errors") 0)) (tembuf (generate-new-buffer " smtpmail temp")) (case-fold-search nil) delimline result (mailbuf (current-buffer)) (smtpmail-mail-address (or (and mail-specify-envelope-from (mail-envelope-from)) (let ((from (mail-fetch-field "from"))) (and from (car (cdr ...)))) (smtpmail-user-mail-address))) (smtpmail-code-conv-from (if enable-multibyte-characters (let ((sendmail-coding-system smtpmail-code-conv-from)) (select-message-coding-system))))) (unwind-protect (save-current-buffer (set-buffer tembuf) (erase-buffer) (set-buffer-file-coding-system smtpmail-code-conv-from nil t) (insert-buffer-substring mailbuf) (goto-char (point-max)) (or (= (preceding-char) 10) (insert 10)) (mail-sendmail-undelimit-header) (setq delimline (point-marker)) (if mail-aliases (expand-mail-aliases (point-min) delimline)) (goto-char (point-min)) (while (and (re-search-forward "\n\n\n*" delimline t) (< (point) delimline)) (replace-match "\n")) (let ((case-fold-search t)) (goto-char (point-min)) (if (re-search-forward "^Subject:\\([ 	]*\n\\)+\\b" delimline t) (replace-match "") (if (and (re-search-forward "^Subject:\\([ 	]*\n\\)+" delimline t) (= ... delimline)) (replace-match ""))) (goto-char (point-min)) (if (not (re-search-forward "^From:" delimline t)) (let* ((login smtpmail-mail-address) (fullname ...)) (cond (... ... ... ...) (... ... ... ...) (... ...)))) (goto-char (point-min)) (if (re-search-forward "^Message-Id:" delimline t) nil (insert "Message-Id: " (message-make-message-id) "\n")) (goto-char (point-min)) (if (re-search-forward "^Date:" delimline t) nil (insert "Date: " (message-make-date) "\n")) (let (charset) (goto-char (point-min)) (and (eq mail-send-nonascii (quote mime)) (not (re-search-forward "^MIME-version:" delimline t)) (progn (skip-chars-forward "\0-\x7f") (/= ... ...)) smtpmail-code-conv-from (setq charset (coding-system-get smtpmail-code-conv-from ...)) (goto-char delimline) (insert "MIME-version: 1.0\n" "Content-type: text/plain; charset=" (symbol-name charset) "\nContent-Transfer-Encoding: 8bit\n"))) (goto-char (1+ delimline)) (if (eval mail-mailer-swallows-blank-line) (newline)) (goto-char (point-min)) (if (re-search-forward "^FCC:" delimline t) (let ((coding-system-for-write ...)) (mail-do-fcc delimline))) (if mail-interactive (save-current-buffer (set-buffer errbuf) (erase-buffer)))) (mail-encode-header (point-min) delimline) (setq smtpmail-address-buffer (generate-new-buffer "*smtp-mail*")) (setq smtpmail-recipient-address-list (smtpmail-deduce-address-list tembuf (point-min) delimline)) (kill-buffer smtpmail-address-buffer) (smtpmail-do-bcc delimline) (if (not smtpmail-queue-mail) (if (not (null smtpmail-recipient-address-list)) (if (setq result (smtpmail-via-smtp smtpmail-recipient-address-list tembuf)) (progn (error "Sending failed: %s" result))) (error "Sending failed; no recipients")) (let* ((file-data (expand-file-name ... smtpmail-queue-dir)) (file-data (convert-standard-filename file-data)) (file-elisp (concat file-data ".el")) (buffer-data (create-file-buffer file-data)) (buffer-elisp (create-file-buffer file-elisp)) (buffer-scratch "*queue-mail*")) (if (file-exists-p smtpmail-queue-dir) nil (make-directory smtpmail-queue-dir t)) (save-current-buffer (set-buffer buffer-data) (erase-buffer) (set-buffer-file-coding-system (coding-system-change-eol-conversion ... ...) nil t) (insert-buffer-substring tembuf) (write-file file-data) (set-buffer buffer-elisp) (erase-buffer) (insert (concat "(setq smtpmail-recipient-address-list '" ... ")\n")) (write-file file-elisp) (set-buffer (generate-new-buffer buffer-scratch)) (insert (concat file-data "\n")) (append-to-file (point-min) (point-max) (expand-file-name smtpmail-queue-index-file smtpmail-queue-dir))) (kill-buffer buffer-scratch) (kill-buffer buffer-data) (kill-buffer buffer-elisp)))) (kill-buffer tembuf) (if (bufferp errbuf) (kill-buffer errbuf))))
  smtpmail-send-it()
  message-multi-smtp-send-mail()
  gnus-agent-send-mail()
  message-send-mail(nil)
  message-send-via-mail(nil)
  message-send(nil)
  message-send-and-exit(nil)
  funcall-interactively(message-send-and-exit nil)
  call-interactively(message-send-and-exit nil nil)
  command-execute(message-send-and-exit)

bug#21733: 25.0.50; Umlaut in address makes smtpmail-via-smtp fail

[Random832]

Stephen Berman <stephen.berman@gmx.net> writes:
> I tried to send an email with Gnus to an address containing an umlaut
> ('ä') in the domain name, and it failed with the error "Sending failed:
> 501 Syntax error in parameters or arguments".  When I tried sending the
> same email to the same address using the GMX webmail interface, it
> succeeded.  A copy of the sent email showed an address containing only
> ascii characters instead of the address with the umlaut, so I assume the
> latter is aliased to the former and GMX automatically found this.

Did this address begin with "xn--" in the segment which had contained
non-ascii characters in the original, such as "xn--nt-via" for "nät"?

> It
> would be nice if smtpmail.el could do the same thing instead of erroring
> out.  I can reproduce the same error using the fake address
> <stephen.berman@gmx.nät> (backtrace attached).

There is a file called punycode.el in libidn which encodes strings in
the way I mentioned above, maybe it would be worthwhile to include this
in Emacs - or at least have a hook which will use it if possible and
error out immediately (for domains containing non-ASCII characters) if
it's not present. This would be relevant to more places than just email.

This uses a command-line tool (part of libidn) to do the actual
decoding, I don't know how feasible it would be to do it in pure elisp.

bug#21733: 25.0.50; Umlaut in address makes smtpmail-via-smtp fail

[Andreas Schwab]

Random832 <random832@fastmail.com> writes:

> This uses a command-line tool (part of libidn) to do the actual
> decoding, I don't know how feasible it would be to do it in pure elisp.

Gnus already used libidn if available, see gnus-use-idna.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

bug#21733: 25.0.50; Umlaut in address makes smtpmail-via-smtp fail

[Stephen Berman]

On Thu, 22 Oct 2015 10:00:34 -0400 Random832 <random832@fastmail.com> wrote:

> Stephen Berman <stephen.berman@gmx.net> writes:
>> I tried to send an email with Gnus to an address containing an umlaut
>> ('ä') in the domain name, and it failed with the error "Sending failed:
>> 501 Syntax error in parameters or arguments".  When I tried sending the
>> same email to the same address using the GMX webmail interface, it
>> succeeded.  A copy of the sent email showed an address containing only
>> ascii characters instead of the address with the umlaut, so I assume the
>> latter is aliased to the former and GMX automatically found this.
>
> Did this address begin with "xn--" in the segment which had contained
> non-ascii characters in the original, such as "xn--nt-via" for "nät"?

Yes, indeed!

>> It
>> would be nice if smtpmail.el could do the same thing instead of erroring
>> out.  I can reproduce the same error using the fake address
>> <stephen.berman@gmx.nät> (backtrace attached).
>
> There is a file called punycode.el in libidn which encodes strings in
> the way I mentioned above, maybe it would be worthwhile to include this
> in Emacs - or at least have a hook which will use it if possible and
> error out immediately (for domains containing non-ASCII characters) if
> it's not present. This would be relevant to more places than just email.
>
> This uses a command-line tool (part of libidn) to do the actual
> decoding, I don't know how feasible it would be to do it in pure elisp.

Very interesting, thanks for pointing this out!  I hope something like
this can be added to Emacs.

Steve Berman

bug#21733: 25.0.50; Umlaut in address makes smtpmail-via-smtp fail

[Stephen Berman]

On Thu, 22 Oct 2015 16:19:59 +0200 Andreas Schwab <schwab@suse.de> wrote:

> Random832 <random832@fastmail.com> writes:
>
>> This uses a command-line tool (part of libidn) to do the actual
>> decoding, I don't know how feasible it would be to do it in pure elisp.
>
> Gnus already used libidn if available, see gnus-use-idna.

Thanks, I didn't know about that option.  Would've been nice if the
error could have mentioned it.

Steve Berman

bug#21733: 25.0.50; Umlaut in address makes smtpmail-via-smtp fail

[Andreas Schwab]

Stephen Berman <stephen.berman@gmx.net> writes:

> On Thu, 22 Oct 2015 16:19:59 +0200 Andreas Schwab <schwab@suse.de> wrote:
>
>> Random832 <random832@fastmail.com> writes:
>>
>>> This uses a command-line tool (part of libidn) to do the actual
>>> decoding, I don't know how feasible it would be to do it in pure elisp.
>>
>> Gnus already used libidn if available, see gnus-use-idna.
>
> Thanks, I didn't know about that option.  Would've been nice if the
> error could have mentioned it.

Your MTA knows nothing about Gnus.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

bug#21733: 25.0.50; Umlaut in address makes smtpmail-via-smtp fail

[Stephen Berman]

On Thu, 22 Oct 2015 17:04:52 +0200 Andreas Schwab <schwab@suse.de> wrote:

> Stephen Berman <stephen.berman@gmx.net> writes:
>
>> On Thu, 22 Oct 2015 16:19:59 +0200 Andreas Schwab <schwab@suse.de> wrote:
>>
>>> Random832 <random832@fastmail.com> writes:
>>>
>>>> This uses a command-line tool (part of libidn) to do the actual
>>>> decoding, I don't know how feasible it would be to do it in pure elisp.
>>>
>>> Gnus already used libidn if available, see gnus-use-idna.
>>
>> Thanks, I didn't know about that option.  Would've been nice if the
>> error could have mentioned it.
>
> Your MTA knows nothing about Gnus.

But maybe Gnus could intercept the error message and modify it
accordingly (no idea if this is possible and feasible, but it would be
nice).

Steve Berman

bug#21733: 25.0.50; Umlaut in address makes smtpmail-via-smtp fail

[Lars Ingebrigtsen]

Random832 <random832@fastmail.com> writes:

> This uses a command-line tool (part of libidn) to do the actual
> decoding, I don't know how feasible it would be to do it in pure elisp.

Yes, Emacs should definitely support IDNA addresses out of the box.  How
hard can it be to create an all-Elisp IDNA library, I wonder?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#21733: 25.0.50; Umlaut in address makes smtpmail-via-smtp fail

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Random832 <random832@fastmail.com> writes:
>
>> This uses a command-line tool (part of libidn) to do the actual
>> decoding, I don't know how feasible it would be to do it in pure elisp.
>
> Yes, Emacs should definitely support IDNA addresses out of the box.  How
> hard can it be to create an all-Elisp IDNA library, I wonder?

Not very.  I've slapped together a basic IDNA encoding library now.
Clean up and document tomorrow, I think.

But I guess the question remains where the encoding should be done.  I
think this is up to the Mail User Agent?  So it would in any case be
wrong for smtpmail to touch this...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#21733: 25.0.50; Umlaut in address makes smtpmail-via-smtp fail

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> But I guess the question remains where the encoding should be done.  I
> think this is up to the Mail User Agent?  So it would in any case be
> wrong for smtpmail to touch this...

This has apparently been fixed now (Message does the IDNA stuff), so I'm
closing this bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no