bug#25645: 26.0.50; message.el: Fcc+handler results in write coding issues in emacs 26

[Yuri D'Elia <wavexx@thregr.org>]

I'm using message.el in conjunction with mu4e, which uses Fcc and a
message-fcc-handler-function to save the outgoing message directly into
a maildir folder. Nothing fancy: Fcc is set via a message-send-hook,
then in the handler we just 'write-file and some bookeeping.

It works as expected with emacs 25.2, but fails on 26 when the body
contains unicode characters. Somehow when the mail is encoded in quoted
printable with an utf-8 charset, the buffer seem to contain raw bytes as
if the buffer has an incorrect buffer-file-coding-system.

When 'write-file is issued, emacs prompts for a coding system, which is
unexpected.

The stack is as follows:

  read-from-minibuffer("Select coding system (default raw-text): " nil (keymap (10 . minibuffer-complete-and-exit) (13 . minibuffer-complete-and-exit) keymap (menu-bar keymap (minibuf "Minibuf" keymap (tab menu-item "Complete" minibuffer-complete :help "Complete as far as possible") (space menu-item "Complete Word" minibuffer-complete-word :help "Complete at most one word") (63 menu-item "List Completions" minibuffer-completion-help :help "Display all possible completions") "Minibuf")) (27 keymap (118 . switch-to-completions)) (prior . switch-to-completions) (63 . minibuffer-completion-help) (32 . minibuffer-complete-word) (9 . minibuffer-complete) keymap (menu-bar keymap (minibuf "Minibuf" keymap (previous menu-item "Previous History Item" previous-history-element :help "Put previous minibuffer history element in the minibuffer") (next menu-item "Next History Item" next-history-element :help "Put next minibuffer history element in the minibuffer") (isearch-backward menu-item "Isearch History Backward" isearch-backward :help "Incrementally search minibuffer history backward") (isearch-forward menu-item "Isearch History Forward" isearch-forward :help "Incrementally search minibuffer history forward") (return menu-item "Enter" exit-minibuffer :key-sequence "
" :help "Terminate input and exit minibuffer") (quit menu-item "Quit" abort-recursive-edit :help "Abort input and exit minibuffer") "Minibuf")) (10 . exit-minibuffer) (13 . exit-minibuffer) (7 . abort-recursive-edit) (C-tab . file-cache-minibuffer-complete) (9 . self-insert-command) (XF86Back . previous-history-element) (up . previous-line-or-history-element) (prior . previous-history-element) (XF86Forward . next-history-element) (down . next-line-or-history-element) (next . next-history-element) (27 keymap (114 . previous-matching-history-element) (115 . next-matching-history-element) (112 . previous-history-element) (110 . next-history-element))) nil coding-system-history "raw-text" nil)
  completing-read-default("Select coding system (default raw-text): " (("utf-8-hfs") ("utf-8-hfs-mac") ("utf-8-hfs-dos") ("utf-8-hfs-unix") ("georgian-academy") ("georgian-academy-mac") ("georgian-academy-dos") ("georgian-academy-unix") ("georgian-ps") ("georgian-ps-mac") ("georgian-ps-dos") ("georgian-ps-unix") ("cp1256") ("cp1256-mac") ("cp1256-dos") ("cp1256-unix") ("windows-1256") ("windows-1256-mac") ("windows-1256-dos") ("windows-1256-unix") ("iso-8859-6") ("iso-8859-6-mac") ("iso-8859-6-dos") ("iso-8859-6-unix") ("tcvn-5712") ("tcvn-5712-mac") ("tcvn-5712-dos") ("tcvn-5712-unix") ("tcvn") ("tcvn-mac") ("tcvn-dos") ("tcvn-unix") ("vietnamese-tcvn") ("vietnamese-tcvn-mac") ("vietnamese-tcvn-dos") ("vietnamese-tcvn-unix") ("cp1258") ("cp1258-mac") ("cp1258-dos") ("cp1258-unix") ("windows-1258") ("windows-1258-mac") ("windows-1258-dos") ("windows-1258-unix") ("viqr") ("viqr-mac") ("viqr-dos") ("viqr-unix") ("vietnamese-viqr") ("vietnamese-viqr-mac") ...) nil t nil coding-system-history "raw-text" nil)
  read-coding-system("Select coding system (default raw-text): " raw-text)
  select-safe-coding-system-interactively(1 315 (raw-text no-conversion) (utf-8) nil raw-text)
  select-safe-coding-system(1 315 utf-8 nil "/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S")
  #<subr write-region>(nil nil "/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S" nil t "/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S" nil)
  ad-Advice-write-region(#<subr write-region> nil nil "/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S" nil t "/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S")
  apply(ad-Advice-write-region #<subr write-region> (nil nil "/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S" nil t "/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S"))
  write-region(nil nil "/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S" nil t "/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S")
  basic-save-buffer-2()
  basic-save-buffer-1()
  basic-save-buffer(nil)
  save-buffer()
  write-file("/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S")
  (let ((no-coding-system-for-write (quote raw-text))) (write-file file))
  (lambda (G2 G3 file) (setq message-fcc-handler-function (symbol-value G2)) (let ((no-coding-system-for-write (quote raw-text))) (write-file file)) (mu4e~proc-add file (or (symbol-value G3) "/")))(--old-handler-- --maildir-- "/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S")
  apply((lambda (G2 G3 file) (setq message-fcc-handler-function (symbol-value G2)) (let ((no-coding-system-for-write (quote raw-text))) (write-file file)) (mu4e~proc-add file (or (symbol-value G3) "/"))) --old-handler-- --maildir-- "/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S")
  (lambda (&rest --cl-rest--) (apply (function (lambda (G2 G3 file) (setq message-fcc-handler-function (symbol-value G2)) (let ((no-coding-system-for-write (quote raw-text))) (write-file file)) (mu4e~proc-add file (or (symbol-value G3) "/")))) (quote --old-handler--) (quote --maildir--) --cl-rest--))("/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S")
  message-do-fcc()
  message-send(nil)
  message-send-and-exit(nil)
  funcall-interactively(message-send-and-exit nil)
  call-interactively(message-send-and-exit nil nil)
  command-execute(message-send-and-exit)

Note that in the body of the handler I currently have:

  (let ((no-coding-system-for-write (quote raw-text))) (write-file file))

The no-coding-system-for-write is just for debugging (setting
coding-system-for-write to 'raw-text fixes the issue). I don't expect
the handler to have to set it explicitly though.


In GNU Emacs 26.0.50.4 (x86_64-pc-linux-gnu, X toolkit, Xaw scroll bars)
 of 2017-02-05 built on eab16011nb
Repository revision: 148100d98319499f0ac6f57b8be08cbd14884a5c
Windowing system distributor 'The X.Org Foundation', version 11.0.11901000
System Description:	Debian GNU/Linux 9.0 (stretch)

Recent messages:
Quit
Mark set [2 times]

Configured using:
 'configure --with-x-toolkit=lucid --without-gconf --without-gsettings
 --with-modules 'CFLAGS=-O3 -march=native -pipe '
 LDFLAGS=-fwhole-program'

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND DBUS NOTIFY ACL LIBSELINUX GNUTLS
LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS LUCID X11
MODULES LIBSYSTEMD

Important settings:
  value of $LC_COLLATE: C
  value of $LC_TIME: en_DK.UTF-8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: mu4e-headers

Minor modes in effect:
  hl-line-mode: t
  show-paren-mode: t
  global-undo-tree-mode: t
  undo-tree-mode: t
  window-numbering-mode: t
  shell-dirtrack-mode: t
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
~/.emacs.d/lisp/epa hides /usr/local/share/emacs/26.0.50/lisp/epa

Features:
(shadow whitespace flyspell ispell face-remap emacsbug
fill-column-indicator minibuffer-complete-cycle shr svg dom dbus xml
mu4e desktop frameset mu4e-speedbar speedbar sb-image ezimage dframe
mu4e-main mu4e-context mu4e-view cal-menu calendar cal-loaddefs
mu4e-headers mu4e-compose mu4e-draft mu4e-actions rfc2368 smtpmail
mu4e-mark mu4e-message flow-fill html2text mu4e-proc mu4e-proc-mu
mu4e-utils doc-view jka-compr image-mode mu4e-lists mu4e-vars message
subr-x puny rfc822 mml mml-sec epa epg gnus-util rmail rmail-loaddefs
mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils
mailheader hl-line mu4e-meta num3-mode fic-mode dim
color-theme-solarized-custom term disp-table ehelp solarized-theme
solarized-definitions color-theme wid-edit sendmail rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mail-utils reporter hlinum linum
slime-banner slime-asdf grep slime-fancy slime-trace-dialog
slime-fontifying-fu slime-package-fu slime-references
slime-compiler-notes-tree slime-scratch slime-presentations bridge
slime-mdot-fu slime-enclosing-context slime-fuzzy slime-fancy-trace
slime-fancy-inspector slime-c-p-c slime-editing-commands slime-autodoc
slime-repl slime-parse slime etags xref project arc-mode archive-mode pp
hyperspec rst expand-region text-mode-expansions er-basic-expansions
expand-region-core expand-region-custom dired-x dired dired-loaddefs
modeline-posn indent-guide paren undo-tree diff window-numbering
phi-search-mc multiple-cursors mc-hide-unmatched-lines-mode
mc-separate-operations rectangular-region-mode mc-mark-pop mc-mark-more
mc-cycle-cursors mc-edit-lines multiple-cursors-core rect phi-search
phi-search-core edmacro kmacro url-privacy simple-wiki derived
volatile-highlights finder-inf doc-mode-autoloads goto-chg-autoloads
hide-region-autoloads lemon-mode-autoloads gh-common gh-profile rx s
marshal eieio-compat ht json map dash num3-mode-autoloads
occur-x-autoloads rainbow-blocks-autoloads rainbow-mode-autoloads
tup-mode-autoloads vline-autoloads whole-line-or-region-autoloads info
package epg-config url-handlers url-parse url-vars debian-el
debian-el-loaddefs w3m-load slime-autoloads org-install ido seq
ess-toolbar ess-mouse mouseme thingatpt browse-url ess-menu ess-swv
ess-noweb ess-noweb-font-lock-mode ess-bugs-l essd-els ess-sas-d
ess-sas-l ess-sas-a ess-sta-d ess-sta-l cc-vars cc-defs make-regexp
ess-sp6-d ess-dde ess-sp3-d ess-julia ess-r-d ess-r-syntax
ess-r-completion ess-roxy essddr noutline outline hideshow ess-help
ess-r-package ess-s-l ess ess-inf ess-tracebug compile tramp
tramp-compat tramp-loaddefs trampver ucs-normalize shell pcomplete
comint ansi-color ring parse-time format-spec advice auth-source cl-seq
eieio byte-opt bytecomp byte-compile cl-extra help-mode cconv eieio-core
cl-macs eieio-loaddefs password-cache ess-mode ess-noweb-mode ess-utils
ess-generics cl gv cl-loaddefs pcase cl-lib ess-custom executable
easymenu ess-compat ess-site emacs-goodies-el emacs-goodies-custom
emacs-goodies-loaddefs easy-mmode preview-latex tex-site auto-loads
time-date mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page menu-bar
rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript case-table
epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote dbusbind inotify
dynamic-setting font-render-setting x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 16 513968 28098)
 (symbols 48 44131 1)
 (miscs 40 942 1546)
 (strings 32 98425 12555)
 (string-bytes 1 2968654)
 (vectors 16 60897)
 (vector-slots 8 1022526 10392)
 (floats 8 593 133)
 (intervals 56 761 686)
 (buffers 976 15))