bug#25645: 26.0.50; message.el: Fcc+handler results in write coding issues in emacs 26

bug#25645: 26.0.50; message.el: Fcc+handler results in write coding issues in emacs 26

[Yuri D'Elia]

I'm using message.el in conjunction with mu4e, which uses Fcc and a
message-fcc-handler-function to save the outgoing message directly into
a maildir folder. Nothing fancy: Fcc is set via a message-send-hook,
then in the handler we just 'write-file and some bookeeping.

It works as expected with emacs 25.2, but fails on 26 when the body
contains unicode characters. Somehow when the mail is encoded in quoted
printable with an utf-8 charset, the buffer seem to contain raw bytes as
if the buffer has an incorrect buffer-file-coding-system.

When 'write-file is issued, emacs prompts for a coding system, which is
unexpected.

The stack is as follows:

  read-from-minibuffer("Select coding system (default raw-text): " nil (keymap (10 . minibuffer-complete-and-exit) (13 . minibuffer-complete-and-exit) keymap (menu-bar keymap (minibuf "Minibuf" keymap (tab menu-item "Complete" minibuffer-complete :help "Complete as far as possible") (space menu-item "Complete Word" minibuffer-complete-word :help "Complete at most one word") (63 menu-item "List Completions" minibuffer-completion-help :help "Display all possible completions") "Minibuf")) (27 keymap (118 . switch-to-completions)) (prior . switch-to-completions) (63 . minibuffer-completion-help) (32 . minibuffer-complete-word) (9 . minibuffer-complete) keymap (menu-bar keymap (minibuf "Minibuf" keymap (previous menu-item "Previous History Item" previous-history-element :help "Put previous minibuffer history element in the minibuffer") (next menu-item "Next History Item" next-history-element :help "Put next minibuffer history element in the minibuffer") (isearch-backward menu-item "Isearch History Backward" isearch-backward :help "Incrementally search minibuffer history backward") (isearch-forward menu-item "Isearch History Forward" isearch-forward :help "Incrementally search minibuffer history forward") (return menu-item "Enter" exit-minibuffer :key-sequence "
" :help "Terminate input and exit minibuffer") (quit menu-item "Quit" abort-recursive-edit :help "Abort input and exit minibuffer") "Minibuf")) (10 . exit-minibuffer) (13 . exit-minibuffer) (7 . abort-recursive-edit) (C-tab . file-cache-minibuffer-complete) (9 . self-insert-command) (XF86Back . previous-history-element) (up . previous-line-or-history-element) (prior . previous-history-element) (XF86Forward . next-history-element) (down . next-line-or-history-element) (next . next-history-element) (27 keymap (114 . previous-matching-history-element) (115 . next-matching-history-element) (112 . previous-history-element) (110 . next-history-element))) nil coding-system-history "raw-text" nil)
  completing-read-default("Select coding system (default raw-text): " (("utf-8-hfs") ("utf-8-hfs-mac") ("utf-8-hfs-dos") ("utf-8-hfs-unix") ("georgian-academy") ("georgian-academy-mac") ("georgian-academy-dos") ("georgian-academy-unix") ("georgian-ps") ("georgian-ps-mac") ("georgian-ps-dos") ("georgian-ps-unix") ("cp1256") ("cp1256-mac") ("cp1256-dos") ("cp1256-unix") ("windows-1256") ("windows-1256-mac") ("windows-1256-dos") ("windows-1256-unix") ("iso-8859-6") ("iso-8859-6-mac") ("iso-8859-6-dos") ("iso-8859-6-unix") ("tcvn-5712") ("tcvn-5712-mac") ("tcvn-5712-dos") ("tcvn-5712-unix") ("tcvn") ("tcvn-mac") ("tcvn-dos") ("tcvn-unix") ("vietnamese-tcvn") ("vietnamese-tcvn-mac") ("vietnamese-tcvn-dos") ("vietnamese-tcvn-unix") ("cp1258") ("cp1258-mac") ("cp1258-dos") ("cp1258-unix") ("windows-1258") ("windows-1258-mac") ("windows-1258-dos") ("windows-1258-unix") ("viqr") ("viqr-mac") ("viqr-dos") ("viqr-unix") ("vietnamese-viqr") ("vietnamese-viqr-mac") ...) nil t nil coding-system-history "raw-text" nil)
  read-coding-system("Select coding system (default raw-text): " raw-text)
  select-safe-coding-system-interactively(1 315 (raw-text no-conversion) (utf-8) nil raw-text)
  select-safe-coding-system(1 315 utf-8 nil "/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S")
  #<subr write-region>(nil nil "/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S" nil t "/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S" nil)
  ad-Advice-write-region(#<subr write-region> nil nil "/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S" nil t "/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S")
  apply(ad-Advice-write-region #<subr write-region> (nil nil "/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S" nil t "/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S"))
  write-region(nil nil "/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S" nil t "/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S")
  basic-save-buffer-2()
  basic-save-buffer-1()
  basic-save-buffer(nil)
  save-buffer()
  write-file("/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S")
  (let ((no-coding-system-for-write (quote raw-text))) (write-file file))
  (lambda (G2 G3 file) (setq message-fcc-handler-function (symbol-value G2)) (let ((no-coding-system-for-write (quote raw-text))) (write-file file)) (mu4e~proc-add file (or (symbol-value G3) "/")))(--old-handler-- --maildir-- "/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S")
  apply((lambda (G2 G3 file) (setq message-fcc-handler-function (symbol-value G2)) (let ((no-coding-system-for-write (quote raw-text))) (write-file file)) (mu4e~proc-add file (or (symbol-value G3) "/"))) --old-handler-- --maildir-- "/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S")
  (lambda (&rest --cl-rest--) (apply (function (lambda (G2 G3 file) (setq message-fcc-handler-function (symbol-value G2)) (let ((no-coding-system-for-write (quote raw-text))) (write-file file)) (mu4e~proc-add file (or (symbol-value G3) "/")))) (quote --old-handler--) (quote --maildir--) --cl-rest--))("/tmp/test.maildir/cur/1486485941.759962ca005e4922.eab16011nb:2,S")
  message-do-fcc()
  message-send(nil)
  message-send-and-exit(nil)
  funcall-interactively(message-send-and-exit nil)
  call-interactively(message-send-and-exit nil nil)
  command-execute(message-send-and-exit)

Note that in the body of the handler I currently have:

  (let ((no-coding-system-for-write (quote raw-text))) (write-file file))

The no-coding-system-for-write is just for debugging (setting
coding-system-for-write to 'raw-text fixes the issue). I don't expect
the handler to have to set it explicitly though.


In GNU Emacs 26.0.50.4 (x86_64-pc-linux-gnu, X toolkit, Xaw scroll bars)
 of 2017-02-05 built on eab16011nb
Repository revision: 148100d98319499f0ac6f57b8be08cbd14884a5c
Windowing system distributor 'The X.Org Foundation', version 11.0.11901000
System Description:	Debian GNU/Linux 9.0 (stretch)

Recent messages:
Quit
Mark set [2 times]

Configured using:
 'configure --with-x-toolkit=lucid --without-gconf --without-gsettings
 --with-modules 'CFLAGS=-O3 -march=native -pipe '
 LDFLAGS=-fwhole-program'

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND DBUS NOTIFY ACL LIBSELINUX GNUTLS
LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS LUCID X11
MODULES LIBSYSTEMD

Important settings:
  value of $LC_COLLATE: C
  value of $LC_TIME: en_DK.UTF-8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: mu4e-headers

Minor modes in effect:
  hl-line-mode: t
  show-paren-mode: t
  global-undo-tree-mode: t
  undo-tree-mode: t
  window-numbering-mode: t
  shell-dirtrack-mode: t
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
~/.emacs.d/lisp/epa hides /usr/local/share/emacs/26.0.50/lisp/epa

Features:
(shadow whitespace flyspell ispell face-remap emacsbug
fill-column-indicator minibuffer-complete-cycle shr svg dom dbus xml
mu4e desktop frameset mu4e-speedbar speedbar sb-image ezimage dframe
mu4e-main mu4e-context mu4e-view cal-menu calendar cal-loaddefs
mu4e-headers mu4e-compose mu4e-draft mu4e-actions rfc2368 smtpmail
mu4e-mark mu4e-message flow-fill html2text mu4e-proc mu4e-proc-mu
mu4e-utils doc-view jka-compr image-mode mu4e-lists mu4e-vars message
subr-x puny rfc822 mml mml-sec epa epg gnus-util rmail rmail-loaddefs
mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils
mailheader hl-line mu4e-meta num3-mode fic-mode dim
color-theme-solarized-custom term disp-table ehelp solarized-theme
solarized-definitions color-theme wid-edit sendmail rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mail-utils reporter hlinum linum
slime-banner slime-asdf grep slime-fancy slime-trace-dialog
slime-fontifying-fu slime-package-fu slime-references
slime-compiler-notes-tree slime-scratch slime-presentations bridge
slime-mdot-fu slime-enclosing-context slime-fuzzy slime-fancy-trace
slime-fancy-inspector slime-c-p-c slime-editing-commands slime-autodoc
slime-repl slime-parse slime etags xref project arc-mode archive-mode pp
hyperspec rst expand-region text-mode-expansions er-basic-expansions
expand-region-core expand-region-custom dired-x dired dired-loaddefs
modeline-posn indent-guide paren undo-tree diff window-numbering
phi-search-mc multiple-cursors mc-hide-unmatched-lines-mode
mc-separate-operations rectangular-region-mode mc-mark-pop mc-mark-more
mc-cycle-cursors mc-edit-lines multiple-cursors-core rect phi-search
phi-search-core edmacro kmacro url-privacy simple-wiki derived
volatile-highlights finder-inf doc-mode-autoloads goto-chg-autoloads
hide-region-autoloads lemon-mode-autoloads gh-common gh-profile rx s
marshal eieio-compat ht json map dash num3-mode-autoloads
occur-x-autoloads rainbow-blocks-autoloads rainbow-mode-autoloads
tup-mode-autoloads vline-autoloads whole-line-or-region-autoloads info
package epg-config url-handlers url-parse url-vars debian-el
debian-el-loaddefs w3m-load slime-autoloads org-install ido seq
ess-toolbar ess-mouse mouseme thingatpt browse-url ess-menu ess-swv
ess-noweb ess-noweb-font-lock-mode ess-bugs-l essd-els ess-sas-d
ess-sas-l ess-sas-a ess-sta-d ess-sta-l cc-vars cc-defs make-regexp
ess-sp6-d ess-dde ess-sp3-d ess-julia ess-r-d ess-r-syntax
ess-r-completion ess-roxy essddr noutline outline hideshow ess-help
ess-r-package ess-s-l ess ess-inf ess-tracebug compile tramp
tramp-compat tramp-loaddefs trampver ucs-normalize shell pcomplete
comint ansi-color ring parse-time format-spec advice auth-source cl-seq
eieio byte-opt bytecomp byte-compile cl-extra help-mode cconv eieio-core
cl-macs eieio-loaddefs password-cache ess-mode ess-noweb-mode ess-utils
ess-generics cl gv cl-loaddefs pcase cl-lib ess-custom executable
easymenu ess-compat ess-site emacs-goodies-el emacs-goodies-custom
emacs-goodies-loaddefs easy-mmode preview-latex tex-site auto-loads
time-date mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page menu-bar
rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript case-table
epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote dbusbind inotify
dynamic-setting font-render-setting x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 16 513968 28098)
 (symbols 48 44131 1)
 (miscs 40 942 1546)
 (strings 32 98425 12555)
 (string-bytes 1 2968654)
 (vectors 16 60897)
 (vector-slots 8 1022526 10392)
 (floats 8 593 133)
 (intervals 56 761 686)
 (buffers 976 15))

bug#25645: 26.0.50; message.el: Fcc+handler results in write coding issues in emacs 26

[Eli Zaretskii]

> From: Yuri D'Elia <wavexx@thregr.org>
> Date: Tue, 07 Feb 2017 18:00:01 +0100
> 
> Somehow when the mail is encoded in quoted printable with an utf-8
> charset, the buffer seem to contain raw bytes as if the buffer has
> an incorrect buffer-file-coding-system.

Buffer contents is entirely unrelated to the value of
buffer-file-coding-system.  The latter only determines how to encode
the buffer contents when writing it to a file or sending it to another
process or network connection.

If you get raw bytes in the buffer, the culprit is in the code which
inserted those bytes, or read them from some file.

> When 'write-file is issued, emacs prompts for a coding system, which is
> unexpected.

This is expected when there are raw bytes in the buffer.  The problem
is elsewhere.

bug#25645: 26.0.50; message.el: Fcc+handler results in write coding issues in emacs 26

[Yuri D'Elia]

On Tue, Feb 07 2017, Eli Zaretskii wrote:
> If you get raw bytes in the buffer, the culprit is in the code which
> inserted those bytes, or read them from some file.

Yes, the content of the buffer is generated by
message-encode-message-body in message.el itself.

You only need to insert an unicode letter in the message's buffer to
trigger the issue when recoding to qp-utf8.

I wrote to Lars prior to filing this report (cc-ed).

> This is expected when there are raw bytes in the buffer.  The problem
> is elsewhere.

Since the fcc-handler is expected to do something with the buffer (write
it out), I expect message.el to preset the buffer coding system if the
buffer becomes unibyte in nature.

bug#25645: 26.0.50; message.el: Fcc+handler results in write coding issues in emacs 26

[Eli Zaretskii]

> From: Yuri D'Elia <wavexx@thregr.org>
> Date: Tue, 07 Feb 2017 18:31:15 +0100
> Cc: 25645@debbugs.gnu.org, Lars Ingebrigtsen <larsi@gnus.org>
> 
> On Tue, Feb 07 2017, Eli Zaretskii wrote:
> > If you get raw bytes in the buffer, the culprit is in the code which
> > inserted those bytes, or read them from some file.
> 
> Yes, the content of the buffer is generated by
> message-encode-message-body in message.el itself.
> 
> You only need to insert an unicode letter in the message's buffer to
> trigger the issue when recoding to qp-utf8.
> 
> I wrote to Lars prior to filing this report (cc-ed).

Lars, any news on this issue?

Thanks.

bug#25645: 26.0.50; message.el: Fcc+handler results in write coding issues in emacs 26

[Lars Ingebrigtsen]

Yuri D'Elia <wavexx@thregr.org> writes:

> It works as expected with emacs 25.2, but fails on 26 when the body
> contains unicode characters. Somehow when the mail is encoded in quoted
> printable with an utf-8 charset, the buffer seem to contain raw bytes as
> if the buffer has an incorrect buffer-file-coding-system.

I'm unable to reproduce this bug.  If I put

Fcc: /tmp/fcc

in my headers and enter text like "föo" in the body, the file is saved
as it should be.

Could the problem be in your handler?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#25645: 26.0.50; message.el: Fcc+handler results in write coding issues in emacs 26

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Lars, any news on this issue?

I'm unable to reproduce the error, and I don't think it should block the
release, since it doesn't seem to affect many people.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#25645: 26.0.50; message.el: Fcc+handler results in write coding issues in emacs 26

[Yuri D'Elia]

On Wed, Sep 13 2017, Lars Ingebrigtsen wrote:
>> It works as expected with emacs 25.2, but fails on 26 when the body
>> contains unicode characters. Somehow when the mail is encoded in quoted
>> printable with an utf-8 charset, the buffer seem to contain raw bytes as
>> if the buffer has an incorrect buffer-file-coding-system.
>
> I'm unable to reproduce this bug.  If I put
>
> Fcc: /tmp/fcc
>
> in my headers and enter text like "föo" in the body, the file is saved
> as it should be.

Adding an Fcc is not enough, if you read the initial report, the issue
is triggered by attempting to call write-file within a
message-fcc-handler-function.

bug#25645: 26.0.50; message.el: Fcc+handler results in write coding issues in emacs 26

[Lars Ingebrigtsen]

Yuri D'Elia <wavexx@thregr.org> writes:

> Adding an Fcc is not enough, if you read the initial report, the issue
> is triggered by attempting to call write-file within a
> message-fcc-handler-function.

Do you have a minimal recipe that demonstrates the bug?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#25645: 26.0.50; message.el: Fcc+handler results in write coding issues in emacs 26

[Yuri D'Elia]

On Wed, Sep 13 2017, Lars Ingebrigtsen wrote:
> Yuri D'Elia <wavexx@thregr.org> writes:
>
>> Adding an Fcc is not enough, if you read the initial report, the issue
>> is triggered by attempting to call write-file within a
>> message-fcc-handler-function.
>
> Do you have a minimal recipe that demonstrates the bug?

My minimal recipe still involves using mu4e at the moment.

I can try to make a self-contained test case, but I might need a couple
of days.

bug#25645: 26.0.50; message.el: Fcc+handler results in write coding issues in emacs 26

[Alexis]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> Lars, any news on this issue?
>
> I'm unable to reproduce the error, and I don't think it should 
> block
> the release, since it doesn't seem to affect many people.

In addition to myself, at least several other mu4e users seem to 
have
been affected by this:

https://groups.google.com/forum/#!topic/mu-discuss/dIj3W_R-M4A

https://groups.google.com/forum/#!topic/mu-discuss/NJgRONE7Xuk

https://groups.google.com/forum/#!topic/mu-discuss/MaYcS5uRcsM

https://groups.google.com/forum/#!topic/mu-discuss/RC1Scq2wCe4

and maybe:

https://groups.google.com/forum/#!topic/mu-discuss/dxfpdcidtoc

i've not yet been able to consistently reproduce it, so haven't 
yet been
able to narrow down whether the issue is with mu4e or with 
Emacs. (Maybe
something has changed in Emacs that mu4e needs to become aware 
of). i'll
try to work on this as soon as i can, unless Yuri is able to 
produce an
MRE first. :-)


Alexis.

bug#25645: 26.0.50; message.el: Fcc+handler results in write coding issues in emacs 26

[Yuri D'Elia]

On Fri, Sep 22 2017, Alexis wrote:
> i've not yet been able to consistently reproduce it, so haven't yet been
> able to narrow down whether the issue is with mu4e or with Emacs. (Maybe
> something has changed in Emacs that mu4e needs to become aware of). i'll
> try to work on this as soon as i can, unless Yuri is able to produce an
> MRE first. :-)

I didn't find the time yet. I wanted to make a minimal test case using
only message-mode.

bug#25645: reproducer?

[Alex Bennée]

Does this count as a reproducer?

(defun bug-25645-reproducer ()
  "Reproducer."
  (let ((buf (get-buffer-create "*bug25645*")))
    (switch-to-buffer buf)
    (message-mode)
    (goto-char (point-max))
    (insert "\né")
    (with-temp-buffer
	(insert-buffer-substring buf)
	(message-clone-locals buf)
	(message-encode-message-body)
        (get-buffer-create "*bug2564-copy*")
        (copy-to-buffer "*bug2564-copy*" (point-min) (point-max)))))

Certainly if I comment out the (message-clone-locals) in message-do-fcc
behaviour reverts to pre-emacs26.

--
Alex Bennée

bug#25645: reproducer?

[Eli Zaretskii]

[Please try to avoid changing the Subject line, as it makes it harder
for me finding messages relevant to a single discussion.]

> From: Alex Bennée <alex.bennee@linaro.org>
> Date: Tue, 03 Oct 2017 15:27:12 +0100
> 
> 
> Does this count as a reproducer?

It does, but since you have already established that
message-clone-locals is the culprit, the next question is: what is the
exact list of local variables that gets cloned here?  Can you post the
full list?

Thanks.

bug#25645: reproducer?

[Alex Bennée]

Eli Zaretskii <eliz@gnu.org> writes:

> [Please try to avoid changing the Subject line, as it makes it harder
> for me finding messages relevant to a single discussion.]

Ahh sorry I thought debbugs would munge an appropriate subject when it
received the email.

>
>> From: Alex Bennée <alex.bennee@linaro.org>
>> Date: Tue, 03 Oct 2017 15:27:12 +0100
>>
>>
>> Does this count as a reproducer?
>
> It does, but since you have already established that
> message-clone-locals is the culprit, the next question is: what is the
> exact list of local variables that gets cloned here?  Can you post the
> full list?

Sure. From my message log, it seems we clone locals for that buffer at
least twice:

Saving file /home/alex/Maildir/drafts/cur/1507076739.4e7e0c7397b19db2.zen:2,DS...
Wrote /home/alex/Maildir/drafts/cur/1507076739.4e7e0c7397b19db2.zen:2,DS
[mu4e] Saved (10 lines)
Sending...
message-clone-locals: One more run with dump of copy
copying: message-reply-buffer -> nil
copying: message-inserted-headers -> nil
copying: message-send-actions -> nil
copying: message-return-action -> nil
copying: message-exit-actions -> nil
copying: message-kill-actions -> nil
copying: message-postpone-actions -> nil
copying: message-draft-article -> nil
copying: message-reply-headers -> nil
copying: message-newsreader -> nil
copying: message-mailer -> nil
copying: message-post-method -> (nnspool )
copying: message-sent-message-via -> nil
copying: message-checksum -> nil
copying: message-mime-part -> 0
copying: message-signature -> my-sig-function
copying: message-send-mail-real-function -> nil
copying: message-default-charset -> utf-8
copying: message-send-hook -> (#[nil \b\302=\203	.\303 \210	\203.

bug#25645: reproducer?

[Eli Zaretskii]

> From: Alex Bennée <alex.bennee@linaro.org>
> Cc: 25645@debbugs.gnu.org
> Date: Wed, 04 Oct 2017 01:31:39 +0100
> 
> copying: message-default-charset -> utf-8

A stab in the dark: what if you filter out message-default-charset
from the cloned locals?

bug#25645: reproducer?

[Alex Bennée]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Alex Bennée <alex.bennee@linaro.org>
>> Cc: 25645@debbugs.gnu.org
>> Date: Wed, 04 Oct 2017 01:31:39 +0100
>>
>> copying: message-default-charset -> utf-8
>
> A stab in the dark: what if you filter out message-default-charset
> from the cloned locals?

Good guess:

(defun message-clone-locals (buffer &optional varstr)
  "Clone the local variables from BUFFER to the current buffer."
  (message "message-clone-locals: %s" buffer)
  (let ((locals (with-current-buffer buffer (buffer-local-variables)))
	(regexp "^gnus\\|^nn\\|^message\\|^sendmail\\|^smtp\\|^user-mail-address"))
    (mapcar
     (lambda (local)
       (when (and (consp local)
		  (car local)
		  (string-match regexp (symbol-name (car local)))
		  (or (null varstr)
		      (string-match varstr (symbol-name (car local)))))
	 (ignore-errors
           (unless (string-match "message-default-charset" (symbol-name (car local)))
             (message "copying: %s -> %s" (car local) (cdr local))
	     (set (make-local-variable (car local))
		  (cdr local))))))
     locals)))

Although I feel a slightly neater solution is needed.

So is the problem we are attempting to double encode an already encoded
buffer or something else?

--
Alex Bennée

bug#25645: reproducer?

[Eli Zaretskii]

> From: Alex Bennée <alex.bennee@linaro.org>
> Cc: 25645@debbugs.gnu.org
> Date: Wed, 04 Oct 2017 09:46:17 +0100
> 
> > A stab in the dark: what if you filter out message-default-charset
> > from the cloned locals?
> 
> Good guess:
> 
> (defun message-clone-locals (buffer &optional varstr)
>   "Clone the local variables from BUFFER to the current buffer."
>   (message "message-clone-locals: %s" buffer)
>   (let ((locals (with-current-buffer buffer (buffer-local-variables)))
> 	(regexp "^gnus\\|^nn\\|^message\\|^sendmail\\|^smtp\\|^user-mail-address"))
>     (mapcar
>      (lambda (local)
>        (when (and (consp local)
> 		  (car local)
> 		  (string-match regexp (symbol-name (car local)))
> 		  (or (null varstr)
> 		      (string-match varstr (symbol-name (car local)))))
> 	 (ignore-errors
>            (unless (string-match "message-default-charset" (symbol-name (car local)))
>              (message "copying: %s -> %s" (car local) (cdr local))
> 	     (set (make-local-variable (car local))
> 		  (cdr local))))))
>      locals)))
> 
> Although I feel a slightly neater solution is needed.

Hmm... what is non-neat about this one?  The existing filtering
already includes quite a bit of ad-hoc'ery.

> So is the problem we are attempting to double encode an already encoded
> buffer or something else?

Something like that.  I could look deeper, if needed, but frankly the
thick forest of Gnus encoding-related stuff is not something I like to
wander into.  It's easy to break things there because effects are not
localized.

Thanks.

bug#25645: reproducer?

[Eli Zaretskii]

> From: Alex Bennée <alex.bennee@linaro.org>
> Cc: 25645@debbugs.gnu.org
> Date: Wed, 04 Oct 2017 09:46:17 +0100
> 
> > A stab in the dark: what if you filter out message-default-charset
> > from the cloned locals?
> 
> Good guess:
> 
> (defun message-clone-locals (buffer &optional varstr)
>   "Clone the local variables from BUFFER to the current buffer."
>   (message "message-clone-locals: %s" buffer)
>   (let ((locals (with-current-buffer buffer (buffer-local-variables)))
> 	(regexp "^gnus\\|^nn\\|^message\\|^sendmail\\|^smtp\\|^user-mail-address"))
>     (mapcar
>      (lambda (local)
>        (when (and (consp local)
> 		  (car local)
> 		  (string-match regexp (symbol-name (car local)))
> 		  (or (null varstr)
> 		      (string-match varstr (symbol-name (car local)))))
> 	 (ignore-errors
>            (unless (string-match "message-default-charset" (symbol-name (car local)))
>              (message "copying: %s -> %s" (car local) (cdr local))
> 	     (set (make-local-variable (car local))
> 		  (cdr local))))))
>      locals)))

Thanks, I installed this, and I'm marking the bug done.