all messages for Emacs-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
From: Stefan Monnier <monnier@IRO.UMontreal.CA>
To: Simon Josefsson <jas@extundo.com>
Cc: 1174@emacsbugs.donarmstrong.com,
	Frank Schmitt <ich@frank-schmitt.net>,
	ding@gnus.org
Subject: bug#1174: 23.0.60; Some UTF-8 mails displaying wrongly in Emacs 23
Date: Mon, 01 Dec 2008 16:04:01 -0500	[thread overview]
Message-ID: <jwv7i6j393g.fsf-monnier+emacsbugreports__32418.3910423473$1228166127$gmane$org@gnu.org> (raw)
In-Reply-To: <87k5ampwej.fsf@marauder.physik.uni-ulm.de> (Reiner Steib's message of "Sat, 29 Nov 2008 13:08:20 +0100")

> Simon, could you please clarify why you reverted Stefan's change in
> `nnimap-demule'?  It breaks reading UTF-8 articles via nnimap.el in
> Emacs 23.

Having looked at the code again, I'm more than ever confident that
string-to-unibyte is the right thing to use.  Maybe the code I installed
back then failed to fallback to string-as-unibyte when string-to-unibyte
was not available, which caused a bug for Simon?

In any case the newly committed code has a prenthesis typo that makes
it still use the old code and ignore the new config var
nnimap-demule-use-string-to-multibyte.

Also I recommend to just use the patch below instead.  The first hunk
removes an unnecessary use of nnimap-demule since the output will be
inserted into a unibyte buffer.


        Stefan


--- nnimap.el.~1.50.~	2008-12-01 15:38:55.000000000 -0500
+++ nnimap.el	2008-12-01 15:49:53.000000000 -0500
@@ -608,12 +608,11 @@
       (with-current-buffer nnimap-server-buffer
 	(setq uid imap-current-message
 	      mbx imap-current-mailbox
-	      headers (nnimap-demule
-		       (if (imap-capability 'IMAP4rev1)
+	      headers (if (imap-capability 'IMAP4rev1)
 			   ;; xxx don't just use car? alist doesn't contain
 			   ;; anything else now, but it might...
 			   (nth 2 (car (imap-message-get uid 'BODYDETAIL)))
-			 (imap-message-get uid 'RFC822.HEADER)))
+                        (imap-message-get uid 'RFC822.HEADER))
 	      lines (imap-body-lines (imap-message-body imap-current-message))
 	      chars (imap-message-get imap-current-message 'RFC822.SIZE)))
       (nnheader-insert-nov
@@ -901,40 +900,17 @@
   (when (nnimap-possibly-change-server server)
     (nnoo-status-message 'nnimap server)))
 
-(defvar nnimap-demule-use-string-to-multibyte (fboundp 'string-to-multibyte)
-  "Temporary internal debug variable.
-If you have problems (UTF-8 not decoded correctly on IMAP) with
-the default value, please report it as a bug!")
-;; FIXME: Clarify if we need to make this variable conditional on the Emacs
-;; version (Emacs 22 vs. Emacs 23;Emacs 21 doesn't have `string-to-multibyte'
-;; anyhow).  --rsteib
-;;
-;; http://thread.gmane.org/gmane.emacs.gnus.general/67112
-;; (bug#464, reported by James Cloos)
-;; http://thread.gmane.org/gmane.emacs.bugs/21524
-;; (bug#1174, reported by Frank Schmitt)
-
-(defun nnimap-demule (string)
-  ;; BEWARE: we used to use string-as-multibyte here which is braindead
-  ;; because it will turn accidental emacs-mule-valid byte sequences
-  ;; into multibyte chars.  --Stef
-  ;; Reverted, braindead got 7.5 out of 10 on imdb, so it can't be
-  ;; that bad. --Simon
-  (gnus-message 9 "nnimap-demule-use-string-to-multibyte: %s"
-		nnimap-demule-use-string-to-multibyte)
-  (if nnimap-demule-use-string-to-multibyte
-      ;; Stefan
-      (funcall (if (and (fboundp 'string-to-multibyte)
-			(subrp (symbol-function 'string-to-multibyte)))
-		   'string-to-multibyte
-		 'identity)
-	       (or string "")))
-  ;; Simon
-  (funcall (if (and (fboundp 'string-as-multibyte)
-		    (subrp (symbol-function 'string-as-multibyte)))
-	       'string-as-multibyte
-	     'identity)
-	   (or string "")))
+;; We used to use a string-as-multibyte here, but it is really incorrect.
+;; This function is used when we're about to insert a unibyte string
+;; into a potentially multibyte buffer.  The string is either an article
+;; header or body (or both?), undecoded.  When Emacs is asked to convert
+;; a unibyte string to multibyte, it may either use the equivalent of
+;; nothing (e.g. non-Mule XEmacs), string-make-unibyte (i.e. decode using
+;; locale), string-as-multibyte (decode using emacs-internal coding system)
+;; or string-to-multibyte (keep the data undecoded as a sequence of bytes).
+;; Only the last one preserves the data such that we can reliably later on
+;; decode the text using the mime info.
+(defalias 'nnimap-demule 'mm-string-to-multibyte)
 
 (defun nnimap-make-callback (article gnus-callback buffer)
   "Return a callback function."






  parent reply	other threads:[~2008-12-01 21:04 UTC|newest]

Thread overview: 39+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <jwviqpz1u4o.fsf-monnier+emacsbugreports@gnu.org>
2008-10-15 20:21 ` bug#1174: 23.0.60; Some UTF-8 mails displaying wrongly in Emacs 23 Frank Schmitt
2008-10-16  7:32   ` Andreas Schwab
2008-10-16  8:36     ` Frank Schmitt
2008-10-16 17:39       ` Eli Zaretskii
2008-10-16 18:00         ` Frank Schmitt
2008-10-16 18:15           ` Eli Zaretskii
2008-10-16 20:05             ` Frank Schmitt
2008-10-17  8:28         ` Andreas Schwab
2008-10-17  8:52           ` Frank Schmitt
2008-10-17  9:53             ` Eli Zaretskii
2008-10-17 10:02               ` Frank Schmitt
2008-10-17 10:08               ` Frank Schmitt
2008-10-17 15:39                 ` Stefan Monnier
2008-10-17 18:19                   ` Reiner Steib
     [not found]                   ` <871vyf6q40.fsf@marauder.physik.uni-ulm.de>
2008-10-17 18:36                     ` Frank Schmitt
     [not found]                     ` <m3ej2fdq5x.fsf@mid.gehheimdienst.de>
2008-11-29 12:08                       ` Reiner Steib
     [not found]                       ` <87k5ampwej.fsf@marauder.physik.uni-ulm.de>
2008-11-29 12:18                         ` Simon Josefsson
2008-12-01 21:04                         ` Stefan Monnier [this message]
     [not found]                         ` <jwv7i6j393g.fsf-monnier+emacsbugreports@gnu.org>
2008-12-01 22:48                           ` Reiner Steib
     [not found]                           ` <87y6yzbjgf.fsf@marauder.physik.uni-ulm.de>
2008-12-02  7:36                             ` Stefan Monnier
     [not found]                             ` <jwv3ah7av77.fsf-monnier+emacsbugreports@gnu.org>
2008-12-04 19:43                               ` Reiner Steib
     [not found]                               ` <87iqpz4tg2.fsf@marauder.physik.uni-ulm.de>
2008-12-04 21:43                                 ` Frank Schmitt
     [not found]                         ` <878wr2ohde.fsf@mocca.josefsson.org>
2008-11-29 15:30                           ` Reiner Steib
     [not found]                           ` <87d4geim7p.fsf@marauder.physik.uni-ulm.de>
2008-11-29 21:30                             ` Stefan Monnier
2008-11-29 22:14                             ` James Cloos
     [not found]                             ` <jwvfxlaz0ov.fsf-monnier+emacsbugreports@gnu.org>
2008-11-30 13:12                               ` Reiner Steib
2009-01-12 10:54                           ` Simon Josefsson
2009-01-12 11:03                             ` Frank Schmitt
     [not found]                             ` <m3d4es3hxu.fsf@mid.gehheimdienst.de>
2009-01-12 11:10                               ` Simon Josefsson
     [not found]                               ` <873afoixtr.fsf@mocca.josefsson.org>
     [not found]                                 ` <x6ab9u2kxr@gzp>
2009-01-14 20:12                                   ` Reiner Steib
     [not found]                                   ` <877i4xlk97.fsf@marauder.physik.uni-ulm.de>
2009-01-14 20:20                                     ` Processed: " Emacs bug Tracking System
2009-01-14 22:08                                     ` Stefan Monnier
2008-10-23  7:24                   ` Frank Schmitt
2008-10-23 13:58                     ` Stefan Monnier
2008-10-17  9:12           ` Frank Schmitt
2008-10-17  5:15       ` Richard M. Stallman
2008-10-28 22:54       ` James Cloos
2008-12-04 22:05   ` bug#1174: marked as done (23.0.60; Some UTF-8 mails displaying wrongly in Emacs 23) Emacs bug Tracking System
2009-02-18  8:55   ` Emacs bug Tracking System

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='jwv7i6j393g.fsf-monnier+emacsbugreports__32418.3910423473$1228166127$gmane$org@gnu.org' \
    --to=monnier@iro.umontreal.ca \
    --cc=1174@emacsbugs.donarmstrong.com \
    --cc=ding@gnus.org \
    --cc=ich@frank-schmitt.net \
    --cc=jas@extundo.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/emacs.git
	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.