From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.devel,gmane.emacs.gnus.general Subject: Re: Why multibyte for original-article buffer Date: Thu, 28 Feb 2008 23:22:09 -0500 Message-ID: References: NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: ger.gmane.org 1204258947 19157 80.91.229.12 (29 Feb 2008 04:22:27 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Fri, 29 Feb 2008 04:22:27 +0000 (UTC) Cc: bugs@gnus.org, ding@gnus.org, emacs-devel@gnu.org To: Katsumi Yamaoka Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Fri Feb 29 05:22:53 2008 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1JUwln-0004EY-PU for ged-emacs-devel@m.gmane.org; Fri, 29 Feb 2008 05:22:52 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1JUwlH-0002ev-9Z for ged-emacs-devel@m.gmane.org; Thu, 28 Feb 2008 23:22:19 -0500 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1JUwlB-0002d9-QJ for emacs-devel@gnu.org; Thu, 28 Feb 2008 23:22:13 -0500 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1JUwlA-0002bx-CV for emacs-devel@gnu.org; Thu, 28 Feb 2008 23:22:13 -0500 Original-Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1JUwlA-0002bg-4e for emacs-devel@gnu.org; Thu, 28 Feb 2008 23:22:12 -0500 Original-Received: from ironport2-out.pppoe.ca ([206.248.154.182]) by monty-python.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1JUwl9-0006Kc-Qc for emacs-devel@gnu.org; Thu, 28 Feb 2008 23:22:11 -0500 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ah4FANQWx0dMCpz5Wmdsb2JhbACQZwEgnQ6BAQ X-IronPort-AV: E=Sophos;i="4.25,424,1199682000"; d="scan'208";a="15381104" Original-Received: from smtp.pppoe.ca ([65.39.196.238]) by ironport2-out.pppoe.ca with ESMTP; 28 Feb 2008 23:22:10 -0500 Original-Received: from pastel.home ([76.10.156.249]) by smtp.pppoe.ca (Internet Mail Server v1.0) with ESMTP id JGZ81910; Thu, 28 Feb 2008 23:22:10 -0500 Original-Received: by pastel.home (Postfix, from userid 20848) id A2CB57FA7; Thu, 28 Feb 2008 23:22:09 -0500 (EST) In-Reply-To: (Katsumi Yamaoka's message of "Fri, 29 Feb 2008 09:35:30 +0900") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.60 (gnu/linux) X-detected-kernel: by monty-python.gnu.org: Genre and OS details not recognized. X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:90824 gmane.emacs.gnus.general:66403 Archived-At: >> I've been running with the following patch with good results. >> Also the patch makes sense to me: the original article is a sequence of >> bytes (i.e. encoded chars), so a unibyte buffer makes a lot of sense >> (it's more efficient and is likely to hide fewer bugs). > So, it is better to make also the " *nntpd*" buffer, the process > buffer for pop3, etc. be unibyte. If such changes cause a problem, > it will be due to a code that copies data from those buffers to > a multibyte buffer and then decodes the data. Actually, making > the original article buffer be unibyte prevents me from reading > a Japanese 8bit shift_jis message like this. > =E6=97=A5=E6=9C=AC=E8=AA=9E > (Try `g' in the summary buffer.) Indeed, I see that. The patch below will fix it. >> Yet the code currently explicitly sets the buffer to multibyte mode. >> Does anybody know why? > IIRC, it was done about ten years ago. I guess it was probably > a workaround for a problem like the one mentioned above. Yes, the Gnus code is full of such things. I suggest it's time to take a step back and think about how things *should* work. Then change the parts for which we know how they should work. This will introduce bugs, but then we'll know that the bugs are elsewhere and that fixing them will get us closer to a good solution. Stefan @@ -6564,7 +6564,13 @@ (with-current-buffer gnus-original-article-buffer (and (equal (car gnus-original-article) group) (eq (cdr gnus-original-article) article)))) - (insert-buffer-substring gnus-original-article-buffer) + ;; `insert-buffer-substring' would incorrectly use the + ;; equivalent of string-make-multibyte which amounts to decodi= ng + ;; with locale-coding-system, causing failure of + ;; subsequent decoding. + (insert (mm-string-to-multibyte + (with-current-buffer gnus-original-article-buffer + (buffer-substring (point-min) (point-max))))) 'article) ;; Check the backlog. ((and gnus-keep-backlog