From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Eric Abrahamsen Newsgroups: gmane.emacs.bugs Subject: bug#35383: 27.0.50; Complete process of decoding Gnus group names Date: Tue, 23 Apr 2019 14:42:23 -0700 Message-ID: <87h8ao760w.fsf@ericabrahamsen.net> References: <878sw1995f.fsf@ericabrahamsen.net> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="217404"; mail-complaints-to="usenet@blaine.gmane.org" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) To: 35383@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Tue Apr 23 23:43:12 2019 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1hJ3C4-000uSH-9x for geb-bug-gnu-emacs@m.gmane.org; Tue, 23 Apr 2019 23:43:12 +0200 Original-Received: from localhost ([127.0.0.1]:59896 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hJ3C3-0003qj-BQ for geb-bug-gnu-emacs@m.gmane.org; Tue, 23 Apr 2019 17:43:11 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:32861) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hJ3Bx-0003qV-26 for bug-gnu-emacs@gnu.org; Tue, 23 Apr 2019 17:43:06 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hJ3Bu-0004GV-Ne for bug-gnu-emacs@gnu.org; Tue, 23 Apr 2019 17:43:04 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:40966) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hJ3Bu-0004Fq-Ec for bug-gnu-emacs@gnu.org; Tue, 23 Apr 2019 17:43:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hJ3Bu-0006qx-2s for bug-gnu-emacs@gnu.org; Tue, 23 Apr 2019 17:43:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eric Abrahamsen Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 23 Apr 2019 21:43:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 35383 X-GNU-PR-Package: emacs Original-Received: via spool by 35383-submit@debbugs.gnu.org id=B35383.155605575426310 (code B ref 35383); Tue, 23 Apr 2019 21:43:02 +0000 Original-Received: (at 35383) by debbugs.gnu.org; 23 Apr 2019 21:42:34 +0000 Original-Received: from localhost ([127.0.0.1]:54510 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hJ3BR-0006qI-Po for submit@debbugs.gnu.org; Tue, 23 Apr 2019 17:42:34 -0400 Original-Received: from ericabrahamsen.net ([52.70.2.18]:56948 helo=mail.ericabrahamsen.net) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hJ3BP-0006q1-Cf for 35383@debbugs.gnu.org; Tue, 23 Apr 2019 17:42:32 -0400 Original-Received: from localhost (75-172-115-9.tukw.qwest.net [75.172.115.9]) (Authenticated sender: eric@ericabrahamsen.net) by mail.ericabrahamsen.net (Postfix) with ESMTPSA id 8845BFA05C for <35383@debbugs.gnu.org>; Tue, 23 Apr 2019 21:42:24 +0000 (UTC) In-Reply-To: (Andy Moreton's message of "Tue, 23 Apr 2019 16:42:05 +0100") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:158156 Archived-At: --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Andy Moreton writes: > On Mon 22 Apr 2019, Eric Abrahamsen wrote: > >> This is part two and the completion of bug#33653, which changed Gnus's >> obarrays into hash tables, and group names from symbols to (encoded) >> strings. The commits in the recently-pushed "scratch/gnus-decoded" >> branch change group names to decoded strings. >> >> Bug 33653 was a mess for a few reasons: I partitioned the changes >> poorly, didn't call for testers, and it turned out that I was locally >> testing a mismash of that change plus a couple changes included in this >> branch, which hid some bugs. >> >> This time around I'll keep it cleaner: I'll locally test only this >> change in isolation, I'm writing a semi-interactive test suite for Gnus, >> and in a few weeks I'll rope in three or four users to road-test the >> changes. The upside is that, once these changes are stabilized and put >> in, it will eliminate a whole class of potential bugs. >> >> In the meantime I'm hanging this here as a placeholder. If any brave >> soul does decide to give it a test-run in the meantime, back up your >> .newsrc.eld file first! > > There don;t seem to bnbe any problems with the .nersrc.eld, as I have > used the same file in emacs-26, master and scratch/gnus-decoded branches > without obvious problems. > > There is a problem in the server buffer in the scratch/gnus-decoded buffer > with group names. In my setup: > 1) Start gnus, and open the server buffer > 2) Select the feedbase server. This is configured as: > > (gnus-secondary-select-methods > '((nntp "feedbase" > (nntp-open-connection-function nntp-open-tls-stream) > (nntp-port-number 563) > (nntp-address "feedbase.org")) > ... > > 3) Note that some group names are not decoded correctly in the server > buffer. The group names are decoded properly in the article buffer. > > For example, (showing only names with problems): > > server buffer (master branch): > > K 140: feedbase.blog.k=C3=B8benhavnsskiklub > K 208: feedbase.news.brk.h=C3=B8ringer > K 2017: feedbase.news.dr.hovedstadsomr=C3=A5det > > > server buffer (scratch/gnus-decoded branch): > > K 140: feedbase.blog.k\303\270benhavnsskiklub > K 208: feedbase.news.brk.h\303\270ringer > K 2017: feedbase.news.dr.hovedstadsomr\303\245det > > The raw bytes (shown as escapes above) match the UTF-8 for the expected > characters, so the decoding is not quite right. Aha! Good catch. The attached patch fixes things for me. At first I was a little hesitant to assume all group names are decodable as 'utf-8-emacs, but then I figured that's all that `gnus-group-decoded-name' does, and that's worked for years, so I guess this is safe. And thanks for the nntp server with non-ascii group names! I had looked for one of those with no success, this will be very useful for testing. Eric --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=gnus-browse-srv-decoded.diff diff --git a/lisp/gnus/gnus-srvr.el b/lisp/gnus/gnus-srvr.el index 2328f1da16..da110c8bbf 100644 --- a/lisp/gnus/gnus-srvr.el +++ b/lisp/gnus/gnus-srvr.el @@ -784,11 +784,13 @@ gnus-browse-foreign-server (while (not (eobp)) (ignore-errors (push (cons - (buffer-substring - (point) - (progn - (skip-chars-forward "^ \t") - (point))) + (decode-coding-string + (buffer-substring + (point) + (progn + (skip-chars-forward "^ \t") + (point))) + 'utf-8-emacs) (let ((last (read cur))) (cons (read cur) last))) groups)) @@ -796,18 +798,20 @@ gnus-browse-foreign-server (while (not (eobp)) (ignore-errors (push (cons - (if (eq (char-after) ?\") - (read cur) - (let ((p (point)) (name "")) - (skip-chars-forward "^ \t\\\\") - (setq name (buffer-substring p (point))) - (while (eq (char-after) ?\\) - (setq p (1+ (point))) - (forward-char 2) - (skip-chars-forward "^ \t\\\\") - (setq name (concat name (buffer-substring - p (point))))) - name)) + (decode-coding-string + (if (eq (char-after) ?\") + (read cur) + (let ((p (point)) (name "")) + (skip-chars-forward "^ \t\\\\") + (setq name (buffer-substring p (point))) + (while (eq (char-after) ?\\) + (setq p (1+ (point))) + (forward-char 2) + (skip-chars-forward "^ \t\\\\") + (setq name (concat name (buffer-substring + p (point))))) + name)) + 'utf-8-emacs) (let ((last (read cur))) (cons (read cur) last))) groups)) --=-=-=--