From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: LdBeth Newsgroups: gmane.emacs.devel Subject: Re: [PATCH] Gnus; Restore multi encoding support for NNTP Date: Sat, 01 Jan 2022 11:32:19 +0800 Message-ID: References: <87wnjqb62b.fsf@gnus.org> <87sfueb3y1.fsf@gnus.org> <87wnjm6tbs.fsf@gnus.org> <874k6o7okc.fsf@gnus.org> Mime-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: multipart/mixed; boundary="Multipart_Sat_Jan__1_11:32:19_2022-1" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="18052"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?ISO-8859-4?Q?Goj=F2?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.2 (x86_64-apple-darwin18.7.0) MULE/6.0 (HANACHIRUSATO) Cc: Eric Abrahamsen , LdBeth , Emacs Devel To: Lars Ingebrigtsen Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sat Jan 01 04:33:22 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1n3V90-0004Wv-48 for ged-emacs-devel@m.gmane-mx.org; Sat, 01 Jan 2022 04:33:22 +0100 Original-Received: from localhost ([::1]:39872 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1n3V8y-0005rb-Gw for ged-emacs-devel@m.gmane-mx.org; Fri, 31 Dec 2021 22:33:20 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:52108) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n3V8M-0005D1-Dm for emacs-devel@gnu.org; Fri, 31 Dec 2021 22:32:42 -0500 Original-Received: from out203-205-251-72.mail.qq.com ([203.205.251.72]:46424) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n3V8B-0003SN-U4 for emacs-devel@gnu.org; Fri, 31 Dec 2021 22:32:37 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1641007941; bh=Pghf4V8VG9wbu8SX6lmzWofo4szg4f5p70wfLgQiVGY=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=QL5R3yNNipdy7Hs1iE4oKE8ut1lwuX2qwR61MagD756i8Nj4QDMxwGBnzIGueZWet cabARlwgUtq9uxsFTRz1udGUEPqyOTgNvzzo/Gu9KSA2Od+20DgPTiCMq+0CnkDgVH bQXRMZKh/dxobt8oQ7PB/5rKtxzOTdlKtItc4emg= Original-Received: from Costume-Party.local ([125.111.88.17]) by newxmesmtplogicsvrszc6.qq.com (NewEsmtp) with SMTP id 8140286D; Sat, 01 Jan 2022 11:32:20 +0800 X-QQ-mid: xmsmtpt1641007940tuif1bfpo X-QQ-XMAILINFO: No4GWyI4cWt6CAGBQoo5mTrYFEXuTrvDZ7zYqclaK0Wso+YePEdZO6wdEYPVAI lxTFIQK9St3YlpGaKYAcs+4bQtW6pCzBal630EdoiJyvLbqlS6+TG7tMcluQuVzkQ8I+BG22fkrr q2uPG2sIiX+BHOhI7jvpc/R/un0fU9B4EMxlSjqWW5hjwbBF7YbTqPdH3gijlVoHgbSq0gwp54ws upE+zDDMblR9x2bgKieLyV4kEMfJtxHV2NLpIgIckBTRGE8k2YeKSRmgPh0CZEdQsr9IA8ZdTQS2 mUPEayfwB4MuymU9um/ing8AaZsZ5E0Gao7xEFHoOg56AM39ZDyGm2SpjVw3Q9Fkgl7BxkLiNoM9 mbYhxP/qeuMReBVA1x7DMO+om1ZqauJjOklpzidegoLHi8keySrnQ/Mg0aDtgXrdikSVbU09KCBg oPCXY4EdPdPEDzsuYbChjxQbwDd7zzfsGtIrfhwJl0TrCHFO9wVQMH+Lzyli1+vJtD2mURcMOkwo wXpkmdvPMEjQC4vbnPPiTcSW0HJkf1L2VqGcPEm/Kutf7aw1lRjuyx7ARw4SEtB0UITG44DYWXak 0HObhXuxrlY7lphV/e4MZyLmxbUtV6p21Lkc8wfKjC7z32+ll2e8DefQM0GH5iQfsT5/hgaQGUkp 7ZQBlI7N+UFj4UQqrAP9FsOZoVZURv/0aonvxrmEHABZb2B7/AX+5wY1llqbrv/e5+rTQtIL5VtD /7E9/ekrANGNkcZnYxB4FPX94Q5/El8xDwC1aZP2WKyoNPnwsFyWgh/aMY0phHyPLWA0XDI7Ea/r T7U6B/3jYNqPapeMQsc3L+Ju6e03sxu0niC6qLda Original-Received: by Costume-Party.local (Postfix, from userid 501) id BF834203D67C6D; Sat, 1 Jan 2022 11:32:19 +0800 (CST) X-OQ-MSGID: In-Reply-To: X-Face: %[!P\u/BKFRGn_9h9|yO"ho?C0ej^LmM}WMb-`Jfj8OsS^^AKmHYGlD@^|7SEA3UzOGPFbB"OFczY?'\JtJ\lR'@&Y5j; s8{$&|3D>^i.U4l2h?1qpD.+{[$~j]vBeHZf^|BGyL8{/`4 X-Attribution: ldb Received-SPF: pass client-ip=203.205.251.72; envelope-from=andpuke@foxmail.com; helo=out203-205-251-72.mail.qq.com X-Spam_score_int: 27 X-Spam_score: 2.7 X-Spam_bar: ++ X-Spam_report: (2.7 / 5.0 requ) DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HELO_DYNAMIC_IPADDR=1.951, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, RDNS_DYNAMIC=0.982, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:283777 Archived-At: --Multipart_Sat_Jan__1_11:32:19_2022-1 Content-Type: text/plain; charset=US-ASCII >>>>> In >>>>> LdBeth wrote: ldb> Right now this patch has no problem accessing, subscribing servers ldb> with GBK coding system and save the group names with their text ldb> property (test agains the git master branch). The only one missing ldb> puzzle is, the text property would be lost at some point after read in ldb> the newsrc.eld file. I'll do a trace later to find out if this can be ldb> worked out. I have now removed the "extraneous" decoding rountines when convert the gnus-newsrc-alist to hashtable. I did some test on the server I use and it works fine to me. The minimal .gnus.el I use: ``` (setq gnus-select-method '(nnnil "")) (add-to-list 'gnus-secondary-select-methods '(nntp "news.newsfan.net")) (setq gnus-group-name-charset-group-alist '((".*" . gbk))) ``` Notice that after entering the group there would still be wrongly decoded article names, but that can be solved by setting up `gnus-summary-show-article-charset-alist` `mm-coding-system-priorities` etc. These are not related to this patch, and are quite complex so I'd rather to not cover them here. Btw, happy new year. --Multipart_Sat_Jan__1_11:32:19_2022-1 Content-Type: text/plain; type=patch; name="gnus.patch"; charset=US-ASCII Content-Disposition: attachment; filename="gnus.patch" Content-Transfer-Encoding: 7bit diff --git a/lisp/gnus/gnus-group.el b/lisp/gnus/gnus-group.el index b042930..9db3d11 100644 --- a/lisp/gnus/gnus-group.el +++ b/lisp/gnus/gnus-group.el @@ -1230,6 +1230,12 @@ gnus-group-decoded-name (let ((charset (gnus-group-name-charset nil string))) (gnus-group-name-decode string charset))) +(defun gnus-group-encoded-name (string) + ;; search for `charset' property added by `decode-coding-string' + (let ((pos (text-property-not-all 0 (length string) 'charset nil string))) + (if pos (encode-coding-string string (get-text-property pos 'charset string)) + string))) + (defun gnus-group-list-groups (&optional level unread lowest update-level) "List newsgroups with level LEVEL or lower that have unread articles. Default is all subscribed groups. diff --git a/lisp/gnus/gnus-int.el b/lisp/gnus/gnus-int.el index 255c11f..4fcc44d 100644 --- a/lisp/gnus/gnus-int.el +++ b/lisp/gnus/gnus-int.el @@ -472,7 +472,7 @@ gnus-request-compact-group (result (funcall (gnus-get-function gnus-command-method 'request-compact-group) - (gnus-group-real-name group) + (gnus-group-real-name (gnus-group-encoded-name group)) (nth 1 gnus-command-method) t))) result)) @@ -493,7 +493,8 @@ gnus-request-group (setq gnus-command-method (inline (gnus-server-to-method gnus-command-method)))) (funcall (inline (gnus-get-function gnus-command-method 'request-group)) - (gnus-group-real-name group) (nth 1 gnus-command-method) + (gnus-group-real-name (gnus-group-encoded-name group)) + (nth 1 gnus-command-method) dont-check info))) @@ -503,7 +504,8 @@ gnus-request-group-description (func 'request-group-description)) (when (gnus-check-backend-function func group) (funcall (gnus-get-function gnus-command-method func) - (gnus-group-real-name group) (nth 1 gnus-command-method))))) + (gnus-group-real-name (gnus-group-encoded-name group)) + (nth 1 gnus-command-method))))) (defun gnus-request-group-scan (group info) "Request that GROUP get a complete rescan." @@ -511,13 +513,15 @@ gnus-request-group-scan (func 'request-group-scan)) (when (gnus-check-backend-function func group) (funcall (gnus-get-function gnus-command-method func) - (gnus-group-real-name group) (nth 1 gnus-command-method) info)))) + (gnus-group-real-name (gnus-group-encoded-name group)) + (nth 1 gnus-command-method) info)))) (defun gnus-close-group (group) "Request the GROUP be closed." (let ((gnus-command-method (inline (gnus-find-method-for-group group)))) (funcall (gnus-get-function gnus-command-method 'close-group) - (gnus-group-real-name group) (nth 1 gnus-command-method)))) + (gnus-group-real-name (gnus-group-encoded-name group)) + (nth 1 gnus-command-method)))) (defun gnus-retrieve-headers (articles group &optional fetch-old) "Request headers for ARTICLES in GROUP. @@ -531,14 +535,14 @@ gnus-retrieve-headers (gnus-agent-retrieve-headers articles group fetch-old)) (t (funcall (gnus-get-function gnus-command-method 'retrieve-headers) - articles (gnus-group-real-name group) + articles (gnus-group-real-name (gnus-group-encoded-name group)) (nth 1 gnus-command-method) fetch-old))))) (defun gnus-retrieve-articles (articles group) "Request ARTICLES in GROUP." (let ((gnus-command-method (gnus-find-method-for-group group))) (funcall (gnus-get-function gnus-command-method 'retrieve-articles) - articles (gnus-group-real-name group) + articles (gnus-group-real-name (gnus-group-encoded-name group)) (nth 1 gnus-command-method)))) (defun gnus-retrieve-groups (groups command-method) @@ -557,7 +561,7 @@ gnus-request-type 'request-type (car gnus-command-method))) 'unknown (funcall (gnus-get-function gnus-command-method 'request-type) - (gnus-group-real-name group) article)))) + (gnus-group-real-name (gnus-group-encoded-name group)) article)))) (defun gnus-request-update-group-status (group status) "Change the status of a group. @@ -568,7 +572,7 @@ gnus-request-update-group-status nil (funcall (gnus-get-function gnus-command-method 'request-update-group-status) - (gnus-group-real-name group) status + (gnus-group-real-name (gnus-group-encoded-name group)) status (nth 1 gnus-command-method))))) (defun gnus-request-set-mark (group action) @@ -578,7 +582,7 @@ gnus-request-set-mark 'request-set-mark (car gnus-command-method))) action (funcall (gnus-get-function gnus-command-method 'request-set-mark) - (gnus-group-real-name group) action + (gnus-group-real-name (gnus-group-encoded-name group)) action (nth 1 gnus-command-method)) (gnus-run-hook-with-args gnus-after-set-mark-hook group action)))) @@ -590,7 +594,8 @@ gnus-request-update-mark mark (gnus-run-hook-with-args gnus-before-update-mark-hook group article mark) (funcall (gnus-get-function gnus-command-method 'request-update-mark) - (gnus-group-real-name group) article mark)))) + (gnus-group-real-name (gnus-group-encoded-name group)) + article mark)))) (defun gnus-request-article (article group &optional buffer) "Request the ARTICLE in GROUP. @@ -598,7 +603,7 @@ gnus-request-article If BUFFER, insert the article in that group." (let ((gnus-command-method (gnus-find-method-for-group group))) (funcall (gnus-get-function gnus-command-method 'request-article) - article (gnus-group-real-name group) + article (gnus-group-real-name (gnus-group-encoded-name group)) (nth 1 gnus-command-method) buffer))) (defun gnus-request-thread (header group) @@ -606,7 +611,7 @@ gnus-request-thread (let ((gnus-command-method (gnus-find-method-for-group group))) (funcall (gnus-get-function gnus-command-method 'request-thread) header - (gnus-group-real-name group)))) + (gnus-group-real-name (gnus-group-encoded-name group))))) (defun gnus-select-group-with-message-id (group message-id) "Activate and select GROUP with the given MESSAGE-ID selected. @@ -654,7 +659,7 @@ gnus-simplify-group-name "Return the simplest representation of the name of GROUP. This is the string that Gnus uses to identify the group." (gnus-group-prefixed-name - (gnus-group-real-name group) + (gnus-group-real-name (gnus-group-encoded-name group)) (gnus-group-method group))) (defun gnus-warp-to-article () @@ -722,7 +727,8 @@ gnus-request-body clean-up t)) ;; Use `head' function. ((fboundp head) - (setq res (funcall head article (gnus-group-real-name group) + (setq res (funcall head article + (gnus-group-real-name (gnus-group-encoded-name group)) (nth 1 gnus-command-method)))) ;; Use `article' function. (t @@ -751,7 +757,7 @@ gnus-request-expunge-group (gnus-server-to-method command-method) command-method))) (funcall (gnus-get-function gnus-command-method 'request-expunge-group) - (gnus-group-real-name group) + (gnus-group-real-name (gnus-group-encoded-name group)) (nth 1 gnus-command-method)))) (defvar mail-source-plugged) @@ -768,7 +774,7 @@ gnus-request-scan (not (gnus-agent-method-p gnus-command-method))) (setq gnus-internal-registry-spool-current-method gnus-command-method) (funcall (gnus-get-function gnus-command-method 'request-scan) - (and group (gnus-group-real-name group)) + (and group (gnus-group-real-name (gnus-group-encoded-name group))) (nth 1 gnus-command-method))))) (defun gnus-request-update-info (info command-method) @@ -792,7 +798,7 @@ gnus-request-marks 'request-marks (car gnus-command-method)) (let ((group (gnus-info-group info))) (and (funcall (gnus-get-function gnus-command-method 'request-marks) - (gnus-group-real-name group) + (gnus-group-real-name (gnus-group-encoded-name group)) info (nth 1 gnus-command-method)) ;; If the minimum article number is greater than 1, then all ;; smaller article numbers are known not to exist; we'll @@ -816,7 +822,8 @@ gnus-request-expire-articles (not-deleted (funcall (gnus-get-function gnus-command-method 'request-expire-articles) - articles (gnus-group-real-name group) (nth 1 gnus-command-method) + articles (gnus-group-real-name (gnus-group-encoded-name group)) + (nth 1 gnus-command-method) force))) (when (and gnus-agent (gnus-agent-method-p gnus-command-method)) @@ -830,7 +837,8 @@ gnus-request-move-article (let* ((gnus-command-method (gnus-find-method-for-group group)) (result (funcall (gnus-get-function gnus-command-method 'request-move-article) - article (gnus-group-real-name group) + article + (gnus-group-real-name (gnus-group-encoded-name group)) (nth 1 gnus-command-method) accept-function last move-is-internal))) (when (and result gnus-agent @@ -864,7 +872,9 @@ gnus-request-accept-article (result (funcall (gnus-get-function gnus-command-method 'request-accept-article) - (if (stringp group) (gnus-group-real-name group) group) + (if (stringp group) + (gnus-group-real-name (gnus-group-encoded-name group)) + group) (cadr gnus-command-method) last))) (when (and gnus-agent @@ -883,7 +893,9 @@ gnus-request-replace-article (message-encode-message-body))) (let* ((func (car (gnus-group-name-to-method group))) (result (funcall (intern (format "%s-request-replace-article" func)) - article (gnus-group-real-name group) buffer))) + article + (gnus-group-real-name (gnus-group-encoded-name group)) + buffer))) (when (and gnus-agent (gnus-agent-method-p gnus-command-method)) (gnus-agent-regenerate-group group (list article))) result)) @@ -892,7 +904,7 @@ gnus-request-restore-buffer "Request a new buffer restored to the state of ARTICLE." (let ((gnus-command-method (gnus-find-method-for-group group))) (funcall (gnus-get-function gnus-command-method 'request-restore-buffer) - article (gnus-group-real-name group) + article (gnus-group-real-name (gnus-group-encoded-name group)) (nth 1 gnus-command-method)))) (defun gnus-request-create-group (group &optional command-method args) @@ -902,13 +914,15 @@ gnus-request-create-group command-method) (gnus-find-method-for-group group)))) (funcall (gnus-get-function gnus-command-method 'request-create-group) - (gnus-group-real-name group) (nth 1 gnus-command-method) args))) + (gnus-group-real-name (gnus-group-encoded-name group)) + (nth 1 gnus-command-method) args))) (defun gnus-request-delete-group (group &optional force) (let* ((gnus-command-method (gnus-find-method-for-group group)) (result (funcall (gnus-get-function gnus-command-method 'request-delete-group) - (gnus-group-real-name group) force (nth 1 gnus-command-method)))) + (gnus-group-real-name (gnus-group-encoded-name group)) + force (nth 1 gnus-command-method)))) (when result (gnus-cache-delete-group group) (gnus-agent-delete-group group)) @@ -918,8 +932,9 @@ gnus-request-rename-group (let* ((gnus-command-method (gnus-find-method-for-group group)) (result (funcall (gnus-get-function gnus-command-method 'request-rename-group) - (gnus-group-real-name group) - (gnus-group-real-name new-name) (nth 1 gnus-command-method)))) + (gnus-group-real-name (gnus-group-encoded-name group)) + (gnus-group-real-name (gnus-group-encoded-name new-name)) + (nth 1 gnus-command-method)))) (when result (gnus-cache-rename-group group new-name) (gnus-agent-rename-group group new-name)) diff --git a/lisp/gnus/gnus-srvr.el b/lisp/gnus/gnus-srvr.el index fa880b7..94c6e2e 100644 --- a/lisp/gnus/gnus-srvr.el +++ b/lisp/gnus/gnus-srvr.el @@ -775,13 +775,12 @@ gnus-browse-foreign-server (while (not (eobp)) (ignore-errors (push (cons - (decode-coding-string - (buffer-substring + (gnus-group-decoded-name + (buffer-substring (point) (progn (skip-chars-forward "^ \t") - (point))) - 'utf-8-emacs) + (point)))) (let ((last (read cur))) (cons (read cur) last))) groups)) @@ -789,7 +788,7 @@ gnus-browse-foreign-server (while (not (eobp)) (ignore-errors (push (cons - (decode-coding-string + (gnus-group-decoded-name (if (eq (char-after) ?\") (read cur) (let ((p (point)) (name "")) @@ -801,8 +800,7 @@ gnus-browse-foreign-server (skip-chars-forward "^ \t\\\\") (setq name (concat name (buffer-substring p (point))))) - name)) - 'utf-8-emacs) + name))) (let ((last (read cur))) (cons (read cur) last))) groups)) diff --git a/lisp/gnus/gnus-start.el b/lisp/gnus/gnus-start.el index 606bd3a..2999d6b 100644 --- a/lisp/gnus/gnus-start.el +++ b/lisp/gnus/gnus-start.el @@ -1831,11 +1831,7 @@ gnus-make-hashtable-from-newsrc-alist (if (setq rest (member method methods)) (setf (gnus-info-method info) (car rest)) (push method methods))) - ;; Check for encoded group names and decode them. - (when (string-match-p "[^[:ascii:]]" (setq gname (gnus-info-group info))) - (let ((decoded (gnus-group-decoded-name gname))) - (setf gname decoded - (gnus-info-group info) decoded))) + (setf gname (gnus-info-group info)) ;; Check for duplicates. (if (gethash gname gnus-newsrc-hashtb) ;; Remove this entry from the alist. @@ -2406,17 +2402,6 @@ gnus-read-newsrc-el-file (when gnus-newsrc-assoc (setq gnus-newsrc-alist gnus-newsrc-assoc)))) (gnus-make-hashtable-from-newsrc-alist) - (when gnus-topic-alist - (setq gnus-topic-alist - (mapcar - (lambda (elt) - (cons (car elt) - (mapcar (lambda (g) - (if (string-match-p "[^[:ascii:]]" g) - (gnus-group-decoded-name g) - g)) - (cdr elt)))) - gnus-topic-alist))) (when (file-newer-than-file-p file ding-file) ;; Old format quick file (gnus-message 5 "Reading %s..." file) @@ -2893,26 +2878,6 @@ gnus-gnus-to-quick-newsrc-format ;; Remove the `gnus-killed-list' from the list of variables ;; to be saved, if required. (delq 'gnus-killed-list (copy-sequence gnus-variable-list))))) - ;; Encode group names in `gnus-newsrc-alist' and - ;; `gnus-topic-alist' in order to keep newsrc.eld files - ;; compatible with older versions of Gnus. At some point, - ;; if/when a new version of Gnus is released, stop doing - ;; this and move the corresponding decode in - ;; `gnus-read-newsrc-el-file' into a conversion routine. - (gnus-newsrc-alist - (mapcar (lambda (info) - (cons (encode-coding-string (car info) 'utf-8-emacs) - (cdr info))) - gnus-newsrc-alist)) - (gnus-topic-alist - (when (memq 'gnus-topic-alist variables) - (mapcar (lambda (elt) - (cons (car elt) ; Topic name - (mapcar (lambda (g) - (encode-coding-string - g 'utf-8-emacs)) - (cdr elt)))) - gnus-topic-alist))) variable) ;; Insert the variables into the file. (while variables --Multipart_Sat_Jan__1_11:32:19_2022-1--