bug#62812: 29.0.60; Persistent article cache not working properly

bug#62812: 29.0.60; Persistent article cache not working properly

[James Thomas]

(From an emacs session started with gnus-mock-start)
1. M-: (setq gnus-use-cache t)
2. M-x gnus
3. RET on 'mails'
4. M P b
5. *
6. q
7. ^
8. a nnml RET cache RET
9. e on 'nnml:cache'
10. Paste the following, with /tmp/emacs-gnus-mock-h7X6w6 replaced with
the value of gnus-home-directory:

  (nnml "cache"
        (nnml-directory "/tmp/emacs-gnus-mock-h7X6w6/News/cache/")
        (nnml-active-file
        "/tmp/emacs-gnus-mock-h7X6w6/News/cache/active"))

  C-c C-c
11. RET on 'nnml:cache'
12. RET on 'mails'

"Group nnml+cache:mails couldn’t be activated"

Further, if you do:

13. q
14. g on 'nnml:cache'
15. RET on 'nnml:cache'

There's a ghost group 'nil', but:

16. RET on 'mails'

And it works (but I've sometimes seen some mails missing until I delete
the visible ones and regenerate the server again).

In GNU Emacs 29.0.60 (build 1, x86_64-pc-linux-gnu, GTK+ Version
 3.24.33, cairo version 1.16.0) of 2023-04-01 built on
 user-Inspiron-15-5518
Repository revision: 6419d78fa6f8a7794893da5a8a5d65f75a5a29fa
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101003
System Description: Ubuntu 22.04.2 LTS

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NOTIFY
INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER X11 XDBE XIM XINPUT2 XPM GTK3 ZLIB

Important settings:
  value of $LANG: en_IN
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: iso-latin-1-unix

Major mode: EXWM

Minor modes in effect:
  shell-dirtrack-mode: t
  display-time-mode: t
  desktop-environment-mode: t
  server-mode: t
  recentf-mode: t
  override-global-mode: t
  icomplete-mode: t
  fido-mode: t
  display-battery-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  buffer-read-only: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
/home/user/.emacs.d/elpa/transient-20230315.1520/transient hides /home/user/code/emacs/lisp/transient
/home/user/.emacs.d/elpa/use-package-20230203.2004/use-package-jump hides /home/user/code/emacs/lisp/use-package/use-package-jump
/home/user/.emacs.d/elpa/use-package-20230203.2004/use-package-ensure hides /home/user/code/emacs/lisp/use-package/use-package-ensure
/home/user/.emacs.d/elpa/use-package-20230203.2004/use-package-lint hides /home/user/code/emacs/lisp/use-package/use-package-lint
/home/user/.emacs.d/elpa/use-package-20230203.2004/use-package-core hides /home/user/code/emacs/lisp/use-package/use-package-core
/home/user/.emacs.d/elpa/use-package-20230203.2004/use-package hides /home/user/code/emacs/lisp/use-package/use-package
/home/user/.emacs.d/elpa/use-package-20230203.2004/use-package-delight hides /home/user/code/emacs/lisp/use-package/use-package-delight
/home/user/.emacs.d/elpa/bind-key-20230203.2004/bind-key hides /home/user/code/emacs/lisp/use-package/bind-key
/home/user/.emacs.d/elpa/use-package-20230203.2004/use-package-bind-key hides /home/user/code/emacs/lisp/use-package/use-package-bind-key
/home/user/.emacs.d/elpa/use-package-20230203.2004/use-package-diminish hides /home/user/code/emacs/lisp/use-package/use-package-diminish

Features:
(shadow sort bbdb-message mail-extr emacsbug cl-print gnus-async
gnus-cite gnus-cus gnus-demon gnus-diary nndiary gnus-draft gnus-dup
gnus-fun gnus-html gnus-kill gnus-logic gnus-mh mh-comp mh-scan mh-gnus
mh-e mh-buffers mh-loaddefs gnus-registry registry eieio-base rmailsum
rmail gnus-salt gnus-topic gnus-uu yenc gnus-vm epa-file utf-7
cursor-sensor nndraft nnmh nnnil gnus-agent gnus-srvr gnus-score
score-mode nnvirtual gnus-msg nntp gnus-cache bbdb-gnus bbdb-mua
bbdb-com bbdb bbdb-site auth-source-xoauth2 auth-source-pass smtpmail
cus-edit cus-start lisp-mnt finder-inf gnus-mock pp dired-aux shr-color
color timezone textsec uni-scripts idna-mapping ucs-normalize
uni-confusable textsec-check network-stream url-http url-gw nsm
url-cache url-auth hippie-exp compile warnings files-x arc-mode
archive-mode rng-xsd xsd-regexp rng-cmpct rng-nxml rng-valid nxml-mode
nxml-outln nxml-rap sgml-mode facemenu ox-odt rng-loc rng-uri rng-parse
rng-match rng-dt rng-util rng-pttrn nxml-parse nxml-ns nxml-enc xmltok
nxml-util ox-latex ox-icalendar org-agenda ox-html table ox-ascii
ox-publish ox org-element org-persist org-id org-refile avl-tree
generator oc-basic ol-w3m ol-mhe ol-irc ol-info ol-gnus nnselect
gnus-art mm-uu mml2015 mm-view mml-smime smime gnutls dig gnus-sum
gnus-group gnus-undo gnus-start gnus-dbus gnus-cloud nnimap nnmail
mail-source utf7 nnoo parse-time gnus-spec gnus-int gnus-range message
sendmail yank-media rfc822 mml mml-sec epa derived epg rfc6068
epg-config mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047
rfc2045 ietf-drums mailabbrev gmm-utils mailheader gnus-win ol-eww eww
url-queue shr pixel-fill kinsoku url-file svg dom puny mm-url gnus
nnheader gnus-util text-property-search mail-utils range mm-util
mail-prsvr ol-docview doc-view filenotify image-mode exif ol-bibtex
bibtex iso8601 ol-bbdb org org-macro org-pcomplete org-list org-footnote
org-faces org-entities noutline outline icons ob-dot ob-R ob-python
python treesit ob ob-tangle org-src ob-ref ob-lob ob-table ob-exp
ob-comint ob-emacs-lisp ob-core ob-eval org-cycle org-table ol rx
org-fold org-fold-core org-keys oc org-loaddefs find-func org-version
org-compat org-macs cal-iso help-fns radix-tree thingatpt cal-move
cal-menu calendar cal-loaddefs calc-undo calc-vec calc-aent calc-yank
calccomp calc-arith calc-alg calc-forms calc-ext calc-misc mule-util
magit-base magit-section crm dash compat compat-29 misearch
multi-isearch jka-compr calc-menu calc calc-loaddefs rect calc-macs
shell pcomplete comint ansi-osc ansi-color time time-date denote xdg
dired dired-loaddefs xref project ring edmacro desktop-environment ediff
ediff-merg ediff-mult ediff-wind ediff-diff ediff-help ediff-init
ediff-util exwm-systemtray xcb-systemtray xcb-xembed exwm exwm-input
xcb-keysyms xcb-xkb exwm-manage exwm-floating xcb-cursor xcb-render
exwm-layout exwm-workspace exwm-core xcb-ewmh xcb-icccm xcb xcb-xproto
xcb-types xcb-debug kmacro server cl-extra help-mode advice recentf
tree-widget wid-edit use-package use-package-ensure use-package-delight
use-package-diminish use-package-bind-key bind-key easy-mmode
use-package-core format-spec icomplete battery dbus xml cus-load pcase
gnus-mock-autoloads csv-mode-autoloads org-remark-autoloads
org-mime-autoloads info package browse-url url url-proxy url-privacy
url-expand url-methods url-history url-cookie generate-lisp-file
url-domsuf url-util mailcap url-handlers url-parse auth-source cl-seq
eieio eieio-core cl-macs password-cache json subr-x map byte-opt gv
bytecomp byte-compile url-vars cl-loaddefs cl-lib rmc iso-transl tooltip
cconv eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type
elisp-mode mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd
fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow
isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting cairo
move-toolbar gtk x-toolkit xinput2 x multi-tty make-network-process
emacs)

Memory information:
((conses 16 899163 122424)
 (symbols 48 51409 1)
 (strings 32 258636 19837)
 (string-bytes 1 7730958)
 (vectors 16 124156)
 (vector-slots 8 2366288 78523)
 (floats 8 475 473)
 (intervals 56 8074 13313)
 (buffers 984 38))

bug#62812: [PATCH] Re: bug#62812: 29.0.60; Persistent article cache not working properly

[James Thomas]

Tags: patch

Turned out to be a trivial fix:

diff --git a/lisp/gnus/nnmail.el b/lisp/gnus/nnmail.el
index e8f157392d4..904b564409a 100644
--- a/lisp/gnus/nnmail.el
+++ b/lisp/gnus/nnmail.el
@@ -667,7 +667,9 @@ nnmail-parse-active
 		  (cond ((symbolp group)
 			 (symbol-name group))
 			((numberp group)
-			 (number-to-string group))))
+			 (number-to-string group))
+                        ((stringp group)
+			 group)))
 	    (if (and (numberp (setq max (read buffer)))
 		     (numberp (setq min (read buffer))))
 		(push (list group (cons min max))

--

bug#62812: [PATCH] Re: bug#62812: 29.0.60; Persistent article cache not working properly

[James Thomas]

James Thomas wrote:

> Turned out to be a trivial fix:

Here's a better one:

diff --git a/lisp/gnus/nnmail.el b/lisp/gnus/nnmail.el
index e8f157392d4..5bd68e87d3f 100644
--- a/lisp/gnus/nnmail.el
+++ b/lisp/gnus/nnmail.el
@@ -667,7 +667,9 @@ nnmail-parse-active
 		  (cond ((symbolp group)
 			 (symbol-name group))
 			((numberp group)
-			 (number-to-string group))))
+			 (number-to-string group))
+                        (t
+			 group)))
 	    (if (and (numberp (setq max (read buffer)))
 		     (numberp (setq min (read buffer))))
 		(push (list group (cons min max))

--

bug#62812: [BUMP, PATCH] Re: bug#62812: 29.0.60; Persistent article cache not working properly

[James Thomas]

Bumping up a seemingly trivial patch sent about 4 months ago... :)

James Thomas wrote:

> James Thomas wrote:
>
>> Turned out to be a trivial fix:
>
> Here's a better one:
>
> diff --git a/lisp/gnus/nnmail.el b/lisp/gnus/nnmail.el
> index e8f157392d4..5bd68e87d3f 100644
> --- a/lisp/gnus/nnmail.el
> +++ b/lisp/gnus/nnmail.el
> @@ -667,7 +667,9 @@ nnmail-parse-active
>  		  (cond ((symbolp group)
>  			 (symbol-name group))
>  			((numberp group)
> -			 (number-to-string group))))
> +			 (number-to-string group))
> +                        (t
> +			 group)))
>  	    (if (and (numberp (setq max (read buffer)))
>  		     (numberp (setq min (read buffer))))
>  		(push (list group (cons min max))
>
> --

--

bug#62812: [PATCH] Re: bug#62812: 29.0.60; Persistent article cache not working properly

[James Thomas]

James Thomas wrote:

> Turned out to be a trivial fix:
>
> diff --git a/lisp/gnus/nnmail.el b/lisp/gnus/nnmail.el
> index e8f157392d4..904b564409a 100644
> --- a/lisp/gnus/nnmail.el
> +++ b/lisp/gnus/nnmail.el
> @@ -667,7 +667,9 @@ nnmail-parse-active
>  		  (cond ((symbolp group)
>  			 (symbol-name group))
>  			((numberp group)
> -			 (number-to-string group))))
> +			 (number-to-string group))
> +                        ((stringp group)
> +			 group)))
>  	    (if (and (numberp (setq max (read buffer)))
>  		     (numberp (setq min (read buffer))))
>  		(push (list group (cons min max))

And it'll merely make it the same as this already-existing code
elsewhere:

diff -u --label ./lisp/gnus/gnus-start.el --label \#\<buffer\ gnus-start.el\> ./lisp/gnus/gnus-start.el /tmp/buffer-content-IYvzRp
--- ./lisp/gnus/gnus-start.el
+++ #<buffer gnus-start.el>
@@ -2149,8 +2149,6 @@
 					     (number-to-string group))
 					    ((symbolp group)
 					     (symbol-name group))
-					    ((stringp group)
-					     group)))))
 		     (numberp (setq max (read cur)))
 		     (numberp (setq min (read cur)))
 		     (null (progn

--

bug#62812: [PATCH] Re: bug#62812: 29.0.60; Persistent article cache not working properly

[Eli Zaretskii]

> From: James Thomas <jimjoe@gmx.net>
> Date: Wed, 23 Aug 2023 19:07:51 +0530
> 
> James Thomas wrote:
> 
> > Turned out to be a trivial fix:
> >
> > diff --git a/lisp/gnus/nnmail.el b/lisp/gnus/nnmail.el
> > index e8f157392d4..904b564409a 100644
> > --- a/lisp/gnus/nnmail.el
> > +++ b/lisp/gnus/nnmail.el
> > @@ -667,7 +667,9 @@ nnmail-parse-active
> >  		  (cond ((symbolp group)
> >  			 (symbol-name group))
> >  			((numberp group)
> > -			 (number-to-string group))))
> > +			 (number-to-string group))
> > +                        ((stringp group)
> > +			 group)))
> >  	    (if (and (numberp (setq max (read buffer)))
> >  		     (numberp (setq min (read buffer))))
> >  		(push (list group (cons min max))
> 
> And it'll merely make it the same as this already-existing code
> elsewhere:
> 
> diff -u --label ./lisp/gnus/gnus-start.el --label \#\<buffer\ gnus-start.el\> ./lisp/gnus/gnus-start.el /tmp/buffer-content-IYvzRp
> --- ./lisp/gnus/gnus-start.el
> +++ #<buffer gnus-start.el>
> @@ -2149,8 +2149,6 @@
>  					     (number-to-string group))
>  					    ((symbolp group)
>  					     (symbol-name group))
> -					    ((stringp group)
> -					     group)))))
>  		     (numberp (setq max (read cur)))
>  		     (numberp (setq min (read cur)))
>  		     (null (progn
> 
> --

Eric, Andrew: any comments?

bug#62812: [PATCH] Re: bug#62812: 29.0.60; Persistent article cache not working properly

[Eric Abrahamsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: James Thomas <jimjoe@gmx.net>
>> Date: Wed, 23 Aug 2023 19:07:51 +0530
>> 
>> James Thomas wrote:
>> 
>> > Turned out to be a trivial fix:
>> >
>> > diff --git a/lisp/gnus/nnmail.el b/lisp/gnus/nnmail.el
>> > index e8f157392d4..904b564409a 100644
>> > --- a/lisp/gnus/nnmail.el
>> > +++ b/lisp/gnus/nnmail.el
>> > @@ -667,7 +667,9 @@ nnmail-parse-active
>> >  		  (cond ((symbolp group)
>> >  			 (symbol-name group))
>> >  			((numberp group)
>> > -			 (number-to-string group))))
>> > +			 (number-to-string group))
>> > +                        ((stringp group)
>> > +			 group)))
>> >  	    (if (and (numberp (setq max (read buffer)))
>> >  		     (numberp (setq min (read buffer))))
>> >  		(push (list group (cons min max))
>> 
>> And it'll merely make it the same as this already-existing code
>> elsewhere:
>> 
>> diff -u --label ./lisp/gnus/gnus-start.el --label \#\<buffer\ gnus-start.el\> ./lisp/gnus/gnus-start.el /tmp/buffer-content-IYvzRp
>> --- ./lisp/gnus/gnus-start.el
>> +++ #<buffer gnus-start.el>
>> @@ -2149,8 +2149,6 @@
>>  					     (number-to-string group))
>>  					    ((symbolp group)
>>  					     (symbol-name group))
>> -					    ((stringp group)
>> -					     group)))))
>>  		     (numberp (setq max (read cur)))
>>  		     (numberp (setq min (read cur)))
>>  		     (null (progn
>> 
>> --
>
> Eric, Andrew: any comments?

Yes! This looks like a correct fix to me.

bug#62812: [PATCH] Re: bug#62812: 29.0.60; Persistent article cache not working properly

[Eli Zaretskii]

> From: Eric Abrahamsen <eric@ericabrahamsen.net>
> Cc: James Thomas <jimjoe@gmx.net>,  Andrew G Cohen <cohen@andy.bu.edu>,
>   62812@debbugs.gnu.org
> Date: Wed, 23 Aug 2023 09:47:17 -0700
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: James Thomas <jimjoe@gmx.net>
> >> Date: Wed, 23 Aug 2023 19:07:51 +0530
> >> 
> >> James Thomas wrote:
> >> 
> >> > Turned out to be a trivial fix:
> >> >
> >> > diff --git a/lisp/gnus/nnmail.el b/lisp/gnus/nnmail.el
> >> > index e8f157392d4..904b564409a 100644
> >> > --- a/lisp/gnus/nnmail.el
> >> > +++ b/lisp/gnus/nnmail.el
> >> > @@ -667,7 +667,9 @@ nnmail-parse-active
> >> >  		  (cond ((symbolp group)
> >> >  			 (symbol-name group))
> >> >  			((numberp group)
> >> > -			 (number-to-string group))))
> >> > +			 (number-to-string group))
> >> > +                        ((stringp group)
> >> > +			 group)))
> >> >  	    (if (and (numberp (setq max (read buffer)))
> >> >  		     (numberp (setq min (read buffer))))
> >> >  		(push (list group (cons min max))
> >> 
> >> And it'll merely make it the same as this already-existing code
> >> elsewhere:
> >> 
> >> diff -u --label ./lisp/gnus/gnus-start.el --label \#\<buffer\ gnus-start.el\> ./lisp/gnus/gnus-start.el /tmp/buffer-content-IYvzRp
> >> --- ./lisp/gnus/gnus-start.el
> >> +++ #<buffer gnus-start.el>
> >> @@ -2149,8 +2149,6 @@
> >>  					     (number-to-string group))
> >>  					    ((symbolp group)
> >>  					     (symbol-name group))
> >> -					    ((stringp group)
> >> -					     group)))))
> >>  		     (numberp (setq max (read cur)))
> >>  		     (numberp (setq min (read cur)))
> >>  		     (null (progn
> >> 
> >> --
> >
> > Eric, Andrew: any comments?
> 
> Yes! This looks like a correct fix to me.

OK, but now I'm confused regarding which change(s) to install.  This
discussion includes 3 changes, so which ones are proposed and
approved?  James, would you please post a single consolidated patch
you think should be installed, with the appropriate commit log
message?

Thanks.

bug#62812: [PATCH] Re: bug#62812: 29.0.60; Persistent article cache not working properly

[James Thomas]

[-- Attachment #1: Type: text/plain, Size: 327 bytes --]

Eli Zaretskii wrote:

> OK, but now I'm confused regarding which change(s) to install.  This
> discussion includes 3 changes, so which ones are proposed and
> approved?  James, would you please post a single consolidated patch
> you think should be installed, with the appropriate commit log
> message?
>
> Thanks.

Attached:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Account-for-string-names-in-active-file.patch --]
[-- Type: text/x-diff, Size: 1016 bytes --]

From f26b0b938f902eb7daffb34abb64567e289b7685 Mon Sep 17 00:00:00 2001
From: James Thomas <jimjoe@gmx.net>
Date: Thu, 24 Aug 2023 12:35:51 +0530
Subject: [PATCH] Account for string names in active file

Account also for strings when reading in group names from an active
file (bug#62812).
* lisp/gnus/nnmail.el (nnmail-parse-active): Make it similar to
gnus-active-to-gnus-format
---
 lisp/gnus/nnmail.el | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/lisp/gnus/nnmail.el b/lisp/gnus/nnmail.el
index e8f157392d4..904b564409a 100644
--- a/lisp/gnus/nnmail.el
+++ b/lisp/gnus/nnmail.el
@@ -667,7 +667,9 @@ nnmail-parse-active
 		  (cond ((symbolp group)
 			 (symbol-name group))
 			((numberp group)
-			 (number-to-string group))))
+			 (number-to-string group))
+                        ((stringp group)
+			 group)))
 	    (if (and (numberp (setq max (read buffer)))
 		     (numberp (setq min (read buffer))))
 		(push (list group (cons min max))
--
2.34.1


[-- Attachment #3: Type: text/plain, Size: 4 bytes --]


--

bug#62812: [PATCH] Re: bug#62812: 29.0.60; Persistent article cache not working properly

[Eli Zaretskii]

> From: James Thomas <jimjoe@gmx.net>
> Cc: Eric Abrahamsen <eric@ericabrahamsen.net>,  cohen@andy.bu.edu,
>   62812@debbugs.gnu.org
> Date: Thu, 24 Aug 2023 12:59:42 +0530
> 
> Eli Zaretskii wrote:
> 
> > OK, but now I'm confused regarding which change(s) to install.  This
> > discussion includes 3 changes, so which ones are proposed and
> > approved?  James, would you please post a single consolidated patch
> > you think should be installed, with the appropriate commit log
> > message?
> >
> > Thanks.
> 
> Attached:

Thanks, installed on the emacs-29 branch, and closing the bug.