bug#72949: [PATCH] Wrong unread count in the Group buffer in nnimap

bug#72949: [PATCH] Wrong unread count in the Group buffer in nnimap

[James Thomas via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 122 bytes --]

I haven't tested this patch (don't really know how to), but this could
very well be the cause of this long-standing bug:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: [PATCH] Correct empty mailbox active count --]
[-- Type: text/x-patch, Size: 865 bytes --]

From 6676798c2a60094ddb8a7476bbbf611d9e7d644c Mon Sep 17 00:00:00 2001
From: James Thomas <jimjoe@gmx.net>
Date: Mon, 2 Sep 2024 11:23:11 +0530
Subject: [PATCH] Correct empty mailbox active count

* lisp/gnus/nnimap.el (nnimap-request-group): Make the change.
---
 lisp/gnus/nnimap.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/gnus/nnimap.el b/lisp/gnus/nnimap.el
index 7b0e42ff89d..8e4ad45c62c 100644
--- a/lisp/gnus/nnimap.el
+++ b/lisp/gnus/nnimap.el
@@ -918,7 +918,7 @@ nnimap-request-group
 	    (nnimap-finish-retrieve-group-infos server info sequences
 						t)
 	    (setq active (nth 2 (assoc group nnimap-current-infos)))))
-	(setq active (or active '(0 . 1)))
+	(setq active (or active '(1 . 0)))
 	(erase-buffer)
 	(insert (format "211 %d %d %d %S\n"
 			(- (cdr active) (car active))
--
2.43.0


[-- Attachment #3: Type: text/plain, Size: 16 bytes --]


Regards,
James

bug#72949: [PATCH] Wrong unread count in the Group buffer in nnimap

[James Thomas via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 215 bytes --]

James Thomas wrote:

> I haven't tested this patch (don't really know how to), but this could
> very well be the cause of this long-standing bug:

Sorry, that was an earlier attempt. This is the one I'm using now:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: [PATCH] Correct empty mailbox active count --]
[-- Type: text/x-patch, Size: 725 bytes --]

From a2c19066746d49f5ed1f61d6da670ab0b72c7341 Mon Sep 17 00:00:00 2001
From: James Thomas <jimjoe@gmx.net>
Date: Mon, 2 Sep 2024 11:35:38 +0530
Subject: [PATCH] Correct empty mailbox active count

* lisp/gnus/nnimap.el (nnimap-request-group): Make the change.
---
 lisp/gnus/nnimap.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/gnus/nnimap.el b/lisp/gnus/nnimap.el
index 7b0e42ff89d..9de5b14d21b 100644
--- a/lisp/gnus/nnimap.el
+++ b/lisp/gnus/nnimap.el
@@ -922,8 +922,8 @@ nnimap-request-group
 	(erase-buffer)
 	(insert (format "211 %d %d %d %S\n"
 			(- (cdr active) (car active))
-			(car active)
 			(cdr active)
+			(car active)
 			group))
 	t))))

--
2.43.0


[-- Attachment #3: Type: text/plain, Size: 16 bytes --]


Regards,
James

bug#72949: [PATCH] Wrong unread count in the Group buffer in nnimap

[Eli Zaretskii]

> Date: Mon, 02 Sep 2024 11:52:30 +0530
> From:  James Thomas via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> James Thomas wrote:
> 
> > I haven't tested this patch (don't really know how to), but this could
> > very well be the cause of this long-standing bug:
> 
> Sorry, that was an earlier attempt. This is the one I'm using now:
> 
> From a2c19066746d49f5ed1f61d6da670ab0b72c7341 Mon Sep 17 00:00:00 2001
> From: James Thomas <jimjoe@gmx.net>
> Date: Mon, 2 Sep 2024 11:35:38 +0530
> Subject: [PATCH] Correct empty mailbox active count
> 
> * lisp/gnus/nnimap.el (nnimap-request-group): Make the change.
> ---
>  lisp/gnus/nnimap.el | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/lisp/gnus/nnimap.el b/lisp/gnus/nnimap.el
> index 7b0e42ff89d..9de5b14d21b 100644
> --- a/lisp/gnus/nnimap.el
> +++ b/lisp/gnus/nnimap.el
> @@ -922,8 +922,8 @@ nnimap-request-group
>  	(erase-buffer)
>  	(insert (format "211 %d %d %d %S\n"
>  			(- (cdr active) (car active))
> -			(car active)
>  			(cdr active)
> +			(car active)
>  			group))
>  	t))))
> 
> --
> 2.43.0

Thanks.  Eric, any comments?

bug#72949: [PATCH] Wrong unread count in the Group buffer in nnimap

[Eli Zaretskii]

Ping!  Eric, any comments?

> Cc: 72949@debbugs.gnu.org
> Date: Sat, 14 Sep 2024 10:41:16 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> 
> > Date: Mon, 02 Sep 2024 11:52:30 +0530
> > From:  James Thomas via "Bug reports for GNU Emacs,
> >  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> > 
> > James Thomas wrote:
> > 
> > > I haven't tested this patch (don't really know how to), but this could
> > > very well be the cause of this long-standing bug:
> > 
> > Sorry, that was an earlier attempt. This is the one I'm using now:
> > 
> > From a2c19066746d49f5ed1f61d6da670ab0b72c7341 Mon Sep 17 00:00:00 2001
> > From: James Thomas <jimjoe@gmx.net>
> > Date: Mon, 2 Sep 2024 11:35:38 +0530
> > Subject: [PATCH] Correct empty mailbox active count
> > 
> > * lisp/gnus/nnimap.el (nnimap-request-group): Make the change.
> > ---
> >  lisp/gnus/nnimap.el | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/lisp/gnus/nnimap.el b/lisp/gnus/nnimap.el
> > index 7b0e42ff89d..9de5b14d21b 100644
> > --- a/lisp/gnus/nnimap.el
> > +++ b/lisp/gnus/nnimap.el
> > @@ -922,8 +922,8 @@ nnimap-request-group
> >  	(erase-buffer)
> >  	(insert (format "211 %d %d %d %S\n"
> >  			(- (cdr active) (car active))
> > -			(car active)
> >  			(cdr active)
> > +			(car active)
> >  			group))
> >  	t))))
> > 
> > --
> > 2.43.0
> 
> Thanks.  Eric, any comments?
> 
> 
> 
> 

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[James Thomas via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Hi,

Xiyue Deng wrote:

> Hi,
>
> i'm having some issues that Gnus sometimes reports that there are new
> messages on an IMAP server but it doesn't show them.  In my case it's
> using Gmail IMAP, and if I check using other mail clients
> (e.g. Thunderbird) or directly check on Gmail website I can seem them.
> I'm not sure whether this issue is specific to IMAP either.  As I'm not
> sure how to get more info, I didn't use report bug yet, but would like
> to ask around and see whether this is a known issue, or otherwise I can
> get some suggestions to get more debug info before I formally file a
> bug.
>
> Any suggestions are welcome!  TIA!

James Thomas wrote:

> I haven't tested this patch (don't really know how to), but this could
> very well be the cause of this long-standing bug:

Could you check if this patch (the 1st one, which swaps the 0 and 1),
makes any difference?

Regards,
James

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[Dan Christensen]

On Sep 11, 2024, James Thomas via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org> wrote:

> James Thomas wrote:
>
>> I haven't tested this patch (don't really know how to), but this could
>> very well be the cause of this long-standing bug:
>
> Could you check if this patch (the 1st one, which swaps the 0 and 1),
> makes any difference?

I've noticed a long-standing bug, and tried this patch.  Here's the bug:
If I use `B m' or `B c' to move or copy a message from an nnimap group
to a non-existent nnimap group, Gnus asks if I want to create the group.
I say yes, and then it gets created with some faulty parameters that
make the *Group* buffer show odd numbers:

  ("nnimap+rocky:testmove1" 3
   ((1 . 1))
   ((unexist 0) (seen 1)) "nnimap:rocky"
   ((modseq . "3") (uidvalidity . "1549221829") (active 0 . 1) (permanent-flags %Answered %Flagged %Deleted %Seen %Draft %*)))))

Note that article 0 is mentioned twice there.  If I close Gnus, get
rid of the "(unexist 0)", and change the active range to 1 . 1, then
things are good again.

I can consistently reproduce this.

I wondered if the patch in this thread would help, but after applying
it, I'm unable to create the new group in this way.  After hitting `y'
to agree to creating a new group, I get:

Debugger entered--Lisp error: (error "Couldn’t create group nnimap+rocky:testmove4")
  signal(error ("Couldn’t create group nnimap+rocky:testmove4"))
  error("Couldn't create group %s" "nnimap+rocky:testmove4")
  gnus-read-move-group-name("Copy" "nnimap+rocky:testmove1" (50086) "nn:")
  gnus-summary-move-article(nil nil nil copy)
  gnus-summary-copy-article(nil)
  funcall-interactively(gnus-summary-copy-article nil)
  call-interactively(gnus-summary-copy-article nil nil)
  command-execute(gnus-summary-copy-article)

So this patch makes Gnus worse for me, rather than better.  I hope this
reproducible strategy helps pin down a solution.

Dan

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[James Thomas via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Dan Christensen wrote:

> On Sep 11, 2024, James Thomas via "Bug reports for GNU Emacs, the
> Swiss army knife of text editors" <bug-gnu-emacs@gnu.org> wrote:
>
>> James Thomas wrote:
>>
>>> I haven't tested this patch (don't really know how to), but this could
>>> very well be the cause of this long-standing bug:
>>
>> Could you check if this patch (the 1st one, which swaps the 0 and 1),
>> makes any difference?
>
> I've noticed a long-standing bug, and tried this patch.  Here's the bug:
> If I use `B m' or `B c' to move or copy a message from an nnimap group
> to a non-existent nnimap group, Gnus asks if I want to create the group.
> I say yes, and then it gets created with some faulty parameters that
> make the *Group* buffer show odd numbers:
>
>   ("nnimap+rocky:testmove1" 3
>    ((1 . 1))
>    ((unexist 0) (seen 1)) "nnimap:rocky"
>    ((modseq . "3") (uidvalidity . "1549221829") (active 0 . 1) (permanent-flags %Answered %Flagged %Deleted %Seen %Draft %*)))))
>
> Note that article 0 is mentioned twice there.  If I close Gnus, get
> rid of the "(unexist 0)", and change the active range to 1 . 1, then
> things are good again.
>
> I can consistently reproduce this.
>
> I wondered if the patch in this thread would help, but after applying
> it, I'm unable to create the new group in this way.  After hitting `y'
> to agree to creating a new group, I get:
>
> Debugger entered--Lisp error: (error "Couldn’t create group nnimap+rocky:testmove4")
>   signal(error ("Couldn’t create group nnimap+rocky:testmove4"))
>   error("Couldn't create group %s" "nnimap+rocky:testmove4")
>   gnus-read-move-group-name("Copy" "nnimap+rocky:testmove1" (50086) "nn:")
>   gnus-summary-move-article(nil nil nil copy)
>   gnus-summary-copy-article(nil)
>   funcall-interactively(gnus-summary-copy-article nil)
>   call-interactively(gnus-summary-copy-article nil nil)
>   command-execute(gnus-summary-copy-article)
>
> So this patch makes Gnus worse for me, rather than better.  I hope this
> reproducible strategy helps pin down a solution.

Sorry! I got confused again... Please try the other patch in that bug
report (the one that swaps the car and cdr). I find that it works with
your use case!

--

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[Dan Christensen]

[-- Attachment #1: Type: text/plain, Size: 1136 bytes --]

On Sep 12, 2024, James Thomas <jimjoe@gmx.net> wrote:

>> Sorry! I got confused again... Please try the other patch in that bug
>> report (the one that swaps the car and cdr). I find that it works with
>> your use case!

Yes, that patch fixes the problem I've had for ages.  But I don't think
that it's correct.  Here's what is done in nnml.el:

	  (nnheader-insert "211 %d %d %d %s\n"
			   (max (1+ (- (cdr active) (car active))) 0)
			   (car active) (cdr active) group)))))))

Notice that the car comes before the cdr, and that you have to add 1
to the difference between the cdr and the car, since if the active
range is 2 . 3, then there are 2 active articles, which is one more
than 3 - 2.  So I suspect that the patch attached below might be
correct.  Someone who knows more about what the "211" line should
contain should review this.

My patch also fixes the problem I've had.  With either patch, after
moving or copying an article to a new group, the new group has (active 1
. 1) as expected, with no unexist entry.

But the two patches would behave differently when active was already set.

Dan


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: nnimap.patch --]
[-- Type: text/x-diff; name="nnimap.patch", Size: 594 bytes --]

diff --git a/lisp/gnus/nnimap.el b/lisp/gnus/nnimap.el
index 17a55f98..cdd9f01f 100644
--- a/lisp/gnus/nnimap.el
+++ b/lisp/gnus/nnimap.el
@@ -918,10 +918,10 @@ nnimap-request-group
 	    (nnimap-finish-retrieve-group-infos server info sequences
 						t)
 	    (setq active (nth 2 (assoc group nnimap-current-infos)))))
-	(setq active (or active '(0 . 1)))
+	(setq active (or active '(1 . 0)))
 	(erase-buffer)
 	(insert (format "211 %d %d %d %S\n"
-			(- (cdr active) (car active))
+			(max (1+ (- (cdr active) (car active))) 0)
 			(car active)
 			(cdr active)
 			group))

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[Andreas Schwab]

On Sep 14 2024, Dan Christensen wrote:

> correct.  Someone who knows more about what the "211" line should
> contain should review this.

See RFC 3977.  It is the answer to the GROUP or LISTGROUP command.

     211 number low high group     Group successfully selected

     group     Name of newsgroup
     number    Estimated number of articles in the group
     low       Reported low water mark
     high      Reported high water mark

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[Dan Christensen]

On Sep 14, 2024, Andreas Schwab <schwab@linux-m68k.org> wrote:

> On Sep 14 2024, Dan Christensen wrote:
>
>> correct.  Someone who knows more about what the "211" line should
>> contain should review this.
>
> See RFC 3977.  It is the answer to the GROUP or LISTGROUP command.
>
>      211 number low high group     Group successfully selected
>
>      group     Name of newsgroup
>      number    Estimated number of articles in the group
>      low       Reported low water mark
>      high      Reported high water mark

Thanks, that matches what I suspected, and makes me more confident that
my patch is doing the right thing.  In particular, the car should come
before the cdr.  But someone who knows the Gnus internals should check.
In particular, for an empty group, the "low" value will be 1 while the
"high" value will be 0.

Dan

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[Andreas Schwab]

On Sep 14 2024, Dan Christensen wrote:

> In particular, for an empty group, the "low" value will be 1 while the
> "high" value will be 0.

   If the group is empty, one of the following three situations will
   occur.  Clients MUST accept all three cases; servers MUST NOT
   represent an empty group in any other way.

   o  The high water mark will be one less than the low water mark, and
      the estimated article count will be zero.  Servers SHOULD use this
      method to show an empty group.  This is the only time that the
      high water mark can be less than the low water mark.

   o  All three numbers will be zero.

   o  The high water mark is greater than or equal to the low water
      mark.  The estimated article count might be zero or non-zero; if
      it is non-zero, the same requirements apply as for a non-empty
      group.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[Dan Christensen]

On Sep 15, 2024, Andreas Schwab <schwab@linux-m68k.org> wrote:

> On Sep 14 2024, Dan Christensen wrote:
>
>> In particular, for an empty group, the "low" value will be 1 while the
>> "high" value will be 0.
>
>    If the group is empty, one of the following three situations will
>    occur.  Clients MUST accept all three cases; servers MUST NOT
>    represent an empty group in any other way.
>
>    o  The high water mark will be one less than the low water mark, and
>       the estimated article count will be zero.  Servers SHOULD use this
>       method to show an empty group.  This is the only time that the
>       high water mark can be less than the low water mark.

Thanks, that again confirms that my proposed patch is good.

Dan

PS: Some messages went to the bug tracker but not the ding list, so
I'm including the link to the bug report here for those on the ding
list who want to see the missing messages:

  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72949

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[Dan Christensen]

An alternative to

  (max (1+ (- (cdr active) (car active))) 0)

would be (range-length active), I believe.  (Untested.)  That function
assume that the range is well-formed, so doesn't use max to avoid
negative numbers, but that's probably fine.  (range-length '(1 . 0))
does correctly return 0.

Dan

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[James Thomas via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Dan Christensen wrote:

> An alternative to
>
>   (max (1+ (- (cdr active) (car active))) 0)
>
> would be (range-length active), I believe.  (Untested.)  That function
> assume that the range is well-formed, so doesn't use max to avoid
> negative numbers, but that's probably fine.  (range-length '(1 . 0))
> does correctly return 0.
>
> Dan

Just wanna chime in with what's in (info "(gnus) Required Back End
Functions"):

  Gnus just discards the total-number, so whether one should take the
  bother to generate it properly (if that is a problem) is left as an
  exercise to the reader.

--

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[Dan Christensen]

On Sep 15, 2024, James Thomas <jimjoe@gmx.net> wrote:

> Dan Christensen wrote:
>
>>   (max (1+ (- (cdr active) (car active))) 0)
>
> Just wanna chime in with what's in (info "(gnus) Required Back End
> Functions"):
>
>   Gnus just discards the total-number, so whether one should take the
>   bother to generate it properly (if that is a problem) is left as an
>   exercise to the reader.

That's interesting.  I wonder if it is still correct.

As I mentioned in an earlier message in this bug report, without that
line of the patch, Gnus gives me an error.  Without that line, Gnus
would calculate the total number of articles as -1, and this line
from gnus-start.el would fail to match because of the minus sign:

    (when (looking-at "[0-9]+ [0-9]+ \\([0-9]+\\) [0-9]+")

I'm not sure if this is the only place that would be affected.  Also,
I see that many other parts of Gnus do a correct calculation of the
total number of articles, so I guess we should follow suit.

Dan

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[James Thomas via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Dan Christensen wrote:

> On Sep 15, 2024, James Thomas <jimjoe@gmx.net> wrote:
>
>> Dan Christensen wrote:
>>
>>>   (max (1+ (- (cdr active) (car active))) 0)
>>
>> Just wanna chime in with what's in (info "(gnus) Required Back End
>> Functions"):
>>
>>   Gnus just discards the total-number, so whether one should take the
>>   bother to generate it properly (if that is a problem) is left as an
>>   exercise to the reader.
>
> That's interesting.  I wonder if it is still correct.
>
> As I mentioned in an earlier message in this bug report, without that
> line of the patch, Gnus gives me an error.  Without that line, Gnus
> would calculate the total number of articles as -1, and this line
> from gnus-start.el would fail to match because of the minus sign:
>
>     (when (looking-at "[0-9]+ [0-9]+ \\([0-9]+\\) [0-9]+")

Of course, a non-numeric character would still be wanton.

> I see that many other parts of Gnus do a correct calculation of the
> total number of articles

Wrt this function? Maybe they didn't realize this. :-)

Regards,
James

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[Dan Christensen]

On Sep 16, 2024, James Thomas <jimjoe@gmx.net> wrote:

> Dan Christensen wrote:
>
>> I see that many other parts of Gnus do a correct calculation of the
>> total number of articles
>
> Wrt this function?

Yes, for example, the code from nnml-request-group that I posted
earlier in the thread, and other places that have "211".

In any case, the correct calculation is simple, so I think it's
good to do it correctly.

Maintainers, should we go ahead and merge this?

Dan

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[James Thomas via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Dan Christensen wrote:

> On Sep 16, 2024, James Thomas <jimjoe@gmx.net> wrote:
>
>> Dan Christensen wrote:
>>
>>> I see that many other parts of Gnus do a correct calculation of the
>>> total number of articles
>>
>> Wrt this function?
>
> Yes, for example, the code from nnml-request-group that I posted
> earlier in the thread, and other places that have "211".

No, I mean something that _uses_ that value? The only function I can
find is 'gnus-browse-foreign-server' which doesn't.

Anyway not a big deal. I guess.

Regards,
James

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server, bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server, bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[James Thomas via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Dan Christensen wrote:

> On Sep 15, 2024, James Thomas wrote:
>
>> Dan Christensen wrote:
>>
>>>   (max (1+ (- (cdr active) (car active))) 0)
>>
>> Just wanna chime in with what's in (info "(gnus) Required Back End
>> Functions"):
>>
>>   Gnus just discards the total-number, so whether one should take the
>>   bother to generate it properly (if that is a problem) is left as an
>>   exercise to the reader.
>
> That's interesting.  I wonder if it is still correct.
>
> As I mentioned in an earlier message in this bug report, without that
> line of the patch, Gnus gives me an error.  Without that line, Gnus
> would calculate the total number of articles as -1, and this line
> from gnus-start.el would fail to match because of the minus sign:

Wait! I think we've miscommunicated: I'd meant the _other_ patch, the
one in (gnus-summary-goto-article "<86zfoqpobd.fsf@gmx.net>"). I use
that and it's working fine here.

Dan Christensen wrote:

> On Sep 12, 2024, James Thomas wrote:
>
>>> Sorry! I got confused again... Please try the other patch in that bug
>>> report (the one that swaps the car and cdr). I find that it works with
>>> your use case!
>
> Yes, that patch fixes the problem I've had for ages.  But I don't think
> that it's correct.  Here's what is done in nnml.el:
>
> 	  (nnheader-insert "211 %d %d %d %s\n"
> 			   (max (1+ (- (cdr active) (car active))) 0)
> 			   (car active) (cdr active) group)))))))
>
> Notice that the car comes before the cdr, and that you have to add 1
> to the difference between the cdr and the car, since if the active
> range is 2 . 3, then there are 2 active articles, which is one more
> than 3 - 2.  So I suspect that the patch attached below might be
> correct.  Someone who knows more about what the "211" line should
> contain should review this.
>
> My patch also fixes the problem I've had.  With either patch, after
> moving or copying an article to a new group, the new group has (active 1
> . 1) as expected, with no unexist entry.
>
> But the two patches would behave differently when active was already set.
>
> Dan

The total shouldn't need any change for these reasons below.

James Thomas wrote:

> Just wanna chime in with what's in (info "(gnus) Required Back End
> Functions"):
>
>   Gnus just discards the total-number, so whether one should take the
>   bother to generate it properly (if that is a problem) is left as an
>   exercise to the reader.
>
> --

>>>> I see that many other parts of Gnus do a correct calculation of the
>>>> total number of articles
>>>
>>> Wrt this function?
>>
>> Yes, for example, the code from nnml-request-group that I posted
>> earlier in the thread, and other places that have "211".
>
> No, I mean something that _uses_ that value? The only function I can
> find is 'gnus-browse-foreign-server' which doesn't.

Could you try again with that patch and a clean newsrc?

Regards,
James

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[Dan Christensen]

[-- Attachment #1: Type: text/plain, Size: 2338 bytes --]

On Sep 16, 2024, James Thomas <jimjoe@gmx.net> wrote:

> Wait! I think we've miscommunicated: I'd meant the _other_ patch, the
> one in (gnus-summary-goto-article "<86zfoqpobd.fsf@gmx.net>"). I use
> that and it's working fine here.

I believe you are talking about the patch to nnimap-request-group that
swaps the car and the cdr:

 	(insert (format "211 %d %d %d %S\n"
 			(- (cdr active) (car active))
-			(car active)
 			(cdr active)
+			(car active)
 			group))

After that patch, the relevant part of nnimap-request-group would look
like:

	(setq active (or active '(0 . 1)))
	(erase-buffer)
	(insert (format "211 %d %d %d %S\n"
			(- (cdr active) (car active))
			(cdr active)
			(car active)
			group))

This code has two bugs that (mostly) cancel each other out.  First, when
the group is empty, it represents that with the active range (0 . 1).
But that active range represents a group with two articles, numbered 0
and 1.  If you look throughout Gnus, the empty group is always
represented by the range (1 . 0).

Second, this code has the car and the cdr in the wrong order.
Everywhere else in Gnus that creates a 211 line from an active
range puts the car before the cdr.  In the case of an empty group,
these two bugs cancel.  But if active already had a non-nil value,
then the car and the cdr would be wrong.  And even if the active
range is always nil at this point, I don't approve of code that
has two bugs that cancel.

After my proposed patch (attached again), the code looks like

	(setq active (or active '(1 . 0)))
	(erase-buffer)
	(insert (format "211 %d %d %d %S\n"
			(max (1+ (- (cdr active) (car active))) 0)
			(car active)
			(cdr active)
			group))

Now the active range is correctly set to (1 . 0) for an empty group.
This alone doesn't work, as the expression (- (cdr active) (car active))
in the current code would then give a negative value for the number of
articles, which breaks at least one other place in Gnus.  So we use
the calculation that nnml-request-group uses to get the correct number
of articles in the range (by adding 1), and also force negative values
to 0, just in case active contains some bogus data.

I think this is the correct way to proceed, in line with how all
the other parts of Gnus create the 211 line.

Dan


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: nnimap.patch --]
[-- Type: text/x-diff; name="nnimap.patch", Size: 594 bytes --]

diff --git a/lisp/gnus/nnimap.el b/lisp/gnus/nnimap.el
index 17a55f98..cdd9f01f 100644
--- a/lisp/gnus/nnimap.el
+++ b/lisp/gnus/nnimap.el
@@ -918,10 +918,10 @@ nnimap-request-group
 	    (nnimap-finish-retrieve-group-infos server info sequences
 						t)
 	    (setq active (nth 2 (assoc group nnimap-current-infos)))))
-	(setq active (or active '(0 . 1)))
+	(setq active (or active '(1 . 0)))
 	(erase-buffer)
 	(insert (format "211 %d %d %d %S\n"
-			(- (cdr active) (car active))
+			(max (1+ (- (cdr active) (car active))) 0)
 			(car active)
 			(cdr active)
 			group))

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[James Thomas via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Dan Christensen wrote:

> On Sep 16, 2024, James Thomas <jimjoe@gmx.net> wrote:
>
>> Wait! I think we've miscommunicated: I'd meant the _other_ patch, the
>> one in (gnus-summary-goto-article "<86zfoqpobd.fsf@gmx.net>"). I use
>> that and it's working fine here.
>
> I believe you are talking about the patch to nnimap-request-group that
> swaps the car and the cdr:
>
>  	(insert (format "211 %d %d %d %S\n"
>  			(- (cdr active) (car active))
> -			(car active)
>  			(cdr active)
> +			(car active)
>  			group))
>
> After that patch, the relevant part of nnimap-request-group would look
> like:
>
> 	(setq active (or active '(0 . 1)))
> 	(erase-buffer)
> 	(insert (format "211 %d %d %d %S\n"
> 			(- (cdr active) (car active))
> 			(cdr active)
> 			(car active)
> 			group))
>
> This code has two bugs that (mostly) cancel each other out.  First, when
> the group is empty, it represents that with the active range (0 . 1).
> But that active range represents a group with two articles, numbered 0
> and 1.  If you look throughout Gnus, the empty group is always
> represented by the range (1 . 0).
>
> Second, this code has the car and the cdr in the wrong order.
> Everywhere else in Gnus that creates a 211 line from an active
> range puts the car before the cdr.  In the case of an empty group,
> these two bugs cancel.  But if active already had a non-nil value,
> then the car and the cdr would be wrong.  And even if the active
> range is always nil at this point, I don't approve of code that
> has two bugs that cancel.
>
> After my proposed patch (attached again), the code looks like
>
> 	(setq active (or active '(1 . 0)))
> 	(erase-buffer)
> 	(insert (format "211 %d %d %d %S\n"
> 			(max (1+ (- (cdr active) (car active))) 0)
> 			(car active)
> 			(cdr active)
> 			group))
>
> Now the active range is correctly set to (1 . 0) for an empty group.
> This alone doesn't work, as the expression (- (cdr active) (car active))
> in the current code would then give a negative value for the number of
> articles, which breaks at least one other place in Gnus.  So we use
> the calculation that nnml-request-group uses to get the correct number
> of articles in the range (by adding 1), and also force negative values
> to 0, just in case active contains some bogus data.
>
> I think this is the correct way to proceed, in line with how all
> the other parts of Gnus create the 211 line.

Thank you for the patient explanation! (I blame the confusion on the on
and off attention I gave to this). Your patch LGTM.

Regards,
James

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[James Thomas via Bug reports for GNU Emacs, the Swiss army knife of text editors]

James Thomas wrote:

> Thank you for the patient explanation! (I blame the confusion on the on
> and off attention I gave to this). Your patch LGTM.

In fact, is the max needed? Wouldn't this work as well?

diff --git a/lisp/gnus/nnimap.el b/lisp/gnus/nnimap.el
index 7b0e42ff89d..251c69706c2 100644
--- a/lisp/gnus/nnimap.el
+++ b/lisp/gnus/nnimap.el
@@ -918,10 +918,10 @@ nnimap-request-group
 	    (nnimap-finish-retrieve-group-infos server info sequences
 						t)
 	    (setq active (nth 2 (assoc group nnimap-current-infos)))))
-	(setq active (or active '(0 . 1)))
+	(setq active (or active '(1 . 0)))
 	(erase-buffer)
 	(insert (format "211 %d %d %d %S\n"
-			(- (cdr active) (car active))
+			(1+ (- (cdr active) (car active)))
 			(car active)
 			(cdr active)
 			group))


--

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[Dan Christensen]

On Sep 18, 2024, James Thomas <jimjoe@gmx.net> wrote:

> James Thomas wrote:
>
> In fact, is the max needed? Wouldn't this work as well?

This is about the line

  (max (1+ (- (cdr active) (car active))) 0)

The max is there in case active contains a pair like (10 . 8), for which
(1+ (- (cdr active) (car active))) evaluates to -1.  While this seems
unlikely to happen, at least two other backends use the above code to
guard against it, and it seems wise to be generous in what we accept.

This is especially true because gnus-parse-active does not accept
negative numbers.

Dan

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[James Thomas via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Dan Christensen wrote:

> On Sep 18, 2024, James Thomas wrote:
>
>> James Thomas wrote:
>>
>> In fact, is the max needed? Wouldn't this work as well?
>
> This is about the line
>
>   (max (1+ (- (cdr active) (car active))) 0)
>
> The max is there in case active contains a pair like (10 . 8), for which
> (1+ (- (cdr active) (car active))) evaluates to -1.  While this seems
> unlikely to happen, at least two other backends use the above code to
> guard against it, and it seems wise to be generous in what we accept.
>
> This is especially true because gnus-parse-active does not accept
> negative numbers.

Cool, thanks!

--

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[Dan Christensen]

[-- Attachment #1: Type: text/plain, Size: 156 bytes --]

So how do we go about getting this merged?  I've copied Eric
on this and have attached the patch again (since there are
several in the thread).

Dan


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: nnimap.patch --]
[-- Type: text/x-diff; name="nnimap.patch", Size: 594 bytes --]

diff --git a/lisp/gnus/nnimap.el b/lisp/gnus/nnimap.el
index 17a55f98..cdd9f01f 100644
--- a/lisp/gnus/nnimap.el
+++ b/lisp/gnus/nnimap.el
@@ -918,10 +918,10 @@ nnimap-request-group
 	    (nnimap-finish-retrieve-group-infos server info sequences
 						t)
 	    (setq active (nth 2 (assoc group nnimap-current-infos)))))
-	(setq active (or active '(0 . 1)))
+	(setq active (or active '(1 . 0)))
 	(erase-buffer)
 	(insert (format "211 %d %d %d %S\n"
-			(- (cdr active) (car active))
+			(max (1+ (- (cdr active) (car active))) 0)
 			(car active)
 			(cdr active)
 			group))

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[Dan Christensen]

On Sep 24, 2024, Dan Christensen <jdc@uwo.ca> wrote:

> So how do we go about getting this merged?

I think someone should apply my nnimap.patch and close this bug report.

The existing code is clearly wrong, and triggers a reproducible bug.

The patch is clearly correct, mirroring what is done elsewhere in Gnus,
and fixing the reproducible bug.  I explained in detail in this thread
why it is correct.  I've also been using the patched code for around
two weeks without issue.

Dan

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[Stefan Kangas]

Dan Christensen <jdc@uwo.ca> writes:

> On Sep 24, 2024, Dan Christensen <jdc@uwo.ca> wrote:
>
>> So how do we go about getting this merged?

I think we're just waiting for someone familiar with Gnus to take a look
at it.

Eric, or someone else who knows Gnus, could you please review this
patch?

> I think someone should apply my nnimap.patch and close this bug report.
>
> The existing code is clearly wrong, and triggers a reproducible bug.
>
> The patch is clearly correct, mirroring what is done elsewhere in Gnus,
> and fixing the reproducible bug.  I explained in detail in this thread
> why it is correct.  I've also been using the patched code for around
> two weeks without issue.

All of us are volunteers here, so it's unfortunately not guaranteed that
we can get to patches in anything that resembles a timely manner.  We
also don't have too many Gnus hackers on board.  While we do our best, I
can unfortunately only recommend patience here.

Please feel free to ping us again if there has been no progress within
another 1-2 months or so.  Ideally, this will be reviewed by someone
familiar with Gnus, but if it comes to it, I could take a cursory look
and push it if nothing stands out as obviously incorrect.

Thanks again for the patch, and for trying to move things forward.

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[Dan Christensen]

On Oct  1, 2024, Stefan Kangas <stefankangas@gmail.com> wrote:

>> On Sep 24, 2024, Dan Christensen <jdc@uwo.ca> wrote:
>>
>>> So how do we go about getting this merged?
>
> I think we're just waiting for someone familiar with Gnus to take a
> look at it.

Ok, thanks for explaining more about the process.  I was hoping that
since the fix is minor, it could go in now, but I'm fine with waiting
for a review.

I also re-read my last message and realize that the tone came across
wrong.  I was trying to concisely summarize the situation, but was too
curt.

Best,

Dan

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[Robert Pluim]

>>>>> On Tue, 1 Oct 2024 22:25:24 +0000, Dan Christensen <jdc@uwo.ca> said:

    Dan> On Oct  1, 2024, Stefan Kangas <stefankangas@gmail.com> wrote:
    >>> On Sep 24, 2024, Dan Christensen <jdc@uwo.ca> wrote:
    >>> 
    >>>> So how do we go about getting this merged?
    >> 
    >> I think we're just waiting for someone familiar with Gnus to take a
    >> look at it.

    Dan> Ok, thanks for explaining more about the process.  I was hoping that
    Dan> since the fix is minor, it could go in now, but I'm fine with waiting
    Dan> for a review.

Iʼve reviewed it, and it looks ok. Since I uses Gnus + nnimap here,
Iʼll apply it and see if it breaks anything.

Robert
-- 

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[Robert Pluim]

>>>>> On Wed, 02 Oct 2024 18:55:28 +0200, Robert Pluim <rpluim@gmail.com> said:

>>>>> On Tue, 1 Oct 2024 22:25:24 +0000, Dan Christensen <jdc@uwo.ca> said:
    Dan> On Oct  1, 2024, Stefan Kangas <stefankangas@gmail.com> wrote:
    >>>> On Sep 24, 2024, Dan Christensen <jdc@uwo.ca> wrote:
    >>>> 
    >>>>> So how do we go about getting this merged?
    >>> 
    >>> I think we're just waiting for someone familiar with Gnus to take a
    >>> look at it.

    Dan> Ok, thanks for explaining more about the process.  I was hoping that
    Dan> since the fix is minor, it could go in now, but I'm fine with waiting
    Dan> for a review.

    Robert> Iʼve reviewed it, and it looks ok. Since I uses Gnus + nnimap here,
    Robert> Iʼll apply it and see if it breaks anything.

It doesnʼt appear to have broken anything here, so I think we can
apply it. I think itʼs small enough not to need copyright assignment,
but Iʼll leave it to the maintainers to confirm.

Robert
-- 

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[Dan Christensen]

On Oct 11, 2024, Robert Pluim <rpluim@gmail.com> wrote:

> It doesnʼt appear to have broken anything here, so I think we can
> apply it. I think itʼs small enough not to need copyright assignment,
> but Iʼll leave it to the maintainers to confirm.

Thanks, Robert!  I submitted the copyright paperwork many years ago,
so we should be good to go.

Dan

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[Robert Pluim]

>>>>> On Fri, 11 Oct 2024 13:45:05 +0000, Dan Christensen <jdc@uwo.ca> said:

    Dan> On Oct 11, 2024, Robert Pluim <rpluim@gmail.com> wrote:
    >> It doesnʼt appear to have broken anything here, so I think we can
    >> apply it. I think itʼs small enough not to need copyright assignment,
    >> but Iʼll leave it to the maintainers to confirm.

    Dan> Thanks, Robert!  I submitted the copyright paperwork many years ago,
    Dan> so we should be good to go.

Hmm, youʼre in etc/AUTHORS, but there are no commits with 'jdc@uwo.ca'
on them. Probably a victim of the Great Gnus Merge a few years ago :-)

Robert
-- 

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: Stefan Kangas <stefankangas@gmail.com>,
>     Eric Abrahamsen <eric@ericabrahamsen.net>,  Eli Zaretskii
>  <eliz@gnu.org>,  <ding@gnus.org>,  <72949@debbugs.gnu.org>
> Date: Fri, 11 Oct 2024 15:36:43 +0200
> 
> >>>>> On Wed, 02 Oct 2024 18:55:28 +0200, Robert Pluim <rpluim@gmail.com> said:
> 
> >>>>> On Tue, 1 Oct 2024 22:25:24 +0000, Dan Christensen <jdc@uwo.ca> said:
>     Dan> On Oct  1, 2024, Stefan Kangas <stefankangas@gmail.com> wrote:
>     >>>> On Sep 24, 2024, Dan Christensen <jdc@uwo.ca> wrote:
>     >>>> 
>     >>>>> So how do we go about getting this merged?
>     >>> 
>     >>> I think we're just waiting for someone familiar with Gnus to take a
>     >>> look at it.
> 
>     Dan> Ok, thanks for explaining more about the process.  I was hoping that
>     Dan> since the fix is minor, it could go in now, but I'm fine with waiting
>     Dan> for a review.
> 
>     Robert> Iʼve reviewed it, and it looks ok. Since I uses Gnus + nnimap here,
>     Robert> Iʼll apply it and see if it breaks anything.
> 
> It doesnʼt appear to have broken anything here, so I think we can
> apply it. I think itʼs small enough not to need copyright assignment,
> but Iʼll leave it to the maintainers to confirm.

It's small enough, yes.  Just please don't forget the
Copyright-paperwork-exempt thingy.

Thanks.

bug#72949: Gnus sometimes reports new messages but not showing them on IMAP server

[Robert Pluim]

tags 72949 fixed
close 72949 31.1
quit

>>>>> On Fri, 11 Oct 2024 18:39:33 +0300, Eli Zaretskii <eliz@gnu.org> said:

    Eli> It's small enough, yes.  Just please don't forget the
    Eli> Copyright-paperwork-exempt thingy.

Given Danʼs comments about assignment, Iʼm not 100% sure itʼs
necessary, but I put it in anyway.

Closing.
Committed as 5c129791c5b

Robert
--