bug#56332: 29.0.50; Large gnus imap groups; articles incorrectly marked as read (old)

[Michael Welsh Duggan <mwd@md5i.com>]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Michael Welsh Duggan <mwd@md5i.com> writes:
>
>>> I will also note that, though the fetch data responses are not in order,
>>> the fetch completion messages are in order.  Though I'm not certain they
>>> have to be.  Here's some data from the Internet, though I can't find
>>> anything in the standard that seems to either confirm or refute this
>>> data:
>>>
>>> https://stackoverflow.com/questions/26034086/does-imap-guarantee-that-servers-send-responses-in-order
>>>
>>> Wouldn't another solution be to sort the results by UID?  They are being
>>> requested in UID order, after all.
>>
>> You should probably read this section of the RFC, especially the
>> "Note:".
>>
>> https://datatracker.ietf.org/doc/html/rfc3501#section-5.5
>
> Reading that, I'm not sure whether the completion messages are
> guaranteed to be in order, either, so I've now changed the code to avoid
> streaming altogether.  Can you check whether that fixes the problem?
>
> (If the completion messages are guaranteed to be in order, we could
> change it back to using streaming and then just reorder the results, as
> you suggest, but I'm not sure it's worth it even if it is guaranteed.)

As expected, I cannot cause a failure using this.  Though I will note
that there is no guarantee that a server will send its untagged fetch
responses in any particular order.  I will admit that it normally makes
very little sense for a server to actually do this in another order, but
I can conceive of two reasons that a server might do this, one of which
would probably not affect Gnus, but the other of which could:

1) If the messages were not requested in a sorted order, I could see
a server generating a sorted order anyway for internal implementation
reasons. (Not a problem for Gnus, since it will always request the
messages in sorted order.)

2) If messages were being retrieved from some sort of big-data
map-reduce store, the messages could conceivable come back in any order
due to messages being stored on different nodes with different
performance characteristics.  The server is not obligated to sort the
results before returning them.

That said, this fix will work for me for Dovecot and may work on all
existent or future IMAP servers, but I don't believe the standard
requires that this will be the case.

-- 
Michael Welsh Duggan
(md5i@md5i.com)