bug#34502: Ask yourself, am I generating a NEW .overview file?

bug#34502: Ask yourself, am I generating a NEW .overview file?

[積丹尼 Dan Jacobson]

nnml bug:
1. exit emacs
2. remove an .overview file. (Say it was corrupted. Or mom didn't like
it, whatever. State secrets, you know.)
3. start emacs and gnus. etc. All is OK.

1. exit emacs
2. remove an .overview file.
3. send yourself a mail
4. start emacs and gnus... upon entering the group,
the new overview file is generated only *containing only that new mail*.
The user is shocked to see all his other mail is "lost forever" there
is the *Summary* buffer.

Workaroud: first
"you should regenerate them by entering the server buffer using ‘^’, and then ‘g’ on the
appropriate mail server (e.g., nnml) to cause regeneration."

So, bug: when gnus regenerates overview files on its own, if it finds a
new mail, it only uses that, not noticing all the existing mails.
It first needs to ask itself: "am I, ho-hum, appending to an existing .overview
file, or gosh-once-in-a-lifetime fresh generating it for the first time?" So it should look
first, if it finds no .overview file, it had better regenerate it
first before incorporating any new mail! Thanks!

What's worse about this bug is gnus will work great for days, months, with no
.overview file for that group. Until one fateful day, when the user
receives a new mail. And "KABAM", all his messages except that new mail
are suddenly gone. No ESC 99999 RET entering the group will retrieve
them back into the *Summary* buffer etc.

Sure, you could say it is his own "fault" for removing the .overview
file without asking grandma, and (seemingly) losing all his mails is
proper "punishment" for the crime, case closed.

But you could simply always check first: does the .overview file I am
about to cheerily append to exist in the first place?

Gnus v5.13
GNU Emacs 26.1 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.24.4)
 of 2019-02-03, modified by Debian

bug#34502: Ask yourself, am I generating a NEW .overview file?

[Lars Ingebrigtsen]

積丹尼 Dan Jacobson <jidanni@jidanni.org> writes:

> nnml bug:
> 1. exit emacs
> 2. remove an .overview file. (Say it was corrupted. Or mom didn't like
> it, whatever. State secrets, you know.)
> 3. start emacs and gnus. etc. All is OK.

When you say "All is OK", do you mean that Gnus regenerates the nnml
.overview file automatically when you start Gnus?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#34502: Ask yourself, am I generating a NEW .overview file?

[積丹尼 Dan Jacobson]

>>>>> "LI" == Lars Ingebrigtsen <larsi@gnus.org> writes:
LI> When you say "All is OK", do you mean that Gnus regenerates the nnml
LI> .overview file automatically when you start Gnus?

Yes, I think. I forgot. Can't remember for more than a month.

bug#34502: Ask yourself, am I generating a NEW .overview file?

[Lars Ingebrigtsen]

積丹尼 Dan Jacobson <jidanni@jidanni.org> writes:

>>>>>> "LI" == Lars Ingebrigtsen <larsi@gnus.org> writes:
> LI> When you say "All is OK", do you mean that Gnus regenerates the nnml
> LI> .overview file automatically when you start Gnus?
>
> Yes, I think. I forgot. Can't remember for more than a month.

I've looked through the code for the nnml regeneration code paths, and I
can't see that Gnus would do that automatically.  Are you sure you
didn't issue one of the commands to regenerate the files?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#34502: Ask yourself, am I generating a NEW .overview file?

[積丹尼 Dan Jacobson]

>>>>> "LI" == Lars Ingebrigtsen <larsi@gnus.org> writes:

LI> I've looked through the code for the nnml regeneration code paths, and I
LI> can't see that Gnus would do that automatically.  Are you sure you
LI> didn't issue one of the commands to regenerate the files?

All I can say is close gnus, remove the .overview file.
Then the next day continue using gnus as normal.
Sooner or later the .overview will grow back again, just like a toenail
does. That's all I know.

bug#34502: Ask yourself, am I generating a NEW .overview file?

[Lars Ingebrigtsen]

積丹尼 Dan Jacobson <jidanni@jidanni.org> writes:

>>>>>> "LI" == Lars Ingebrigtsen <larsi@gnus.org> writes:
>
> LI> I've looked through the code for the nnml regeneration code paths, and I
> LI> can't see that Gnus would do that automatically.  Are you sure you
> LI> didn't issue one of the commands to regenerate the files?
>
> All I can say is close gnus, remove the .overview file.
> Then the next day continue using gnus as normal.
> Sooner or later the .overview will grow back again, just like a toenail
> does. That's all I know.

As you receive emails, the .overview file will get the entries from that
mail, and will keep growing.  Is that what you're seeing?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#34502: Ask yourself, am I generating a NEW .overview file?

[積丹尼 Dan Jacobson]

>>>>> "LI" == Lars Ingebrigtsen <larsi@gnus.org> writes:
LI> As you receive emails, the .overview file will get the entries from that
LI> mail, and will keep growing.  Is that what you're seeing?

So that explains how "all the old mail disappears when the first new
mail arrives."

Therefore gnus should be sure to check before blindly creating a new
.overview file, just like I said at the top of this bug.

bug#34502: Ask yourself, am I generating a NEW .overview file?

[Lars Ingebrigtsen]

積丹尼 Dan Jacobson <jidanni@jidanni.org> writes:

>>>>>> "LI" == Lars Ingebrigtsen <larsi@gnus.org> writes:
> LI> As you receive emails, the .overview file will get the entries from that
> LI> mail, and will keep growing.  Is that what you're seeing?
>
> So that explains how "all the old mail disappears when the first new
> mail arrives."

OK, so your .overview file is never regenerated?  That explains the mystery.

> Therefore gnus should be sure to check before blindly creating a new
> .overview file, just like I said at the top of this bug.

No, it's perfectly legitimate to blow out the .overview file if you want
to.  If you don't want to, don't do that, then.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#34502: Ask yourself, am I generating a NEW .overview file?

[積丹尼 Dan Jacobson]

I'm saying that blindly generating a short overview file when the next
message arrives will cause the user to lose all their other mail.

You might argue that the mail is still there, but try telling that to
the user.

The overview file might be gone for many reasons. E.g., full disks
recently, etc.

It is irresponsible and careless to not add a simple check for the
situation I reported.