bug#33005: 27.0.50; Data loss with Gnus registry

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Hello,

I want to use the Gnus registry to use registry marks and store data as
described in (info "(gnus) Store arbitrary data").  Very often, these
things vanish after restarting Emacs or Gnus.

For example, I use M M i to mark some article "Important".  I have
configured `gnus-summary-line-format' to show registry marks.  Often,
after restarting, the mark is gone.  Likewise, data stored with
`gnus-registry-set-id-key' gets lost, even when the according article
has a registry mark attached.

[BTW, there is a typo in (info "(gnus) Store arbitrary data").  It says
`gnus-registry-extra-entries-precious' would default to (marks) but it
defaults to (mark)]

My final goal is to implement a little Gnu Elpa package
"gnus-article-notes.el" that can be used to attach little text notes to
arbitrary articles.  The package is already done, if you are interested,
I can send you the code.  I have verified that registry marks get lost
independently of that package.  I guess there is something wrong with
registry pruning.  CC'ing Eric Abrahamsen who has recently worked in
this area.


TIA,

Michael.




In GNU Emacs 27.0.50 (build 14, x86_64-pc-linux-gnu, GTK+ Version 3.24.1)
 of 2018-10-10 built on drachen
Repository revision: db1daee25ae82cdd1e0b7a8f50fa55003f31cf67
Windowing system distributor 'The X.Org Foundation', version 11.0.12001000
System Description: Debian GNU/Linux buster/sid

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND DBUS GSETTINGS GLIB NOTIFY
GNUTLS LIBXML2 FREETYPE XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM
THREADS JSON LCMS2 GMP

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Hello,
>
> I want to use the Gnus registry to use registry marks and store data as
> described in (info "(gnus) Store arbitrary data").  Very often, these
> things vanish after restarting Emacs or Gnus.
>
> For example, I use M M i to mark some article "Important".  I have
> configured `gnus-summary-line-format' to show registry marks.  Often,
> after restarting, the mark is gone.  Likewise, data stored with
> `gnus-registry-set-id-key' gets lost, even when the according article
> has a registry mark attached.
>
> [BTW, there is a typo in (info "(gnus) Store arbitrary data").  It says
> `gnus-registry-extra-entries-precious' would default to (marks) but it
> defaults to (mark)]

There were some bugs with registry pruning and precious entries, but
we fixed those quite a while ago. If you're running Emacs master, you
should have the Gnus that incorporates those fixes. On the other hand,
what you say above about the docstring makes me wonder, as in Gnus
master it doesn't say it would default to (marks). You're not loading an
external Gnus installation, are you?

Eric

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> There were some bugs with registry pruning and precious entries, but
> we fixed those quite a while ago. If you're running Emacs master, you
> should have the Gnus that incorporates those fixes.

Yes, I think so, too.  I have a normal recent Emacs master build.  I
typically rebuild daily.

> On the other hand, what you say above about the docstring makes me
> wonder, as in Gnus master it doesn't say it would default to
> (marks). You're not loading an external Gnus installation, are you?

No, I'm not.  In emacs master branch, "doc/misc/gnus.texi" says

  "default this is just @code{(marks)} so the custom registry marks are"

in line 26197.  Not for you?



Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> There were some bugs with registry pruning and precious entries, but
>> we fixed those quite a while ago. If you're running Emacs master, you
>> should have the Gnus that incorporates those fixes.
>
> Yes, I think so, too.  I have a normal recent Emacs master build.  I
> typically rebuild daily.
>
>> On the other hand, what you say above about the docstring makes me
>> wonder, as in Gnus master it doesn't say it would default to
>> (marks). You're not loading an external Gnus installation, are you?
>
> No, I'm not.  In emacs master branch, "doc/misc/gnus.texi" says
>
>   "default this is just @code{(marks)} so the custom registry marks are"
>
> in line 26197.  Not for you?

Sorry, I was looking at the option docstring, not the manual. I'll fix
that typo.

But that was my only easy solution, unfortunately. You could examine the
results of:

(registry-collect-prune-candidates gnus-registry-db
(registry-size gnus-registry-db) nil)

To verify that it is returning candidates that should not be returned.
Unfortunately that function uses `cl-loop', which makes edebugging not
very helpful.

Shameless plug: I'm about to propose a new package called "gnus-mock",
which sets up a temporary/trashable/reproducible Gnus installation that
you can use for testing feature and hunting bugs, without endangering
your own Gnus data. That would be useful in this case. I might have it
done today.

Eric

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> But that was my only easy solution, unfortunately. You could examine the
> results of:
>
> (registry-collect-prune-candidates gnus-registry-db
> (registry-size gnus-registry-db) nil)

Ok, I'll try that.

> Shameless plug: I'm about to propose a new package called "gnus-mock",
> which sets up a temporary/trashable/reproducible Gnus installation that
> you can use for testing feature and hunting bugs, without endangering
> your own Gnus data.

Yes, that's sounds exactly like the thing I wanted :-)


Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> (registry-collect-prune-candidates gnus-registry-db
> (registry-size gnus-registry-db) nil)

Ok, I managed to debug this.

The entry is indeed pruned.  It looks like

Key:  "<153...>"

Value: ((mark Important) (subject ...) ...)

In the `registry-collect-prune-candidates' method, the variable PRECIOUS
is bound to the list (gnorb-ids org-tags mark).

AFAIU, since PRECIOUS-P is defined as

  (lambda (entry-key) (cdr (memq (car-safe entry-key) precious)))

and the symbol mark comes last in the PRECIOUS list, the `cdr' of the
`memq' call is nil.  If I remove the `cdr' call, the entry isn't pruned
any more.  I also don't get why that `cdr' is there.  Or is my value of
PRECIOUS illegal?


Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

On 10/10/18 23:24 PM, Michael Heerdegen wrote:
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> (registry-collect-prune-candidates gnus-registry-db
>> (registry-size gnus-registry-db) nil)
>
> Ok, I managed to debug this.
>
> The entry is indeed pruned.  It looks like
>
> Key:  "<153...>"
>
> Value: ((mark Important) (subject ...) ...)
>
> In the `registry-collect-prune-candidates' method, the variable PRECIOUS
> is bound to the list (gnorb-ids org-tags mark).
>
> AFAIU, since PRECIOUS-P is defined as
>
>   (lambda (entry-key) (cdr (memq (car-safe entry-key) precious)))
>
> and the symbol mark comes last in the PRECIOUS list, the `cdr' of the
> `memq' call is nil.  If I remove the `cdr' call, the entry isn't pruned
> any more.  I also don't get why that `cdr' is there.  Or is my value of
> PRECIOUS illegal?

Nice work! I have no idea why that `cdr' is in there, and as the value
is used as a boolean it seems totally superfluous. I don't use registry
marks, which is probably the reason I never noticed (I didn't write this
code).

I don't see any reason not to remove the `cdr', and will do so unless
someone objects cogently, soon.

Thanks,
Eric

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Nice work! I have no idea why that `cdr' is in there, and as the value
> is used as a boolean it seems totally superfluous. I don't use registry
> marks, which is probably the reason I never noticed (I didn't write this
> code).

Looks like the code has been like that from the beginning: ccd58722df
"Add lisp/gnus/registry.el.", Ted Zlatanov, Tue Apr 5 23:37:02 2011
introduces the file (it has been moved once since then), and the code is
already the same there.

I can only guess that the intention could have been to distinguish (mark) from
(mark SOMETHING): prune the first because the mark field is empty.
Anyway, seems it never worked like that.


Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Anyway, seems it never worked like that.

But what's even more frightening: The default value of
`gnus-registry-extra-entries-precious', (mark), always behaved like ()
and thus never worked as intended, so it seems that registry marks
always where broken with the defaults.  Looks like nobody ever tried to
use them, or all people gave up without writing a bug report.

Should we even make a NEWS entry along the fix saying that the feature
can actually be used now?


Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

On 10/11/18 15:10 PM, Michael Heerdegen wrote:
> Michael Heerdegen <michael_heerdegen@web.de> writes:
>
>> Anyway, seems it never worked like that.
>
> But what's even more frightening: The default value of
> `gnus-registry-extra-entries-precious', (mark), always behaved like ()
> and thus never worked as intended, so it seems that registry marks
> always where broken with the defaults.  Looks like nobody ever tried to
> use them, or all people gave up without writing a bug report.

I don't think very many people use the registry, or at least, they
aren't using it intentionally to store their own information.

> Should we even make a NEWS entry along the fix saying that the feature
> can actually be used now?

I don't think so, bug fixes don't usually get a NEWS entry. But I can
say something on the gnus.general list.

Also, I really like the idea of using the registry to attach notes to
mails -- that's something I've intended for Gnorb for a while. If you'd
like to contribute that to Gnorb I'd be very happy to accept it, or
maybe it's something that could even go into the registry proper. Of
course, if you'd prefer to keep it a separate package that's fine too --
I'd install it!

Thanks,
Eric

bug#33005: 27.0.50; Data loss with Gnus registry

[Eli Zaretskii]

> From: Eric Abrahamsen <eric@ericabrahamsen.net>
> Date: Wed, 10 Oct 2018 16:05:50 -0700
> Cc: 33005@debbugs.gnu.org
> 
> I don't see any reason not to remove the `cdr', and will do so unless
> someone objects cogently, soon.

I see you already did, and on the emacs-26 branch.  Please in the
future ask about commits to the release branch.

Thanks.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Eric Abrahamsen <eric@ericabrahamsen.net>
>> Date: Wed, 10 Oct 2018 16:05:50 -0700
>> Cc: 33005@debbugs.gnu.org
>> 
>> I don't see any reason not to remove the `cdr', and will do so unless
>> someone objects cogently, soon.
>
> I see you already did, and on the emacs-26 branch.  Please in the
> future ask about commits to the release branch.
>
> Thanks.

Sorry about that -- it occurred to me I should have asked right after I
pushed. Will ask in the future.

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> I see you already did, and on the emacs-26 branch.  Please in the
> future ask about commits to the release branch.

Some days ago I asked a question about the habits and expectations
regarding bug fixes, in particular for bugs that were not introduced by
the current release.  Nobody disagreed that committing to the release is
just ok.  So it is not?  Why do you think people know what you expect?
(This is not a rant, I really just want to know what the rules and
expectations are.)


Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

[-- Attachment #1: Type: text/plain, Size: 962 bytes --]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> > Should we even make a NEWS entry along the fix saying that the feature
> > can actually be used now?
>
> I don't think so, bug fixes don't usually get a NEWS entry.

Well, it's at least the first time that you can actually use it.

> But I can say something on the gnus.general list.

Ok.

> Also, I really like the idea of using the registry to attach notes to
> mails -- that's something I've intended for Gnorb for a while. If you'd
> like to contribute that to Gnorb I'd be very happy to accept it, or
> maybe it's something that could even go into the registry proper. Of
> course, if you'd prefer to keep it a separate package that's fine too --
> I'd install it!

I wanted something really really simple, in particular, something that
is not linked to org.  It's an approach different from Gnorb, but maybe
it would fit into Gnus, dunno.  I attach what I have so far - feedback
welcome.


Michael.



[-- Attachment #2: gnus-article-notes.el --]
[-- Type: application/emacs-lisp, Size: 8093 bytes --]

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

On 10/11/18 22:28 PM, Michael Heerdegen wrote:

[...]

> I wanted something really really simple, in particular, something that
> is not linked to org. It's an approach different from Gnorb, but maybe
> it would fit into Gnus, dunno. I attach what I have so far - feedback
> welcome.

Ha, I see what you mean, most of the code is for displaying and editing
the notes. I did something similar in org-annotate[1] (which is pretty
similar in spirit to what you're doing here, but for org files), but
just realized that I don't use the popup for editing, only display.

Well, let's see if anyone is interested in a generalized solution!

BTW I just used `set-local-key' in my function -- I don't think it's
necessary to make a new local keymap for a single-use buffer.

Eric

[1]: https://github.com/girzel/org-annotate/blob/master/org-annotate.el#L150


>
> Michael.
>
>
>;;; gnus-article-notes.el---Attach notes to messages in Gnus -*- lexical-binding: t -*-
>
>;; Copyright (C) 2018 Free Software Foundation, Inc
>
>;; Author: Michael Heerdegen <michael_heerdegen@web.de>
>;; Maintainer: Michael Heerdegen <michael_heerdegen@web.de>
>;; Created: 2017_12_11
>;; Keywords: news registry
>;; Version: 0.1
>;; Package-Requires: ()
>
>
>;; This file is not part of GNU Emacs.
>
>;; GNU Emacs is free software: you can redistribute it and/or modify
>;; it under the terms of the GNU General Public License as published by
> ;; the Free Software Foundation, either version 3 of the License, or
> ;; (at your option) any later version.
>
> ;; GNU Emacs is distributed in the hope that it will be useful,
> ;; but WITHOUT ANY WARRANTY; without even the implied warranty of
> ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> ;; GNU General Public License for more details.
>
> ;; You should have received a copy of the GNU General Public License
> ;; along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.
>
>
> ;;; Commentary:
>
> ;; This simple package allows to attach text notes to articles in
> ;; Gnus.  This is actually just a trivial convenience wrapper around
> ;; `gnus-registry-set-id-key' and `gnus-registry-get-id-key'.
> ;;
> ;; For something less simplistic see the Gnorb package in Gnu Elpa.
> ;; It can save notes in org files, track discussions, and much more.
> ;;
> ;;
> ;; Usage
> ;; =====
> ;;
> ;; The main command is `gnus-article-notes-set-note' bound to "@" in
> ;; the summary keymap.
> ;;
> ;; If the current article has not yet an attached note, hit @ to add
> ;; one.  The article is also flagged with an "@" to indicate that a
> ;; note has been attached.
> ;;
> ;; When an article has already an attached note, "@" displays the note
> ;; in the echo area, and hitting "@" again let's you edit the note.
> ;; "@" with a prefix argument 0 deletes the note after confirmation.
> ;; "@" with any other prefix arg also reads in a note text but using a
> ;; pop-up buffer instead of the minibuffer making editing multi-line
> ;; notes more convenient.
> ;;
> ;;
> ;; Setup
> ;; =====
> ;;
> ;; Somewhere in your initialization you need to enable the Gnus
> ;; registry (where this package saves your notes), load this file, and
> ;; make the key binding:
> ;;
> ;;   (gnus-registry-initialize)
> ;;   (require 'gnus-article-notes)
> ;;   (add-hook
> ;;    'gnus-summary-mode-hook
> ;;    (defun my-gnus-summary-mode-hook-bind-key-for-article-notes ()
> ;;      (define-key gnus-summary-mode-map [?@] #'gnus-article-notes-set-note)))
> ;;
> ;; It is a good idea to read about what enabling the registry means if
> ;; you haven't yet used it: (info "(gnus) The Gnus Registry").  It is
> ;; easy stuff.  You may want to limit how much data Gnus stores in the
> ;; registry to avoid delays when saving (it stores a lot by default).
> ;; I do (setq gnus-registry-max-entries 2000).  Note that pruning a
> ;; full registry will never delete notes unless you change
> ;; `gnus-registry-extra-entries-precious' to not contain `mark'.
> ;; Loading this package adds a "Note" named custom mark to
> ;; `gnus-registry-marks' (by default).
> ;;
> ;; To see the "@" marker for messages with attached notes in the
> ;; summary buffer, you also want something like
> ;;
> ;;   (defalias 'gnus-user-format-function-M
> ;;             'gnus-registry-article-marks-to-chars)
> ;;
> ;; which allows you to use "%uM" (or better with a padding like in
> ;; "%2uM") in `gnus-summary-line-format' to show registry marks - see
> ;; (info "(gnus) Store custom flags and keywords") for details.
> ;;
> ;; Finally you may also want to look at the few customizable options
> ;; defined in this file.
>
>
>
> ;;; Code:
>
>
>
> (eval-when-compile (require 'subr-x))
> (require 'gnus)
> (require 'gnus-registry)
>
> (defvar gnus-article-notes-registry-field 'Note)
> (defvar gnus-article-notes-marker-char ?@)
> (defvar gnus-article-notes-auto-tick nil)
>
> (defvar gnus-article-notes-show-in-summary t)
>
> (defun gnus-article-notes-registry-delete-id-key (id key)
>   (let* ((db gnus-registry-db)
>          (entry (gnus-registry-get-or-make-entry id)))
>     (registry-delete db (list id) nil)
>     (setq entry (assq-delete-all key entry))
>     (gnus-registry-insert db id entry)
>     entry))
>
> (with-eval-after-load 'gnus-registry
>   (add-to-list 'gnus-registry-marks
>                `(,gnus-article-notes-registry-field :char ,gnus-article-notes-marker-char :image nil)))
>
> (defvar gnus-article-notes-popup-window-action '())
>
> ;; We could make the major mode customizable...
> (defun gnus-article-notes-read-string-with-buffer (&optional initial-input keymap comment)
>   (cl-callf or comment ";; Hit C-c C-c when done\n\n") ;FIXME: add key to abort
>   (save-window-excursion
>     (with-temp-buffer
>       (let ((win (display-buffer (current-buffer) gnus-article-notes-popup-window-action)))
>         (select-window win)
>         (insert comment)
>         (when initial-input (insert initial-input))
>         (set-window-point win (point-max))
>         (use-local-map (let ((map (make-sparse-keymap)))
>                          (set-keymap-parent map (or keymap text-mode-map))
>                          (define-key map [(control ?c) (control ?c)] #'exit-recursive-edit)
>                          map))
>         (recursive-edit)
>         (string-trim
>          (replace-regexp-in-string
>           (concat "\\`" (regexp-quote comment)) ""
>           (buffer-string)))))))
>
> (defun gnus-article-notes-set-note (id new-content)
>   (if (not new-content)
>       (gnus-article-notes-registry-delete-id-key id gnus-article-notes-registry-field)
>     (gnus-registry-set-id-key id gnus-article-notes-registry-field new-content)))
>
> (defun gnus-article-notes-display-or-set-note (article id &optional content)
>   "Doc..."
>   (interactive
>    (let* ((articles (gnus-summary-work-articles nil))
>           (article (if (cdr articles) (user-error "Cannot operate on multiple articles")
>                      (car articles)))
>           (id (mail-header-id (gnus-summary-article-header article)))
>           (current-content (gnus-registry-get-id-key id gnus-article-notes-registry-field)))
>      (list article
>            id
>            (if (or (eq this-command last-command) (not current-content) current-prefix-arg)
>                (let ((new-content
>                       (if current-prefix-arg
>                           (if (eq 0 (prefix-numeric-value current-prefix-arg))
>                               (if (yes-or-no-p "Really delete note? ")
>                                   nil
>                                 (user-error "Abort"))
>                             (gnus-article-notes-read-string-with-buffer current-content))
>                         (read-string "New note: " current-content))))
>                  (if (equal "" new-content) nil new-content))
>              `(display . ,current-content)))))
>   (pcase content
>     (`(display . ,content) (message "%s" content))
>     (_ (when (and content gnus-article-notes-auto-tick) (gnus-summary-tick-article-forward 1))
>        (gnus-article-notes-set-note id content)
>        (gnus-registry--set/remove-mark 'Note (not content) (list article)))))
>
> (defun gnus-article-notes-get-additional-articles (group-name)
>   (delq nil
>         (mapcar (lambda (id) (cdr (gnus-request-head id group-name)))
>                 (cl-loop for key being the hash-keys of (oref gnus-registry-db data)
>                          using (hash-values v)
>                          when (assoc gnus-article-notes-registry-field v)
>                          collect key))))
>
>
> (defun gnus-articles-notes-alter-articles-to-read-function (f group-name article-list)
>   (let ((others (funcall f group-name article-list)))
>     (if gnus-article-notes-show-in-summary
>         (cl-union (gnus-article-notes-get-additional-articles group-name)
>                   others)
>       others)))
>
> (add-function :around gnus-alter-articles-to-read-function
>               #'gnus-articles-notes-alter-articles-to-read-function)
>
>
>
> (provide 'gnus-article-notes)
> ;;; gnus-article-notes.el ends here

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> BTW I just used `set-local-key' in my function -- I don't think it's
> necessary to make a new local keymap for a single-use buffer.

Don't know that function - do you mean `local-set-key'?  In this case I
must warn, citing the docstring of `local-set-key':

| The binding goes in the current buffer's local map, which in most
| cases is shared with all other buffers in the same major mode.


Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> BTW I just used `set-local-key' in my function -- I don't think it's
>> necessary to make a new local keymap for a single-use buffer.
>
> Don't know that function - do you mean `local-set-key'?  In this case I
> must warn, citing the docstring of `local-set-key':
>
> | The binding goes in the current buffer's local map, which in most
> | cases is shared with all other buffers in the same major mode.

I did mean `local-set-key' and did not realize that's how it works!
Thanks for the tip.

Eric

bug#33005: 27.0.50; Data loss with Gnus registry

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: Eric Abrahamsen <eric@ericabrahamsen.net>,  33005@debbugs.gnu.org
> Date: Thu, 11 Oct 2018 22:08:08 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I see you already did, and on the emacs-26 branch.  Please in the
> > future ask about commits to the release branch.
> 
> Some days ago I asked a question about the habits and expectations
> regarding bug fixes, in particular for bugs that were not introduced by
> the current release.  Nobody disagreed that committing to the release is
> just ok.  So it is not?

In general, it is.  But a bug fix should be safe enough to be eligible
for the release branch, and the "enough" part changes depending on the
state of the branch.  Currently, since Emacs 26.1 was already
released, only very safe fixes are eligible, and I'd like to be part
of the decision loop regarding the safety of each proposed fix.

> Why do you think people know what you expect?

Because I said it many times here.  The fact is, most people do ask.

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> > Why do you think people know what you expect?
>
> Because I said it many times here.  The fact is, most people do ask.

But CONTRIBUTE tells something different.  I have the feeling that some
people ask because they are lost, not because they know that it is
expected that they ask.  Others just don't ask.

Anyway, if it works for you...but I think it would be better to have
some rules written down somewhere if we want to be attractive to new
contributors.


Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: eric@ericabrahamsen.net,  33005@debbugs.gnu.org
> Date: Fri, 12 Oct 2018 13:04:53 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > > Why do you think people know what you expect?
> >
> > Because I said it many times here.  The fact is, most people do ask.
> 
> But CONTRIBUTE tells something different.

CONTRIBUTE's main audience is people who have no write access to the
Emacs repository, so what it says just tells them on what branch to
base the patches.  The decision to which branch to push is then made
by someone who actually pushes the changes.

I'm okay with adding the request to ask about committing to the
release branch, but at the time I was under the impression that some
people didn't agree with such a policy, so I'm not sure the project as
a whole would like it carved in stone from here to eternity.

> I have the feeling that some people ask because they are lost, not
> because they know that it is expected that they ask.

That's not the reality.  People explicitly ask _me_ whether it's okay
to push to the release branch; the exceptions are almost non-existent.

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Well, let's see if anyone is interested in a generalized solution!

What do you think I should do with the thing?  Propose it to
emacs.gnus.general, or just upload to Gnu Elpa?


Thanks,

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> I don't see any reason not to remove the `cdr', and will do so unless
> someone objects cogently, soon.

Ok, can we close this report - seems we are done?


Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

On 10/12/18 16:22 PM, Michael Heerdegen wrote:
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Well, let's see if anyone is interested in a generalized solution!
>
> What do you think I should do with the thing?  Propose it to
> emacs.gnus.general, or just upload to Gnu Elpa?

I'd put it in Elpa and then alert the list -- things tend to just
disappear if you post them without a permanent home.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

On 10/12/18 16:46 PM, Michael Heerdegen wrote:
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> I don't see any reason not to remove the `cdr', and will do so unless
>> someone objects cogently, soon.
>
> Ok, can we close this report - seems we are done?

Yup, closing now.

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Ok, can we close this report - seems we are done?

Sorry for the bad news - but my registry marks are disappearing again
:-(  Just after restarting Gnus they are gone.

I wanted to try with gnus-mock but it errors:

Debugger entered--Lisp error: (buffer-read-only #<buffer .newsrc.eld|emacs-gnus-mock-ENKWay>)
  replace-match("/tmp/emacs-gnus-mock-ENKWay" t)
  gnus-mock-start()
 [...]

I tried to load the source of gnus-mock to get a more meaningful
backtrace but that also errors:

Debugger entered--Lisp error: (wrong-type-argument stringp nil)
  file-name-directory(nil)
  (expand-file-name "data" (file-name-directory load-file-name))
  (file-name-as-directory (expand-file-name "data" (file-name-directory load-file-name)))
  (defconst gnus-mock-data-dir (file-name-as-directory (expand-file-name "data" (file-name-directory load-file-name))) "Source directory for Gnus mock data.")
  eval-region(1 10407)  ; Reading at buffer position 5125

Hmm...

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Michael Heerdegen <michael_heerdegen@web.de> writes:
>
>> Ok, can we close this report - seems we are done?
>
> Sorry for the bad news - but my registry marks are disappearing again
> :-(  Just after restarting Gnus they are gone.
>
> I wanted to try with gnus-mock but it errors:
>
> Debugger entered--Lisp error: (buffer-read-only #<buffer .newsrc.eld|emacs-gnus-mock-ENKWay>)
>   replace-match("/tmp/emacs-gnus-mock-ENKWay" t)
>   gnus-mock-start()
>  [...]
>
> I tried to load the source of gnus-mock to get a more meaningful
> backtrace but that also errors:
>
> Debugger entered--Lisp error: (wrong-type-argument stringp nil)
>   file-name-directory(nil)
>   (expand-file-name "data" (file-name-directory load-file-name))
>   (file-name-as-directory (expand-file-name "data" (file-name-directory load-file-name)))
>   (defconst gnus-mock-data-dir (file-name-as-directory (expand-file-name "data" (file-name-directory load-file-name))) "Source directory for Gnus mock data.")
>   eval-region(1 10407)  ; Reading at buffer position 5125
>
> Hmm...

Ah, this was a question I had when I was first writing gnus-mock --
load-file-name was reliably present for all my tests, but they were only
on Linux. Can you tell me what system you're running? Presumably
there are better alternatives to load-file-name under different
circumstances.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Michael Heerdegen <michael_heerdegen@web.de> writes:
>
>> Michael Heerdegen <michael_heerdegen@web.de> writes:
>>
>>> Ok, can we close this report - seems we are done?
>>
>> Sorry for the bad news - but my registry marks are disappearing again
>> :-(  Just after restarting Gnus they are gone.
>>
>> I wanted to try with gnus-mock but it errors:
>>
>> Debugger entered--Lisp error: (buffer-read-only #<buffer .newsrc.eld|emacs-gnus-mock-ENKWay>)
>>   replace-match("/tmp/emacs-gnus-mock-ENKWay" t)
>>   gnus-mock-start()
>>  [...]
>>
>> I tried to load the source of gnus-mock to get a more meaningful
>> backtrace but that also errors:
>>
>> Debugger entered--Lisp error: (wrong-type-argument stringp nil)
>>   file-name-directory(nil)
>>   (expand-file-name "data" (file-name-directory load-file-name))
>>   (file-name-as-directory (expand-file-name "data" (file-name-directory load-file-name)))
>>   (defconst gnus-mock-data-dir (file-name-as-directory (expand-file-name "data" (file-name-directory load-file-name))) "Source directory for Gnus mock data.")
>>   eval-region(1 10407)  ; Reading at buffer position 5125
>>
>> Hmm...
>
> Ah, this was a question I had when I was first writing gnus-mock --
> load-file-name was reliably present for all my tests, but they were only
> on Linux. Can you tell me what system you're running? Presumably
> there are better alternatives to load-file-name under different
> circumstances.

Hey do let me know what system you're running! If I do:

1. emacs -Q
2. M-x load-file -> ~/.emacs.d/elpa/gnus-mock-0.4.2/gnus-mock.el

Then gnus-mock-data-dir is set correctly. I don't know what else might
be going wrong!

Eric

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Hey do let me know what system you're running!

Sorry for the delay.  Ok, Debian testing here.

> If I do:
>
> 1. emacs -Q
> 2. M-x load-file -> ~/.emacs.d/elpa/gnus-mock-0.4.2/gnus-mock.el
>
> Then gnus-mock-data-dir is set correctly. I don't know what else might
> be going wrong!

I tried again.  I get gnus-mock-data-dir ->
"/home/micha/.emacs.d/elpa/gnus-mock-0.4.2/data/".  When I do M-x
gnus-mock-start I currently get this error:

Debugger entered--Lisp error: (file-missing "Setting current directory" "No such file or directory" "/home/lisp/gnus")
  make-process(:name "gnus-mock" :buffer nil :command ("emacs" "-Q" "--load" "/tmp/emacs-gnus-mock-jvYPdC/init.el") :stderr "*gnus mock errors*")
  (let ((default-directory (expand-file-name "lisp/gnus" (file-name-as-directory (expand-file-name "../.." gnus-mock-emacs-program))))) (make-process :name "gnus-mock" :buffer nil :command (list gnus-mock-emacs-program "-Q" "--load" init-file) :stderr "*gnus mock errors*"))
  (let ((init-file (expand-file-name "init.el" mock-tmp-dir))) (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (let ((standard-output ...) (print-circle nil)) (prin1 (list ... ... mock-tmp-dir ... "mockturtle" ... ... ... mock-tmp-dir ... ... ... ... ... ... ... ... ... ...)) (princ "\n\n") (prin1 '...) (princ "\n") (prin1 (list ... ... gnus-mock-data-dir)) (if gnus-mock-cleanup-p (progn ... ...)) (if gnus-mock-use-images (progn ... ...)) (write-file init-file))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))) (copy-directory gnus-mock-data-dir (file-name-as-directory mock-tmp-dir) nil nil t) (mapc #'(lambda (path) (make-directory path t)) (mapcar #'(lambda (dir) (format "%s/test/%s" mock-tmp-dir dir)) '("Welcome/new" "Welcome/tmp" "Welcome/.nnmaildir/marks" "incoming/tmp" "incoming/new" "incoming/cur" "incoming/.nnmaildir/marks" "incoming/.nnmaildir/nov" "mails/tmp" "mails/new" "mails/.nnmaildir/marks"))) (make-directory (format "%s/drafts/drafts" mock-tmp-dir) t) (make-directory (format "%s/drafts/queue" mock-tmp-dir)) (if gnus-mock-init-file (progn (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (insert-file-contents gnus-mock-init-file) (append-to-file ... ... init-file)) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))))) (if gnus-mock-gnus-file (progn (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (insert-file-contents gnus-mock-gnus-file) (append-to-file ... ... ...)) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))))) (if gnus-mock-dovecot-imap-program (progn (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (insert "\n\n") (prin1 ... ...) (append-to-file ... ... ...)) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))))) (save-current-buffer (set-buffer (find-file-noselect (expand-file-name ".newsrc.eld" mock-tmp-dir))) (while (re-search-forward "REPLACE_ME" (point-max) t) (replace-match mock-tmp-dir t)) (if gnus-mock-dovecot-imap-program (progn (goto-char (point-max)) (insert "\n\n") (prin1 '(setq gnus-newsrc-alist (append gnus-newsrc-alist ...)) (current-buffer)))) (basic-save-buffer)) (let ((default-directory (expand-file-name "lisp/gnus" (file-name-as-directory (expand-file-name "../.." gnus-mock-emacs-program))))) (make-process :name "gnus-mock" :buffer nil :command (list gnus-mock-emacs-program "-Q" "--load" init-file) :stderr "*gnus mock errors*")))
  (condition-case err (let ((init-file (expand-file-name "init.el" mock-tmp-dir))) (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (let (... ...) (prin1 ...) (princ "\n\n") (prin1 ...) (princ "\n") (prin1 ...) (if gnus-mock-cleanup-p ...) (if gnus-mock-use-images ...) (write-file init-file))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))) (copy-directory gnus-mock-data-dir (file-name-as-directory mock-tmp-dir) nil nil t) (mapc #'(lambda (path) (make-directory path t)) (mapcar #'(lambda (dir) (format "%s/test/%s" mock-tmp-dir dir)) '("Welcome/new" "Welcome/tmp" "Welcome/.nnmaildir/marks" "incoming/tmp" "incoming/new" "incoming/cur" "incoming/.nnmaildir/marks" "incoming/.nnmaildir/nov" "mails/tmp" "mails/new" "mails/.nnmaildir/marks"))) (make-directory (format "%s/drafts/drafts" mock-tmp-dir) t) (make-directory (format "%s/drafts/queue" mock-tmp-dir)) (if gnus-mock-init-file (progn (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn ... ...) (and ... ...)))))) (if gnus-mock-gnus-file (progn (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn ... ...) (and ... ...)))))) (if gnus-mock-dovecot-imap-program (progn (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn ... ... ...) (and ... ...)))))) (save-current-buffer (set-buffer (find-file-noselect (expand-file-name ".newsrc.eld" mock-tmp-dir))) (while (re-search-forward "REPLACE_ME" (point-max) t) (replace-match mock-tmp-dir t)) (if gnus-mock-dovecot-imap-program (progn (goto-char (point-max)) (insert "\n\n") (prin1 '(setq gnus-newsrc-alist ...) (current-buffer)))) (basic-save-buffer)) (let ((default-directory (expand-file-name "lisp/gnus" (file-name-as-directory (expand-file-name "../.." gnus-mock-emacs-program))))) (make-process :name "gnus-mock" :buffer nil :command (list gnus-mock-emacs-program "-Q" "--load" init-file) :stderr "*gnus mock errors*"))) ((debug error) (if (and gnus-mock-cleanup-p (file-exists-p mock-tmp-dir)) (progn (delete-directory mock-tmp-dir t))) (signal (car err) (cdr err))))
  (let ((mock-tmp-dir (make-temp-file "emacs-gnus-mock-" t))) (condition-case err (let ((init-file (expand-file-name "init.el" mock-tmp-dir))) (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (let ... ... ... ... ... ... ... ... ...)) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))) (copy-directory gnus-mock-data-dir (file-name-as-directory mock-tmp-dir) nil nil t) (mapc #'(lambda (path) (make-directory path t)) (mapcar #'(lambda (dir) (format "%s/test/%s" mock-tmp-dir dir)) '("Welcome/new" "Welcome/tmp" "Welcome/.nnmaildir/marks" "incoming/tmp" "incoming/new" "incoming/cur" "incoming/.nnmaildir/marks" "incoming/.nnmaildir/nov" "mails/tmp" "mails/new" "mails/.nnmaildir/marks"))) (make-directory (format "%s/drafts/drafts" mock-tmp-dir) t) (make-directory (format "%s/drafts/queue" mock-tmp-dir)) (if gnus-mock-init-file (progn (let ((temp-buffer ...)) (save-current-buffer (set-buffer temp-buffer) (unwind-protect ... ...))))) (if gnus-mock-gnus-file (progn (let ((temp-buffer ...)) (save-current-buffer (set-buffer temp-buffer) (unwind-protect ... ...))))) (if gnus-mock-dovecot-imap-program (progn (let ((temp-buffer ...)) (save-current-buffer (set-buffer temp-buffer) (unwind-protect ... ...))))) (save-current-buffer (set-buffer (find-file-noselect (expand-file-name ".newsrc.eld" mock-tmp-dir))) (while (re-search-forward "REPLACE_ME" (point-max) t) (replace-match mock-tmp-dir t)) (if gnus-mock-dovecot-imap-program (progn (goto-char (point-max)) (insert "\n\n") (prin1 '... (current-buffer)))) (basic-save-buffer)) (let ((default-directory (expand-file-name "lisp/gnus" (file-name-as-directory ...)))) (make-process :name "gnus-mock" :buffer nil :command (list gnus-mock-emacs-program "-Q" "--load" init-file) :stderr "*gnus mock errors*"))) ((debug error) (if (and gnus-mock-cleanup-p (file-exists-p mock-tmp-dir)) (progn (delete-directory mock-tmp-dir t))) (signal (car err) (cdr err)))))
  gnus-mock-start()
  funcall-interactively(gnus-mock-start)
  call-interactively(gnus-mock-start record nil)
  command-execute(gnus-mock-start record)
  execute-extended-command(nil "gnus-mock-start" "gnus-mock")
  funcall-interactively(execute-extended-command nil "gnus-mock-start" "gnus-mock")
  call-interactively(execute-extended-command nil nil)
  command-execute(execute-extended-command)


Regards,

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

On 10/14/19 11:53 AM, Michael Heerdegen wrote:
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Hey do let me know what system you're running!
>
> Sorry for the delay.  Ok, Debian testing here.
>
>> If I do:
>>
>> 1. emacs -Q
>> 2. M-x load-file -> ~/.emacs.d/elpa/gnus-mock-0.4.2/gnus-mock.el
>>
>> Then gnus-mock-data-dir is set correctly. I don't know what else might
>> be going wrong!
>
> I tried again.  I get gnus-mock-data-dir ->
> "/home/micha/.emacs.d/elpa/gnus-mock-0.4.2/data/".  When I do M-x
> gnus-mock-start I currently get this error:
>
> Debugger entered--Lisp error: (file-missing "Setting current
> directory" "No such file or directory" "/home/lisp/gnus")
>   make-process(:name "gnus-mock" :buffer nil :command ("emacs" "-Q"
> "--load" "/tmp/emacs-gnus-mock-jvYPdC/init.el") :stderr "*gnus mock
> errors*")

Thanks! Okay, it looks like I made a dumb assumption about how people
would be using this -- I assumed this package would only be used for
*developing* Gnus, and thus `gnus-mock-emacs-program' would always be
pointed at a "src/emacs" executable in a build tree. I don't even have
emacs installed as a package on my machine, and only ever run it from
one of the source directories. But of course that's not the code I wrote
-- by default it will find an installed version of emacs if you have
one, and then this default-directory setting will fail.

In the interim, you can set `gnus-mock-emacs-program' to point to an
emacs executable in a source tree checkout, and it ought to work fine. I
should probably change the code to require that, or else figure out how
to start up successfully from an installed emacs program.

Eric

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> In the interim, you can set `gnus-mock-emacs-program' to point to an
> emacs executable in a source tree checkout, and it ought to work fine.

Would it make sense to change the default of gnus-mock-emacs-program to
something like (expand-file-name "src/emacs" source-directory) or
similar?


Oh, and you also noted that part

> Sorry for the bad news - but my registry marks are disappearing again
> :-(  Just after restarting Gnus they are gone.

in one of my previous messages, right?


Regards,

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

On 10/15/19 16:28 PM, Michael Heerdegen wrote:
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> In the interim, you can set `gnus-mock-emacs-program' to point to an
>> emacs executable in a source tree checkout, and it ought to work fine.
>
> Would it make sense to change the default of gnus-mock-emacs-program to
> something like (expand-file-name "src/emacs" source-directory) or
> similar?

Yes, that would probably work. But it still make assumptions about how
the user is working on emacs. Probably I should just default the option
to nil and ask users to always set it explicitly.

>
> Oh, and you also noted that part
>
>> Sorry for the bad news - but my registry marks are disappearing again
>> :-(  Just after restarting Gnus they are gone.
>
> in one of my previous messages, right?

Ha, yes, I got distracted. Unfortunately I haven't been able to
reproduce so far. I start Gnus with this in the init file:

(setq gnus-registry-max-entries 10)
(gnus-registry-initialize)
(defalias 'gnus-user-format-function-M 'gnus-registry-article-marks-to-chars)
(setq gnus-summary-line-format "%uM%U%R%z%I%(%[%4L: %-23,23f%]%) %s\n")

Mark some articles as Important or ToDo, stop and restart Gnus, and
everything is still present. I opened a bunch of other groups, so that
pruning (if that were the issue) would have dumped the entries with the
marks, but they are kept as expected.

Anything other way to narrow this down?

eric

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> > Would it make sense to change the default of gnus-mock-emacs-program to
> > something like (expand-file-name "src/emacs" source-directory) or
> > similar?
>
> Yes, that would probably work. But it still make assumptions about how
> the user is working on emacs. Probably I should just default the option
> to nil and ask users to always set it explicitly.

Wouldn't that be the worst default since people are expected to use
emacs -Q to start gnus-mock?


Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> > Oh, and you also noted that part
> >
> >> Sorry for the bad news - but my registry marks are disappearing again
> >> :-(  Just after restarting Gnus they are gone.
> >
> > in one of my previous messages, right?

> Anything other way to narrow this down?

Hmm, currently I can't reproduce either.  I'll start using registry
marks again in real life and see if it works reliable now.

One little thing that bothers me however: when I hit M M i, or when I
remove a mark with M M I, the article line gets doubled: one line has
the mark, and the other doesn't.  Do you see that too?


Regards,

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> > Would it make sense to change the default of gnus-mock-emacs-program to
>> > something like (expand-file-name "src/emacs" source-directory) or
>> > similar?
>>
>> Yes, that would probably work. But it still make assumptions about how
>> the user is working on emacs. Probably I should just default the option
>> to nil and ask users to always set it explicitly.
>
> Wouldn't that be the worst default since people are expected to use
> emacs -Q to start gnus-mock?

People are expected to use `gnus-mock-start' to start it, which does
"emacs -Q" internally. The only requirement is that the emacs be run
from a build directory, and not an installation.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> > Oh, and you also noted that part
>> >
>> >> Sorry for the bad news - but my registry marks are disappearing again
>> >> :-(  Just after restarting Gnus they are gone.
>> >
>> > in one of my previous messages, right?
>
>> Anything other way to narrow this down?
>
> Hmm, currently I can't reproduce either.  I'll start using registry
> marks again in real life and see if it works reliable now.
>
> One little thing that bothers me however: when I hit M M i, or when I
> remove a mark with M M I, the article line gets doubled: one line has
> the mark, and the other doesn't.  Do you see that too?

Whoa, definitely not, and that's not how it's supposed to work. I wonder
what could cause that... Do you have
`gnus-summary-display-while-building' set to anything?

Oh, I remember you've got some advice on message limiting? I forget
what, exactly. Maybe try disabling that and see if it has anything to do
with it?

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> People are expected to use `gnus-mock-start' to start it, which does
> "emacs -Q" internally.

Ah, I remember.

> The only requirement is that the emacs be run
> from a build directory, and not an installation.

I tried it from my normal Emacs session and got this:

Debugger entered--Lisp error: (buffer-read-only #<buffer .newsrc.eld>)
  replace-match("/tmp/emacs-gnus-mock-738lXO" t)
  gnus-mock-start()
  funcall-interactively(gnus-mock-start)
  call-interactively(gnus-mock-start record nil)
  command-execute(gnus-mock-start record)

Is that because it is an installed Emacs, or something else?  Should I
investigate?


Thanks,

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Whoa, definitely not, and that's not how it's supposed to work. I wonder
> what could cause that... Do you have
> `gnus-summary-display-while-building' set to anything?

No.

> Oh, I remember you've got some advice on message limiting? I forget
> what, exactly. Maybe try disabling that and see if it has anything to do
> with it?

I currently have no such advices enabled afaict.

How can I debug this - where is the old article line supposed to be
removed?  I see that line duplication happens in
`gnus-summary-update-article', but it's quite complex.


Thanks,

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Hi again,

I played around a bit more, and now I can't reproduce any more.  It
could have been related to my advice of
`gnus-alter-articles-to-read-function'.


Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> People are expected to use `gnus-mock-start' to start it, which does
>> "emacs -Q" internally.
>
> Ah, I remember.
>
>> The only requirement is that the emacs be run
>> from a build directory, and not an installation.
>
> I tried it from my normal Emacs session and got this:
>
> Debugger entered--Lisp error: (buffer-read-only #<buffer .newsrc.eld>)
>   replace-match("/tmp/emacs-gnus-mock-738lXO" t)
>   gnus-mock-start()
>   funcall-interactively(gnus-mock-start)
>   call-interactively(gnus-mock-start record nil)
>   command-execute(gnus-mock-start record)
>
> Is that because it is an installed Emacs, or something else?  Should I
> investigate?

Man, who would have guessed this would be so difficult... I have no idea
why a data file that ships with the package would become read-only.
I suppose I could wrap that in `inhibit-read-only', but if it's read
only for a reason that will only fail later down the line. Can you check
two things, and then I'll give up for now?

1. Put a breakpoint or edebug or otherwise slow the function down, and
after the data directory has been copied to /tmp, make sure that it
belongs to your user and you have the right permissions?

2. Try this version of gnus-mock-start and see if it works:

(defun gnus-mock-start ()
  (interactive)
  (let ((mock-tmp-dir (make-temp-file "emacs-gnus-mock-" t)))
    (condition-case-unless-debug err
	(let ((init-file (expand-file-name "init.el" mock-tmp-dir)))
	  (with-temp-buffer
	    (let ((standard-output (current-buffer))
		  (print-circle nil))
	      (prin1
	       `(setq gnus-home-directory ,mock-tmp-dir
		      init-file-user "mockturtle"
		      sendmail-program
		      ,(expand-file-name gnus-mock-sendmail-program
					 mock-tmp-dir)
		      message-directory ,mock-tmp-dir
		      gnus-startup-file
		      ,(expand-file-name ".newsrc" mock-tmp-dir)
		      gnus-init-file
		      ,(expand-file-name ".gnus.el" mock-tmp-dir)
		      nndraft-directory
		      ,(expand-file-name "drafts/" mock-tmp-dir)
		      gnus-agent-directory
		      ,(expand-file-name "agent/" mock-tmp-dir)
		      gnus-directory
		      ,(expand-file-name "News/" mock-tmp-dir)))
	      (princ "\n\n")
	      ;; Constant that can be checked if we need to know it's a mock
	      ;; session.
	      (prin1 '(defconst gnus-mock-p t))
	      (princ "\n")
	      ;; Constant for use in `gnus-mock-reload', which is defined in
	      ;; the .gnus.el startup file.
 	      (prin1 `(defconst gnus-mock-data-dir ,gnus-mock-data-dir))
	      (when gnus-mock-cleanup-p
		(princ "\n")
		(prin1 `(add-hook 'kill-emacs-hook
				  (lambda () (delete-directory
					      ,mock-tmp-dir t)))))
	      (when gnus-mock-use-images
		(princ "\n")
		(prin1 `(add-to-list 'load-path
				     ,(format "%s/data" mock-tmp-dir))))
	      (write-file init-file)))
	  ;; Put our data and config in place.
	  (copy-directory
	   gnus-mock-data-dir
	   (file-name-as-directory mock-tmp-dir) nil nil t)
	  ;; Git doesn't let us commit empty directories, so create our
	  ;; necessary empty maildir bits, and draft directories.
	  (mapc (lambda (path) (make-directory path t))
		(mapcar (lambda (dir)
			  (format "%s/test/%s" mock-tmp-dir dir))
			'("Welcome/new" "Welcome/tmp" "Welcome/.nnmaildir/marks"
			  "incoming/tmp" "incoming/new" "incoming/cur"
			  "incoming/.nnmaildir/marks" "incoming/.nnmaildir/nov"
			  "mails/tmp" "mails/new" "mails/.nnmaildir/marks")))
	  (make-directory (format "%s/drafts/drafts" mock-tmp-dir) t)
	  (make-directory (format "%s/drafts/queue" mock-tmp-dir))
	  ;; Possibly insert additional config.
	  (when gnus-mock-init-file
	    (with-temp-buffer
	      (insert-file-contents gnus-mock-init-file)
	      (append-to-file
	       (point-min) (point-max) init-file)))
	  (when gnus-mock-gnus-file
	    (with-temp-buffer
	      (insert-file-contents gnus-mock-gnus-file)
	      (append-to-file
	       (point-min) (point-max)
	       (expand-file-name ".gnus.el" mock-tmp-dir))))
	  ;; Possibly add an nnimap server.
	  (when gnus-mock-dovecot-imap-program
	    (with-temp-buffer
	      (insert "\n\n")
	      (prin1
	       `(add-to-list
		 'gnus-secondary-select-methods
		 (quote (nnimap
			 "Mocky"
			 (nnimap-stream shell)
			 (nnimap-shell-program
			  ,(concat
			    gnus-mock-dovecot-imap-program
			    (format " -o mail_location=maildir:%s/imapmail/mail"
				    mock-tmp-dir))))))
	       (current-buffer))
	      (append-to-file
	       (point-min) (point-max)
	       (expand-file-name ".gnus.el" mock-tmp-dir))))
	  ;; There are absolute paths in the .newsrc.eld file, so doctor
	  ;; that file.
	  (let ((inhibit-read-only t))
	   (with-current-buffer (find-file-noselect
				 (expand-file-name ".newsrc.eld" mock-tmp-dir))
	     (while (re-search-forward "REPLACE_ME" (point-max) t)
	       (replace-match mock-tmp-dir t))
	     (when gnus-mock-dovecot-imap-program
	       (goto-char (point-max))
	       (insert "\n\n")
	       (prin1
		'(setq
		  gnus-newsrc-alist
		  (append
		   gnus-newsrc-alist
		   '(("nnimap+Mocky:INBOX" 3 nil ((unexist) (seen (1 . 32)))
		      "nnimap:Mocky" ((modseq . "33") (uidvalidity . "1541087103")
				      (active 1 . 32)
				      (permanent-flags %Answered %Flagged %Deleted
						       %Seen %Draft %*)))
		     ("nnimap+Mocky:emacs-devel" 3 nil ((unexist 0))
		      "nnimap:Mocky" ((modseq . "21") (uidvalidity . "1541087104")
				      (active 0 . 20)
				      (permanent-flags %Answered %Flagged %Deleted
						       %Seen %Draft %*)))
		     ("nnimap+Mocky:\320\237\321\200\320\270\320\262\320\265\321\202\320\274\320\270\321\200"
		      3 nil ((unexist) (seen 1))
		      "nnimap:Mocky" ((modseq . "2") (uidvalidity . "1541087105")
				      (active 1 . 1)
				      (permanent-flags %Answered %Flagged %Deleted
						       %Seen %Draft %*))))))
		(current-buffer)))
	     (basic-save-buffer)))
	  (let ((default-directory
		  (expand-file-name
		   "lisp/gnus"
		   (file-name-as-directory
		    (expand-file-name "../.."
				      gnus-mock-emacs-program)))))
	    (make-process :name "gnus-mock" :buffer nil
			  :command (list gnus-mock-emacs-program
					 "-Q" "--load" init-file)
			  :stderr "*gnus mock errors*")))
      (error (when (and gnus-mock-cleanup-p
			(file-exists-p mock-tmp-dir))
	       (delete-directory mock-tmp-dir t))
	     (signal (car err) (cdr err))))))

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

On 10/17/19 12:23 PM, Michael Heerdegen wrote:
> Hi again,
>
> I played around a bit more, and now I can't reproduce any more.  It
> could have been related to my advice of
> `gnus-alter-articles-to-read-function'.

Ah, right -- that was the advice I was remembering. Well, ping if it
comes back...

bug#33005: 27.0.50; Data loss with Gnus registry

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Hmm, currently I can't reproduce either.

It amuses me to see the verb "reproduce" used intransitively in this
context, because what that means is that you are unable to produce
offspring.  I don't think that's what you meant ;-).

With a bug, the thing to say is "I can't reproduce it."

Of course, people understand anyway.

-- 
Dr Richard Stallman
Founder, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> 1. Put a breakpoint or edebug or otherwise slow the function down, and
> after the data directory has been copied to /tmp, make sure that it
> belongs to your user and you have the right permissions?

That's the case.

> 2. Try this version of gnus-mock-start and see if it works:

I did something different.  Seems that so-long is to blame:

| Debugger entered--entering a function:
| * hack-local-variables(no-mode)
|   run-mode-hooks(so-long-mode-hook)
|   #f(compiled-function () (interactive nil) #<bytecode 0x156db2ff061d>)()
|   apply(#f(compiled-function () (interactive nil) #<bytecode 0x156db2ff061d>) nil)
|   so-long-mode()
|   so-long()
|   so-long--set-auto-mode((closure (t) (&optional keep-mode-if-same) "Select major mode appropriate for current buffer.\n..." (let ((try-locals (not (inhibit-local-variables-p))) end done mode modes) (save-excursion (goto-char (point-min)) (skip-chars-forward " \11\n") (and enable-local-variables try-locals (setq end (set-auto-mode-1)) (if (save-excursion (search-forward ":" end t)) (while (let ... ...) (skip-chars-forward " \11") (let ... ... ... ...)) (setq modes (cons ... modes))))) (if modes (catch 'nop (let ((--dolist-tail-- ...)) (while --dolist-tail-- (let ... ... ...))))) (and (not done) enable-local-variables local-enable-local-variables try-locals (setq mode (hack-local-variables t)) (not (memq mode modes)) (if (not (functionp mode)) (message "Ignoring unknown mode `%s'" mode) (setq done t) (set-auto-mode-0 mode keep-mode-if-same))) (and (not done) (setq mode (save-excursion (goto-char (point-min)) (if (looking-at auto-mode-interpreter-regexp) (match-string 2)))) (setq done (assoc-default (file-name-nondirectory mode) (mapcar #'... interpreter-mode-alist) #'string-match-p)) (set-auto-mode-0 done keep-mode-if-same)) (if done nil (if (setq done (save-excursion (goto-char ...) (save-restriction ... ...))) (set-auto-mode-0 done keep-mode-if-same))) (if done nil (if buffer-file-name (let ((name buffer-file-name) (remote-id ...) (case-insensitive-p ...)) (setq name (file-name-sans-versions name)) (if (and ... ...) (progn ...)) (while name (setq mode ...) (if ... ... ...) (if mode ...))))) (if done nil (if (setq done (save-excursion (goto-char ...) (save-restriction ... ...))) (set-auto-mode-0 done keep-mode-if-same))) (if done nil (set-buffer-major-mode (current-buffer))))) t)
|   apply(so-long--set-auto-mode (closure (t) (&optional keep-mode-if-same) "Select major mode appropriate for current buffer.\n..." (let ((try-locals (not (inhibit-local-variables-p))) end done mode modes) (save-excursion (goto-char (point-min)) (skip-chars-forward " \11\n") (and enable-local-variables try-locals (setq end (set-auto-mode-1)) (if (save-excursion (search-forward ":" end t)) (while (let ... ...) (skip-chars-forward " \11") (let ... ... ... ...)) (setq modes (cons ... modes))))) (if modes (catch 'nop (let ((--dolist-tail-- ...)) (while --dolist-tail-- (let ... ... ...))))) (and (not done) enable-local-variables local-enable-local-variables try-locals (setq mode (hack-local-variables t)) (not (memq mode modes)) (if (not (functionp mode)) (message "Ignoring unknown mode `%s'" mode) (setq done t) (set-auto-mode-0 mode keep-mode-if-same))) (and (not done) (setq mode (save-excursion (goto-char (point-min)) (if (looking-at auto-mode-interpreter-regexp) (match-string 2)))) (setq done (assoc-default (file-name-nondirectory mode) (mapcar #'... interpreter-mode-alist) #'string-match-p)) (set-auto-mode-0 done keep-mode-if-same)) (if done nil (if (setq done (save-excursion (goto-char ...) (save-restriction ... ...))) (set-auto-mode-0 done keep-mode-if-same))) (if done nil (if buffer-file-name (let ((name buffer-file-name) (remote-id ...) (case-insensitive-p ...)) (setq name (file-name-sans-versions name)) (if (and ... ...) (progn ...)) (while name (setq mode ...) (if ... ... ...) (if mode ...))))) (if done nil (if (setq done (save-excursion (goto-char ...) (save-restriction ... ...))) (set-auto-mode-0 done keep-mode-if-same))) (if done nil (set-buffer-major-mode (current-buffer))))) t)
|   #f(advice-wrapper :around (closure (t) (&optional keep-mode-if-same) "Select major mode appropriate for current buffer.\n\nTo find the right major mode, this function checks for a -*- mode tag\nchecks for a `mode:' entry in the Local Variables section of the file,\nchecks if it uses an interpreter listed in `interpreter-mode-alist',\nmatches the buffer beginning against `magic-mode-alist',\ncompares the filename against the entries in `auto-mode-alist',\nthen matches the buffer beginning against `magic-fallback-mode-alist'.\n\nIf `enable-local-variables' is nil, or if the file name matches\n`inhibit-local-variables-regexps', this function does not check\nfor any mode: tag anywhere in the file.  If `local-enable-local-variables'\nis nil, then the only mode: tag that can be relevant is a -*- one.\n\nIf the optional argument KEEP-MODE-IF-SAME is non-nil, then we\nset the major mode only if that would change it.  In other words\nwe don't actually set it to the same mode the buffer already has." (let ((try-locals (not (inhibit-local-variables-p))) end done mode modes) (save-excursion (goto-char (point-min)) (skip-chars-forward " \11\n") (and enable-local-variables try-locals (setq end (set-auto-mode-1)) (if (save-excursion (search-forward ":" end t)) (while (let ((case-fold-search t)) (or (and (looking-at "mode:") (goto-char (match-end 0))) (re-search-forward "[ \11;]mode:" end t))) (skip-chars-forward " \11") (let ((beg (point))) (if (search-forward ";" end t) (forward-char -1) (goto-char end)) (skip-chars-backward " \11") (setq modes (cons (intern (concat (downcase (buffer-substring beg (point))) "-mode")) modes)))) (setq modes (cons (intern (concat (downcase (buffer-substring (point) end)) "-mode")) modes))))) (if modes (catch 'nop (let ((--dolist-tail-- (nreverse modes))) (while --dolist-tail-- (let ((mode (car --dolist-tail--))) (if (not (functionp mode)) (message "Ignoring unknown mode `%s'" mode) (setq done t) (or (set-auto-mode-0 mode keep-mode-if-same) (throw 'nop nil))) (setq --dolist-tail-- (cdr --dolist-tail--))))))) (and (not done) enable-local-variables local-enable-local-variables try-locals (setq mode (hack-local-variables t)) (not (memq mode modes)) (if (not (functionp mode)) (message "Ignoring unknown mode `%s'" mode) (setq done t) (set-auto-mode-0 mode keep-mode-if-same))) (and (not done) (setq mode (save-excursion (goto-char (point-min)) (if (looking-at auto-mode-interpreter-regexp) (match-string 2)))) (setq done (assoc-default (file-name-nondirectory mode) (mapcar #'(lambda (e) (cons (format "\\`%s\\'" (car e)) (cdr e))) interpreter-mode-alist) #'string-match-p)) (set-auto-mode-0 done keep-mode-if-same)) (if done nil (if (setq done (save-excursion (goto-char (point-min)) (save-restriction (narrow-to-region (point-min) (min (point-max) (+ (point-min) magic-mode-regexp-match-limit))) (assoc-default nil magic-mode-alist #'(lambda (re _dummy) (cond ((functionp re) (funcall re)) ((stringp re) (let ((case-fold-search nil)) (looking-at re))) (t (error "Problem in magic-mode-alist with element %s" re)))))))) (set-auto-mode-0 done keep-mode-if-same))) (if done nil (if buffer-file-name (let ((name buffer-file-name) (remote-id (file-remote-p buffer-file-name)) (case-insensitive-p (file-name-case-insensitive-p buffer-file-name))) (setq name (file-name-sans-versions name)) (if (and (stringp remote-id) (string-match (regexp-quote remote-id) name)) (progn (setq name (substring name (match-end 0))))) (while name (setq mode (if case-insensitive-p (let ((case-fold-search t)) (assoc-default name auto-mode-alist 'string-match)) (or (let ((case-fold-search nil)) (assoc-default name auto-mode-alist 'string-match)) (and auto-mode-case-fold (let ((case-fold-search t)) (assoc-default name auto-mode-alist 'string-match)))))) (if (and mode (consp mode) (car (cdr mode))) (setq mode (car mode) name (substring name 0 (match-beginning 0))) (setq name nil)) (if mode (progn (set-auto-mode-0 mode keep-mode-if-same) (setq done t))))))) (if done nil (if (setq done (save-excursion (goto-char (point-min)) (save-restriction (narrow-to-region (point-min) (min (point-max) (+ (point-min) magic-mode-regexp-match-limit))) (assoc-default nil magic-fallback-mode-alist #'(lambda (re _dummy) (cond ((functionp re) (funcall re)) ((stringp re) (let ((case-fold-search nil)) (looking-at re))) (t (error "Problem with magic-fallback-mode-alist element: %s" re)))))))) (set-auto-mode-0 done keep-mode-if-same))) (if done nil (set-buffer-major-mode (current-buffer))))) so-long--set-auto-mode)(t)
|   apply(#f(advice-wrapper :around (closure (t) (&optional keep-mode-if-same) "Select major mode appropriate for current buffer.\n..." (let ((try-locals (not (inhibit-local-variables-p))) end done mode modes) (save-excursion (goto-char (point-min)) (skip-chars-forward " \11\n") (and enable-local-variables try-locals (setq end (set-auto-mode-1)) (if (save-excursion ...) (while ... ... ...) (setq modes ...)))) (if modes (catch 'nop (let (...) (while --dolist-tail-- ...)))) (and (not done) enable-local-variables local-enable-local-variables try-locals (setq mode (hack-local-variables t)) (not (memq mode modes)) (if (not (functionp mode)) (message "Ignoring unknown mode `%s'" mode) (setq done t) (set-auto-mode-0 mode keep-mode-if-same))) (and (not done) (setq mode (save-excursion (goto-char ...) (if ... ...))) (setq done (assoc-default (file-name-nondirectory mode) (mapcar ... interpreter-mode-alist) #'string-match-p)) (set-auto-mode-0 done keep-mode-if-same)) (if done nil (if (setq done (save-excursion ... ...)) (set-auto-mode-0 done keep-mode-if-same))) (if done nil (if buffer-file-name (let (... ... ...) (setq name ...) (if ... ...) (while name ... ... ...)))) (if done nil (if (setq done (save-excursion ... ...)) (set-auto-mode-0 done keep-mode-if-same))) (if done nil (set-buffer-major-mode (current-buffer))))) so-long--set-auto-mode) t)
|   set-auto-mode(t)
|   (let ((old major-mode)) (set-auto-mode t) (or (eq old major-mode) (hack-local-variables)))
|   (or (not change-major-mode-with-file-name) (get major-mode 'mode-class) (and old-try-locals (not (inhibit-local-variables-p)) (hack-local-variables t)) (let ((old major-mode)) (set-auto-mode t) (or (eq old major-mode) (hack-local-variables))))
|   (condition-case nil (or (not change-major-mode-with-file-name) (get major-mode 'mode-class) (and old-try-locals (not (inhibit-local-variables-p)) (hack-local-variables t)) (let ((old major-mode)) (set-auto-mode t) (or (eq old major-mode) (hack-local-variables)))) (error nil))
|   (let (truename old-try-locals) (if filename (setq filename (if (string-equal filename "") nil (expand-file-name filename)))) (if filename (progn (setq truename (file-truename filename)) (if find-file-visit-truename (setq filename truename)))) (if filename (let ((new-name (file-name-nondirectory filename))) (if (string= new-name "") (error "Empty file name")))) (let ((buffer (and filename (find-buffer-visiting filename)))) (and buffer (not (eq buffer (current-buffer))) (not no-query) (not (y-or-n-p (format "A buffer is visiting %s; proceed? " filename))) (user-error "Aborted"))) (or (equal filename buffer-file-name) (progn (and filename (lock-buffer filename)) (unlock-buffer))) (setq old-try-locals (not (inhibit-local-variables-p)) buffer-file-name filename) (if filename (let ((new-name (file-name-nondirectory buffer-file-name))) (setq default-directory (file-name-directory buffer-file-name)) (or (string= new-name (buffer-name)) (rename-buffer new-name t)))) (setq buffer-backed-up nil) (or along-with-file (clear-visited-file-modtime)) (if truename (progn (setq buffer-file-truename (abbreviate-file-name truename)) (if find-file-visit-truename (setq buffer-file-name truename)))) (setq buffer-file-number (if filename (nthcdr 10 (file-attributes buffer-file-name)) nil)) (kill-local-variable 'write-file-functions) (kill-local-variable 'local-write-file-hooks) (kill-local-variable 'revert-buffer-function) (kill-local-variable 'backup-inhibited) (if vc-mode (setq buffer-read-only nil)) (kill-local-variable 'vc-mode) (and buffer-file-name backup-enable-predicate (not (funcall backup-enable-predicate buffer-file-name)) (progn (make-local-variable 'backup-inhibited) (setq backup-inhibited t))) (let ((oauto buffer-auto-save-file-name)) (cond ((null filename) (setq buffer-auto-save-file-name nil)) ((not buffer-auto-save-file-name) (and buffer-file-name auto-save-default (auto-save-mode t))) (t (setq buffer-auto-save-file-name (make-auto-save-file-name)))) (and oauto buffer-auto-save-file-name (file-exists-p oauto) (rename-file oauto buffer-auto-save-file-name t))) (and buffer-file-name (not along-with-file) (set-buffer-modified-p t)) (condition-case nil (or (not change-major-mode-with-file-name) (get major-mode 'mode-class) (and old-try-locals (not (inhibit-local-variables-p)) (hack-local-variables t)) (let ((old major-mode)) (set-auto-mode t) (or (eq old major-mode) (hack-local-variables)))) (error nil)) (run-hooks 'after-set-visited-file-name-hook))
|   (closure (t) (filename &optional no-query along-with-file) "Change name of file visited in current buff..." (interactive "FSet visited file name: ") (if (buffer-base-buffer) (error "An indirect buffer cannot visit a file")) (let (truename old-try-locals) (if filename (setq filename (if (string-equal filename "") nil (expand-file-name filename)))) (if filename (progn (setq truename (file-truename filename)) (if find-file-visit-truename (setq filename truename)))) (if filename (let ((new-name ...)) (if (string= new-name "") (error "Empty file name")))) (let ((buffer (and filename ...))) (and buffer (not (eq buffer ...)) (not no-query) (not (y-or-n-p ...)) (user-error "Aborted"))) (or (equal filename buffer-file-name) (progn (and filename (lock-buffer filename)) (unlock-buffer))) (setq old-try-locals (not (inhibit-local-variables-p)) buffer-file-name filename) (if filename (let ((new-name ...)) (setq default-directory (file-name-directory buffer-file-name)) (or (string= new-name ...) (rename-buffer new-name t)))) (setq buffer-backed-up nil) (or along-with-file (clear-visited-file-modtime)) (if truename (progn (setq buffer-file-truename (abbreviate-file-name truename)) (if find-file-visit-truename (setq buffer-file-name truename)))) (setq buffer-file-number (if filename (nthcdr 10 (file-attributes buffer-file-name)) nil)) (kill-local-variable 'write-file-functions) (kill-local-variable 'local-write-file-hooks) (kill-local-variable 'revert-buffer-function) (kill-local-variable 'backup-inhibited) (if vc-mode (setq buffer-read-only nil)) (kill-local-variable 'vc-mode) (and buffer-file-name backup-enable-predicate (not (funcall backup-enable-predicate buffer-file-name)) (progn (make-local-variable 'backup-inhibited) (setq backup-inhibited t))) (let ((oauto buffer-auto-save-file-name)) (cond ((null filename) (setq buffer-auto-save-file-name nil)) ((not buffer-auto-save-file-name) (and buffer-file-name auto-save-default ...)) (t (setq buffer-auto-save-file-name ...))) (and oauto buffer-auto-save-file-name (file-exists-p oauto) (rename-file oauto buffer-auto-save-file-name t))) (and buffer-file-name (not along-with-file) (set-buffer-modified-p t)) (condition-case nil (or (not change-major-mode-with-file-name) (get major-mode 'mode-class) (and old-try-locals (not ...) (hack-local-variables t)) (let (...) (set-auto-mode t) (or ... ...))) (error nil)) (run-hooks 'after-set-visited-file-name-hook)))("/tmp/emacs-gnus-mock-WDzPUE/init.el" t)
|   apply((closure (t) (filename &optional no-query along-with-file) "Change name of file visited in current buffer to F..." (interactive "FSet visited file name: ") (if (buffer-base-buffer) (error "An indirect buffer cannot visit a file")) (let (truename old-try-locals) (if filename (setq filename (if (string-equal filename "") nil (expand-file-name filename)))) (if filename (progn (setq truename (file-truename filename)) (if find-file-visit-truename (setq filename truename)))) (if filename (let ((new-name (file-name-nondirectory filename))) (if (string= new-name "") (error "Empty file name")))) (let ((buffer (and filename (find-buffer-visiting filename)))) (and buffer (not (eq buffer (current-buffer))) (not no-query) (not (y-or-n-p (format "A buffer is visiting %s; proceed? " filename))) (user-error "Aborted"))) (or (equal filename buffer-file-name) (progn (and filename (lock-buffer filename)) (unlock-buffer))) (setq old-try-locals (not (inhibit-local-variables-p)) buffer-file-name filename) (if filename (let ((new-name (file-name-nondirectory buffer-file-name))) (setq default-directory (file-name-directory buffer-file-name)) (or (string= new-name (buffer-name)) (rename-buffer new-name t)))) (setq buffer-backed-up nil) (or along-with-file (clear-visited-file-modtime)) (if truename (progn (setq buffer-file-truename (abbreviate-file-name truename)) (if find-file-visit-truename (setq buffer-file-name truename)))) (setq buffer-file-number (if filename (nthcdr 10 (file-attributes buffer-file-name)) nil)) (kill-local-variable 'write-file-functions) (kill-local-variable 'local-write-file-hooks) (kill-local-variable 'revert-buffer-function) (kill-local-variable 'backup-inhibited) (if vc-mode (setq buffer-read-only nil)) (kill-local-variable 'vc-mode) (and buffer-file-name backup-enable-predicate (not (funcall backup-enable-predicate buffer-file-name)) (progn (make-local-variable 'backup-inhibited) (setq backup-inhibited t))) (let ((oauto buffer-auto-save-file-name)) (cond ((null filename) (setq buffer-auto-save-file-name nil)) ((not buffer-auto-save-file-name) (and buffer-file-name auto-save-default (auto-save-mode t))) (t (setq buffer-auto-save-file-name (make-auto-save-file-name)))) (and oauto buffer-auto-save-file-name (file-exists-p oauto) (rename-file oauto buffer-auto-save-file-name t))) (and buffer-file-name (not along-with-file) (set-buffer-modified-p t)) (condition-case nil (or (not change-major-mode-with-file-name) (get major-mode 'mode-class) (and old-try-locals (not (inhibit-local-variables-p)) (hack-local-variables t)) (let ((old major-mode)) (set-auto-mode t) (or (eq old major-mode) (hack-local-variables)))) (error nil)) (run-hooks 'after-set-visited-file-name-hook))) ("/tmp/emacs-gnus-mock-WDzPUE/init.el" t))
|   set-visited-file-name("/tmp/emacs-gnus-mock-WDzPUE/init.el" t)
|   (progn (if (directory-name-p filename) (setq filename (concat filename (file-name-nondirectory (or buffer-file-name (buffer-name)))))) (and confirm (file-exists-p filename) (not (and (eq (framep-on-display) 'ns) (listp last-nonmenu-event) use-dialog-box)) (or (y-or-n-p (format-message "File `%s' exists; overwrite? " filename)) (user-error "Canceled"))) (set-visited-file-name filename (not confirm)))
|   (or (null filename) (string-equal filename "") (progn (if (directory-name-p filename) (setq filename (concat filename (file-name-nondirectory (or buffer-file-name (buffer-name)))))) (and confirm (file-exists-p filename) (not (and (eq (framep-on-display) 'ns) (listp last-nonmenu-event) use-dialog-box)) (or (y-or-n-p (format-message "File `%s' exists; overwrite? " filename)) (user-error "Canceled"))) (set-visited-file-name filename (not confirm))))
|   (closure (t) (filename &optional confirm) "Write current buffer into file FILENAME.\nThis make..." (interactive (list (if buffer-file-name (read-file-name "Write file: " nil nil nil nil) (read-file-name "Write file: " default-directory (expand-file-name (file-name-nondirectory ...) default-directory) nil nil)) (not current-prefix-arg))) (or (null filename) (string-equal filename "") (progn (if (directory-name-p filename) (setq filename (concat filename (file-name-nondirectory ...)))) (and confirm (file-exists-p filename) (not (and (eq ... ...) (listp last-nonmenu-event) use-dialog-box)) (or (y-or-n-p (format-message "File `%s' exists; overwrite? " filename)) (user-error "Canceled"))) (set-visited-file-name filename (not confirm)))) (set-buffer-modified-p t) (and buffer-file-name (file-writable-p buffer-file-name) (setq buffer-read-only nil)) (save-buffer) (vc-refresh-state))("/tmp/emacs-gnus-mock-WDzPUE/init.el")
|   apply((closure (t) (filename &optional confirm) "Write current buffer into file FILENAME.\nThis make..." (interactive (list (if buffer-file-name (read-file-name "Write file: " nil nil nil nil) (read-file-name "Write file: " default-directory (expand-file-name (file-name-nondirectory ...) default-directory) nil nil)) (not current-prefix-arg))) (or (null filename) (string-equal filename "") (progn (if (directory-name-p filename) (setq filename (concat filename (file-name-nondirectory ...)))) (and confirm (file-exists-p filename) (not (and (eq ... ...) (listp last-nonmenu-event) use-dialog-box)) (or (y-or-n-p (format-message "File `%s' exists; overwrite? " filename)) (user-error "Canceled"))) (set-visited-file-name filename (not confirm)))) (set-buffer-modified-p t) (and buffer-file-name (file-writable-p buffer-file-name) (setq buffer-read-only nil)) (save-buffer) (vc-refresh-state)) "/tmp/emacs-gnus-mock-WDzPUE/init.el")
|   write-file("/tmp/emacs-gnus-mock-WDzPUE/init.el")
|   gnus-mock-start()
|   funcall-interactively(gnus-mock-start)
|   call-interactively(gnus-mock-start record nil)
|   command-execute(gnus-mock-start record)

When I delete the (buffer-read-only . t) association from
`so-long-variable-overrides' the problem is fixed.  The only so-long
thing in my init file is (global-so-long-mode 1).  Since some people
will do this and the default behavior breaks gnus-mock I guess you will
want to handle this somehow.


Regards,

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Richard Stallman <rms@gnu.org> writes:

> It amuses me to see the verb "reproduce" used intransitively in this
> context, because what that means is that you are unable to produce
> offspring.  I don't think that's what you meant ;-).

I don't recall, but I guess it wasn't what I wanted to say.  I begin to
wonder why this kind of thing happens only to me.

But this probably doesn't belong in this bug's thread - should I open a
new one?

Regards,

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> | Debugger entered--entering a function:
> | * hack-local-variables(no-mode)
> |   run-mode-hooks(so-long-mode-hook)
> |   #f(compiled-function () (interactive nil) #<bytecode 0x156db2ff061d>)()
> | apply(#f(compiled-function () (interactive nil) #<bytecode
> 0x156db2ff061d>) nil)
> |   so-long-mode()
> |   so-long()
> | so-long--set-auto-mode((closure (t) (&optional keep-mode-if-same)

Phil, what's your opinion about this?  gnus-mock.el wants to edit a file
it has opened with find-file-noselect, which is a common thing to do.
But it fails for me because I have enabled global-so-long-mode in my
init file and it makes the buffer read-only.  This is quite surprising
because the buffer is only used internally by gnus-mock.  Could so-long
be a bit smarter here?

TIA,

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

[-- Attachment #1: Type: text/plain, Size: 65 bytes --]

Eric,

btw, could you please consider to install something like


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: gnus-mock.patch --]
[-- Type: text/x-diff, Size: 727 bytes --]

*** /tmp/ediffydkpuQ	2019-10-18 16:36:16.171359249 +0200
--- /home/micha/.emacs.d/elpa/gnus-mock-0.4.2/gnus-mock.el	2019-10-18 16:34:36.303361867 +0200
***************
*** 128,134 ****
  (defconst gnus-mock-data-dir
    (file-name-as-directory (expand-file-name
  			   "data"
! 			   (file-name-directory load-file-name)))
    "Source directory for Gnus mock data.")

  ;;;###autoload
--- 128,136 ----
  (defconst gnus-mock-data-dir
    (file-name-as-directory (expand-file-name
  			   "data"
! 			   (if load-file-name
!                                (file-name-directory load-file-name)
!                              default-directory)))
    "Source directory for Gnus mock data.")

  ;;;###autoload

[-- Attachment #3: Type: text/plain, Size: 94 bytes --]


so that one can load the source file of gnus-mock to simplify debugging?


Thanks,

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

On 10/18/19 11:18 AM, Michael Heerdegen wrote:
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> 1. Put a breakpoint or edebug or otherwise slow the function down, and
>> after the data directory has been copied to /tmp, make sure that it
>> belongs to your user and you have the right permissions?
>
> That's the case.
>
>> 2. Try this version of gnus-mock-start and see if it works:
>
> I did something different.  Seems that so-long is to blame:

Well it seems pretty odd that a package meant for handling single-line
files would prevent us from editing a file at all. I don't even see why
so-long would even be kicking in, here -- the newsrc.eld template isn't
one long line!

I guess I can do the inhibit-read-only thing, but if I've got to
anticipate something like this happening, it's hard to guess what else I
might need to defend against...

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

On 10/18/19 16:46 PM, Michael Heerdegen wrote:
> Eric,
>
> btw, could you please consider to install something like

Sure, absolutely. I've never had very much confidence in how gnus-mock
handles the "find the load file and nearby directories" process, so it's
good to have some real-world feedback.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

On 10/18/19 16:46 PM, Michael Heerdegen wrote:
> Eric,
>
> btw, could you please consider to install something like

Wait, just to be sure I understand what's happening here: does the
default option value get evaluated at compile time, or load time, and is
there a difference?

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> > btw, could you please consider to install something like
>
> Wait, just to be sure I understand what's happening here: does the
> default option value get evaluated at compile time, or load time,

At load time.

> and is there a difference?

I think that only difference to before is that you get a useful behavior
(works as expected) when you open the file and load the buffer (tested).

I didn't think long about it but instead I had a look how other packages
handle the case.


Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

On 10/18/19 21:23 PM, Michael Heerdegen wrote:
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> > btw, could you please consider to install something like
>>
>> Wait, just to be sure I understand what's happening here: does the
>> default option value get evaluated at compile time, or load time,
>
> At load time.
>
>> and is there a difference?
>
> I think that only difference to before is that you get a useful behavior
> (works as expected) when you open the file and load the buffer (tested).
>
> I didn't think long about it but instead I had a look how other packages
> handle the case.

Good enough for me! Thanks.

bug#33005: 27.0.50; Data loss with Gnus registry

[Phil Sainty]

Hi Michael,

On 19/10/19 3:44 AM, Michael Heerdegen wrote:
> Phil, what's your opinion about this?  gnus-mock.el wants to
> edit a file it has opened with find-file-noselect, which is a
> common thing to do.  But it fails for me because I have enabled
> global-so-long-mode in my init file and it makes the buffer
> read-only.  This is quite surprising because the buffer is only
> used internally by gnus-mock.  Could so-long be a bit smarter
> here?

This sounds very much like:
https://savannah.nongnu.org/bugs/?56835

I have committed[1] a fix for that in my WIP branch here:
https://git.savannah.nongnu.org/cgit/so-long.git/tree/?h=wip

Could you test that and let me know if it fixes the issue for you?

If so, I'll go ahead and merge the changes into Emacs.  (I hadn't
merged it to Emacs yet simply because its a notable change to how
so-long has operated in the past; but I'm mostly sure it's going
to be fine.)

The documentation regarding this is as follows:

;; * Buffers which are not displayed in a window
;; ---------------------------------------------
;; When a file with long lines is visited and the buffer is not
;; displayed right away, it may be that it is not intended to be
;; displayed at all, and that it has instead been visited for
;; behind-the-scenes processing by some library.  Invisible
;; buffers do not typically not cause performance issues, and it
;; might be surprising to the other library if such a buffer were
;; manipulated by `so-long'; so in these situations the
;; `so-long-invisible-buffer-function' value is called instead.
;; By default this arranges for `so-long' to be invoked on the
;; buffer if and when it is displayed, but not otherwise.  This
;; is actually the normal way for `so-long' to be called -- even
;; when a visited file is displayed "right away", it is normal
;; for the buffer to be invisible when `global-so-long-mode'
;; processes it, and the gap between "arranging to call" and
;; "calling" `so-long' is simply extremely brief.


-Phil

[1]:
https://git.savannah.nongnu.org/cgit/so-long.git/commit/so-long.el?h=wip&id=e9d6a4ef4ccde46e65f2bea9e4756ddc8cfab8e5

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> > I didn't think long about it but instead I had a look how other packages
> > handle the case.
>
> Good enough for me! Thanks.

Do you really want this to be nil when the file buffer is evaluated?

(defconst gnus-mock-data-dir
  (when load-file-name
    (file-name-as-directory (expand-file-name
			     "data"
			     (file-name-directory load-file-name))))
  "Source directory for Gnus mock data.")

Then again I need additional work when loading the source to debug.  You
could at least accept `default-directory' when it contains a "data"
named directory, or query for a directory with a useful default instead
of barfing when gnus-mock-start is called.  With other words: what I get
now is just another kind of barfing, no real improvement.

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Phil Sainty <psainty@orcon.net.nz> writes:

> I have committed[1] a fix for that in my WIP branch here:
> https://git.savannah.nongnu.org/cgit/so-long.git/tree/?h=wip
>
> Could you test that and let me know if it fixes the issue for you?

Thanks - all that I read sounds like your branch should fix our problem.
But how can I clone your repo?

bash-5.0:micha:today$ git clone https://git.savannah.nongnu.org/cgit/so-long
Cloning into 'so-long'...
fatal: repository 'https://git.savannah.nongnu.org/cgit/so-long/' not
found

BTW, you can also just send me a checkout privately.

Regards,

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> > I didn't think long about it but instead I had a look how other packages
>> > handle the case.
>>
>> Good enough for me! Thanks.
>
> Do you really want this to be nil when the file buffer is evaluated?
>
> (defconst gnus-mock-data-dir
>   (when load-file-name
>     (file-name-as-directory (expand-file-name
> 			     "data"
> 			     (file-name-directory load-file-name))))
>   "Source directory for Gnus mock data.")
>
> Then again I need additional work when loading the source to debug.  You
> could at least accept `default-directory' when it contains a "data"
> named directory, or query for a directory with a useful default instead
> of barfing when gnus-mock-start is called.  With other words: what I get
> now is just another kind of barfing, no real improvement.

Oops, I didn't realize that default-directory would be set to the
containing directory at load time -- I assumed its chances of finding
the "data" directory correctly were essentially nil. I'll use your
version now...

bug#33005: 27.0.50; Data loss with Gnus registry

[Phil Sainty]

Hi Michael,

On 20/10/19 3:31 AM, Michael Heerdegen wrote:
> how can I clone your repo?

See the bottom of the "summary" tab of the cgit web UI for URLs
that you can clone from:

* git://git.savannah.gnu.org/so-long.git
* https://git.savannah.gnu.org/git/so-long.git
* ssh://git.savannah.gnu.org/srv/git/so-long.git

Or just fetch the .el file directly:

https://git.savannah.nongnu.org/cgit/so-long.git/plain/so-long.el?h=wip

FYI the main project pages also provides this:

* https://savannah.nongnu.org/git/?group=so-long

That page isn't linked to from the cgit URLs I'd given initially,
but in general projects on Savannah will have one of these.


-Phil

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Phil Sainty <psainty@orcon.net.nz> writes:

> Or just fetch the .el file directly:
>
> https://git.savannah.nongnu.org/cgit/so-long.git/plain/so-long.el?h=wip

Ok, I just did that and...it fixes the gnus-mock-start problem, as
expected :-)

So, Eric, I think you don't have to care about this problem.


Thanks,

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Phil Sainty <psainty@orcon.net.nz> writes:
>
>> Or just fetch the .el file directly:
>>
>> https://git.savannah.nongnu.org/cgit/so-long.git/plain/so-long.el?h=wip
>
> Ok, I just did that and...it fixes the gnus-mock-start problem, as
> expected :-)
>
> So, Eric, I think you don't have to care about this problem.

Okay cool -- thanks.

bug#33005: 27.0.50; Data loss with Gnus registry

[Phil Sainty]

On 27/10/19 4:35 AM, Eric Abrahamsen wrote:
> Michael Heerdegen <michael_heerdegen@web.de> writes:
>> [the so-long.el change] fixes the gnus-mock-start problem, as
>> expected :-)
>>
>> So, Eric, I think you don't have to care about this problem.
> 
> Okay cool -- thanks.

Just to note, this change to so-long.el has been merged to the
Emacs master branch.

regards,
-Phil

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Hello,
>
> I want to use the Gnus registry to use registry marks and store data as
> described in (info "(gnus) Store arbitrary data").  Very often, these
> things vanish after restarting Emacs or Gnus.
>
> For example, I use M M i to mark some article "Important".  I have
> configured `gnus-summary-line-format' to show registry marks.  Often,
> after restarting, the mark is gone.  Likewise, data stored with
> `gnus-registry-set-id-key' gets lost, even when the according article
> has a registry mark attached.

I'm back to here: I now get the same broken behavior as before.  All
registry marks and data just gone.  This is bad: until this works
reliably, this stuff should not be advertised in the manual IMHO.

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Michael Heerdegen <michael_heerdegen@web.de> writes:
>
>> Hello,
>>
>> I want to use the Gnus registry to use registry marks and store data as
>> described in (info "(gnus) Store arbitrary data").  Very often, these
>> things vanish after restarting Emacs or Gnus.
>>
>> For example, I use M M i to mark some article "Important".  I have
>> configured `gnus-summary-line-format' to show registry marks.  Often,
>> after restarting, the mark is gone.  Likewise, data stored with
>> `gnus-registry-set-id-key' gets lost, even when the according article
>> has a registry mark attached.
>
> I'm back to here: I now get the same broken behavior as before.  All
> registry marks and data just gone.  This is bad: until this works
> reliably, this stuff should not be advertised in the manual IMHO.

Well damn. And we're coming up on Emacs 27.1, too... I've never seen
this before -- would you mind sending all your registry-related config
(off-list, if necessary), and anything else you might think is relevant,
and I'll try to reproduce?

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Well damn. And we're coming up on Emacs 27.1, too... I've never seen
> this before -- would you mind sending all your registry-related config
> (off-list, if necessary), and anything else you might think is relevant,
> and I'll try to reproduce?

Ok, I debugged it.  FWIW I recently learned that "I'll try to reproduce"
has a meaning that I didn't expect: you need to say "I'll try to
reproduce it" or else Stallman might show up and laugh at you (it
happened to me).

Ok: I have only (gnus-registry-initialize) in my init file.  And this is
already broken:

"gnus-registry.el" initializes gnus-registry-db like this:

  (defvar gnus-registry-db (gnus-registry-make-db)
    "The article registry by Message ID.  See `registry-db'.")

and in `gnus-registry-load' the

  (null (eieio-object-p gnus-registry-db))

always fails so unless you call `gnus-registry-load' with the force arg
you always start with an empty db.

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

On 11/26/19 17:32 PM, Michael Heerdegen wrote:
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Well damn. And we're coming up on Emacs 27.1, too... I've never seen
>> this before -- would you mind sending all your registry-related config
>> (off-list, if necessary), and anything else you might think is relevant,
>> and I'll try to reproduce?
>
> Ok, I debugged it.  FWIW I recently learned that "I'll try to reproduce"
> has a meaning that I didn't expect: you need to say "I'll try to
> reproduce it" or else Stallman might show up and laugh at you (it
> happened to me).

Who knew such dangers lurked on this list?!

> Ok: I have only (gnus-registry-initialize) in my init file.  And this is
> already broken:
>
> "gnus-registry.el" initializes gnus-registry-db like this:
>
>   (defvar gnus-registry-db (gnus-registry-make-db)
>     "The article registry by Message ID.  See `registry-db'.")
>
> and in `gnus-registry-load' the
>
>   (null (eieio-object-p gnus-registry-db))
>
> always fails so unless you call `gnus-registry-load' with the force arg
> you always start with an empty db.

Oh I see, when the file is loaded the variable is already initialized to
an (unloaded) registry object, so it never actually loads/reads its
persistence file.

Except that it certainly seems to! I've never seen this cause a problem,
and no one calls `gnus-registry-load' with the FORCE argument, so if
this was an issue, it seems like it would fail for everyone
immediately...

In fact, I don't know the defvar is initialized to anything at all.
For eieio-persistent objects, you're not supposed to make-instance
yourself, you're supposed to use `eieio-persistent-read' (which is what
loading is supposed to do).

Dunno why it works, though...

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Dunno why it works, though...

Maybe you could (e)debug `gnus-registry-load' and check why the test
succeeds for you?

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

[-- Attachment #1: Type: text/plain, Size: 509 bytes --]

Does it fail for you every time?

(I'm getting on a plane in a bit, and trying to father as much info as I can before the long, dark cutoff)

On November 26, 2019 12:08:53 PM PST, Michael Heerdegen <michael_heerdegen@web.de> wrote:
>Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Dunno why it works, though...
>
>Maybe you could (e)debug `gnus-registry-load' and check why the test
>succeeds for you?
>
>Michael.

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 908 bytes --]

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Does it fail for you every time?

Hard to tell, but I think so.  Whenever I tried today it failed.

I really wonder why it works for you.

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

[-- Attachment #1: Type: text/plain, Size: 492 bytes --]

I wonder, too, now that I look at the code. I will let you know in 15 hours, when I arrive in London :)

On November 26, 2019 12:45:51 PM PST, Michael Heerdegen <michael_heerdegen@web.de> wrote:
>Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Does it fail for you every time?
>
>Hard to tell, but I think so.  Whenever I tried today it failed.
>
>I really wonder why it works for you.
>
>Michael.

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 888 bytes --]

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> I wonder, too, now that I look at the code. I will let you know in 15
> hours, when I arrive in London :)

Great - good flight!

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

On 11/26/19 21:54 PM, Michael Heerdegen wrote:
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> I wonder, too, now that I look at the code. I will let you know in 15
>> hours, when I arrive in London :)

Okay, I mostly slept on the plane. But...

One missing piece of the puzzle: the Gnus startup process runs
`gnus-clear-system', which runs `gnus-shutdown', and the registry has a
shutdown routine that sets `gnus-registry-db' to nil. Starting up Gnus
essentially first shuts it down.

But that still doesn't completely explain why this works. Say a user
starts Gnus cold, and only loads gnus-registry.el via an autoloaded call
to `gnus-registry-initialize' in the init file. The shutdowns run before
the init file is loaded, meaning gnus-registry.el hasn't been loaded,
meaning it hasn't had a chance to add its registry-related shutdown yet.
So we should still be loading the registry with an already-initialized
`gnus-registry-db', and overwriting the user's existing data.

Obviously the code as it stands should be changed: either I should find
another way of preventing double loading, or the defvar shouldn't
initialize the database to anything (I prefer this latter). But it's
still a mystery to me...

Eric

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> But that still doesn't completely explain why this works. Say a user
> starts Gnus cold, and only loads gnus-registry.el via an autoloaded call
> to `gnus-registry-initialize' in the init file. The shutdowns run before
> the init file is loaded, meaning gnus-registry.el hasn't been loaded,
> meaning it hasn't had a chance to add its registry-related shutdown yet.
> So we should still be loading the registry with an already-initialized
> `gnus-registry-db', and overwriting the user's existing data.

But that shouldn't be hard to find out with the help of edebug, variable
watchers, etc. - right?

BTW, are you sure that the behavior you see is seen by anyone else?
Could it be that it works just for you because of your setup?

> Obviously the code as it stands should be changed: either I should find
> another way of preventing double loading, or the defvar shouldn't
> initialize the database to anything (I prefer this latter).

Initializing with an empty database cries for this sort of problem.
This should only be done when loading fails because the save file
doesn't exist.  Then the user should be informed that a new empty
database is created.

BTW, what's so problematic with avoiding repeated loading?  Can't you
just use a bool var to remember?

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> But that still doesn't completely explain why this works. Say a user
>> starts Gnus cold, and only loads gnus-registry.el via an autoloaded call
>> to `gnus-registry-initialize' in the init file. The shutdowns run before
>> the init file is loaded, meaning gnus-registry.el hasn't been loaded,
>> meaning it hasn't had a chance to add its registry-related shutdown yet.
>> So we should still be loading the registry with an already-initialized
>> `gnus-registry-db', and overwriting the user's existing data.
>
> But that shouldn't be hard to find out with the help of edebug, variable
> watchers, etc. - right?

No, I just didn't have time between waking up and landing. I'm on a
business trip so I haven't had much hacking time. This weekend I should
have time.

> BTW, are you sure that the behavior you see is seen by anyone else?
> Could it be that it works just for you because of your setup?

I'd be awfully surprised if it only worked for me -- I'd think we would
have seen bug reports by now. I still blame your usage of debbugs-gnu :)

>> Obviously the code as it stands should be changed: either I should find
>> another way of preventing double loading, or the defvar shouldn't
>> initialize the database to anything (I prefer this latter).
>
> Initializing with an empty database cries for this sort of problem.
> This should only be done when loading fails because the save file
> doesn't exist.  Then the user should be informed that a new empty
> database is created.

Yes, I agree that initializing to an empty database is a bad idea. All
the mechanisms are already in place for detecting when no database has
been created, and making a new one -- there's no reason to hang an empty
database on there. I'd still like to know exactly what's going on,
though.

> BTW, what's so problematic with avoiding repeated loading?  Can't you
> just use a bool var to remember?

I thought that the `eieio-object-p' check WAS the equivalent of the
"bool var". I hadn't actually seen that the defvar was initialized like
that.

Anyway, I don't want to make another sloppy mistake. But I do think just
leaving the defvar at nil and getting rid of `gnus-registry-make-db'
will end up being the solution.

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> I'd be awfully surprised if it only worked for me -- I'd think we
> would have seen bug reports by now.

My impressions is: some people may have tried using the registry, but
most of them gave up, and nearly nobody is using it.

> I still blame your usage of debbugs-gnu :)

I don't.  AFAICT the issue happens with (featurep 'debbugs-gnu) ==> nil
all the time.

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

[-- Attachment #1: Type: text/plain, Size: 733 bytes --]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> I'd be awfully surprised if it only worked for me -- I'd think we
>> would have seen bug reports by now.
>
> My impressions is: some people may have tried using the registry, but
> most of them gave up, and nearly nobody is using it.
>
>> I still blame your usage of debbugs-gnu :)
>
> I don't.  AFAICT the issue happens with (featurep 'debbugs-gnu) ==> nil
> all the time.

No, I didn't really mean that...

Would you try the attached patch? It leaves gnus-registry-db at nil to
start with, and also removes the root of the double-loading (which was a
problem only because it can potentially be very slow).

Thanks,
Eric

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Continued-fixes-to-gnus-registry-loading.patch --]
[-- Type: text/x-patch, Size: 4845 bytes --]

From ea39e2a7a1969e669c32ef4f883cda357ad144a8 Mon Sep 17 00:00:00 2001
From: Eric Abrahamsen <eric@ericabrahamsen.net>
Date: Tue, 3 Dec 2019 09:12:44 +0000
Subject: [PATCH] Continued fixes to gnus registry loading

* lisp/gnus/gnus-registry.el (gnus-registry-db): Do not initialize
this variable to an empty database, that should only be done for new
databases.
(gnus-registry-load): Remove "force" argument, don't check if the
database is already loaded, as we're only going to load it once.
(gnus-registry-initialize): Either load the db directly, or set up a
hook to do it later.
(gnus-registry-install-hooks): Don't load on
gnus-read-newsrc-el-hook.
---
 lisp/gnus/gnus-registry.el | 70 ++++++++++++++++----------------------
 1 file changed, 30 insertions(+), 40 deletions(-)

diff --git a/lisp/gnus/gnus-registry.el b/lisp/gnus/gnus-registry.el
index e6fb382c2f..ff4c640c82 100644
--- a/lisp/gnus/gnus-registry.el
+++ b/lisp/gnus/gnus-registry.el
@@ -340,7 +340,7 @@ gnus-registry-make-db
                   :precious nil
                   :tracked nil)))
 
-(defvar gnus-registry-db (gnus-registry-make-db)
+(defvar gnus-registry-db nil
   "The article registry by Message ID.  See `registry-db'.")
 
 ;; top-level registry data management
@@ -352,40 +352,33 @@ gnus-registry-remake-db
     (gnus-message 4 "Remaking the Gnus registry")
     (setq gnus-registry-db (gnus-registry-make-db))))
 
-(defun gnus-registry-load (&optional force)
-  "Load the registry from the cache file.
-If the registry is already loaded, don't reload unless FORCE is
-non-nil."
+(defun gnus-registry-load ()
+  "Load the registry from the cache file."
   (interactive)
-  (when (or force
-	    ;; The registry is loaded by both
-	    ;; `gnus-registry-initialize' and the read-newsrc hook.
-	    ;; Don't load twice.
-	    (null (eieio-object-p gnus-registry-db)))
-    (let ((file gnus-registry-cache-file))
-      (condition-case nil
-          (gnus-registry-read file)
-	(file-error
-	 ;; Fix previous mis-naming of the registry file.
-	 (let ((old-file-name
-		(concat (file-name-sans-extension
-			 gnus-registry-cache-file)
-			".eioio")))
-	   (if (and (file-exists-p old-file-name)
-		    (yes-or-no-p
-		     (format "Rename registry file from %s to %s? "
-			     old-file-name file)))
-	       (progn
-		 (gnus-registry-read old-file-name)
-		 (setf (oref gnus-registry-db file) file)
-		 (gnus-message 1 "Registry filename changed to %s" file))
-	     (gnus-registry-remake-db t))))
-	(error
-	 (gnus-message
-          1
-          "The Gnus registry could not be loaded from %s, creating a new one"
-          file)
-	 (gnus-registry-remake-db t))))))
+  (let ((file gnus-registry-cache-file))
+    (condition-case nil
+        (gnus-registry-read file)
+      (file-error
+       ;; Fix previous mis-naming of the registry file.
+       (let ((old-file-name
+	      (concat (file-name-sans-extension
+		       gnus-registry-cache-file)
+		      ".eioio")))
+	 (if (and (file-exists-p old-file-name)
+		  (yes-or-no-p
+		   (format "Rename registry file from %s to %s? "
+			   old-file-name file)))
+	     (progn
+	       (gnus-registry-read old-file-name)
+	       (setf (oref gnus-registry-db file) file)
+	       (gnus-message 1 "Registry filename changed to %s" file))
+	   (gnus-registry-remake-db t))))
+      (error
+       (gnus-message
+        1
+        "The Gnus registry could not be loaded from %s, creating a new one"
+        file)
+       (gnus-registry-remake-db t)))))
 
 (defun gnus-registry-read (file)
   "Do the actual reading of the registry persistence file."
@@ -1178,13 +1171,12 @@ gnus-registry-initialize
   (gnus-message 5 "Initializing the registry")
   (gnus-registry-install-hooks)
   (gnus-registry-install-shortcuts)
-  (gnus-registry-load))
+  (if (gnus-alive-p)
+      (gnus-registry-load)
+    (add-hook 'gnus-read-newsrc-el-hook 'gnus-registry-load)))
 
-;; FIXME: Why autoload this function?
-;;;###autoload
 (defun gnus-registry-install-hooks ()
   "Install the registry hooks."
-  (interactive)
   (setq gnus-registry-enabled t)
   (add-hook 'gnus-summary-article-move-hook 'gnus-registry-action)
   (add-hook 'gnus-summary-article-delete-hook 'gnus-registry-action)
@@ -1192,13 +1184,11 @@ gnus-registry-install-hooks
   (add-hook 'nnmail-spool-hook 'gnus-registry-spool-action)
 
   (add-hook 'gnus-save-newsrc-hook 'gnus-registry-save)
-  (add-hook 'gnus-read-newsrc-el-hook 'gnus-registry-load)
 
   (add-hook 'gnus-summary-prepare-hook 'gnus-registry-register-message-ids))
 
 (defun gnus-registry-unload-hook ()
   "Uninstall the registry hooks."
-  (interactive)
   (remove-hook 'gnus-summary-article-move-hook 'gnus-registry-action)
   (remove-hook 'gnus-summary-article-delete-hook 'gnus-registry-action)
   (remove-hook 'gnus-summary-article-expire-hook 'gnus-registry-action)
-- 
2.24.0

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Would you try the attached patch? It leaves gnus-registry-db at nil to
> start with, and also removes the root of the double-loading (which was a
> problem only because it can potentially be very slow).

Seems to work so far, tested for a while (also once quickly with
debbugs-gnu ;-)

So you now (re-)load the db repeatedly and unconditionally?


Thanks,

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

On 12/04/19 17:10 PM, Michael Heerdegen wrote:
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Would you try the attached patch? It leaves gnus-registry-db at nil to
>> start with, and also removes the root of the double-loading (which was a
>> problem only because it can potentially be very slow).
>
> Seems to work so far, tested for a while (also once quickly with
> debbugs-gnu ;-)

Oh good :)

> So you now (re-)load the db repeatedly and unconditionally?

No: unconditionally but only once. I fixed it (I hope) closer to the
root of the problem: calling `gnus-registry-initialize' used to load
once unconditionally, then add the load function to
`gnus-read-newsrc-hook', which would load a second time at startup. Now
`gnus-registry-initialize' checks whether Gnus has already been started
and does one or the other, not both.

I'll test a bit more and then push, this should go into 27.1.

Thanks

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:


> > So you now (re-)load the db repeatedly and unconditionally?
>
> No: unconditionally but only once.

For me it's reloaded for every M-x gnus (but only once per gnus
invocation).  I'm not sure if that's what you meant.

  gnus-registry-load()
  run-hooks(gnus-read-newsrc-el-hook)
  apply(run-hooks gnus-read-newsrc-el-hook)
  gnus-run-hooks(gnus-read-newsrc-el-hook)
  gnus-read-newsrc-el-file("/home/micha/gnu-emacs/.newsrc.el")
  gnus-read-newsrc-file(nil)
  gnus-setup-news(nil nil nil)
  #f(compiled-function () #<bytecode 0x158c0b97bf39>)()
  gnus-1(nil nil nil)
  gnus(nil)
  funcall-interactively(gnus nil)
  call-interactively(gnus nil nil)
  command-execute(gnus)


Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

On December 4, 2019 12:41:07 PM PST, Michael Heerdegen <michael_heerdegen@web.de> wrote:
>Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>
>> > So you now (re-)load the db repeatedly and unconditionally?
>>
>> No: unconditionally but only once.
>
>For me it's reloaded for every M-x gnus (but only once per gnus
>invocation).  I'm not sure if that's what you meant.

Yes, that's what I meant. Every invocation of `gnus' first unloads everything and then reloads it, and the registry is likewise unloaded and reloaded. That seemed like TRT to me.

>  gnus-registry-load()
>  run-hooks(gnus-read-newsrc-el-hook)
>  apply(run-hooks gnus-read-newsrc-el-hook)
>  gnus-run-hooks(gnus-read-newsrc-el-hook)
>  gnus-read-newsrc-el-file("/home/micha/gnu-emacs/.newsrc.el")
>  gnus-read-newsrc-file(nil)
>  gnus-setup-news(nil nil nil)
>  #f(compiled-function () #<bytecode 0x158c0b97bf39>)()
>  gnus-1(nil nil nil)
>  gnus(nil)
>  funcall-interactively(gnus nil)
>  call-interactively(gnus nil nil)
>  command-execute(gnus)
>
>
>Michael.

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Yes, that's what I meant. Every invocation of `gnus' first unloads
> everything and then reloads it, and the registry is likewise unloaded
> and reloaded. That seemed like TRT to me.

Ok by me - I only wanted to be sure that what I see is intended.

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

On 12/04/19 22:02 PM, Michael Heerdegen wrote:
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Yes, that's what I meant. Every invocation of `gnus' first unloads
>> everything and then reloads it, and the registry is likewise unloaded
>> and reloaded. That seemed like TRT to me.
>
> Ok by me - I only wanted to be sure that what I see is intended.

It is! Pace bugs I may have introduced, I feel like this is the right
approach.

The only integration that's lacking is auto save (which doesn't work
well with eieio-persistent). If you quit Gnus without saving, your
intermediate state goes into the dribble file, and you can choose to
load that again later. But the registry doesn't work with the dribble
file (it would be a medium amount of work to make that happen) so you'd
lose any edits to the registry made since the last save.

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> The only integration that's lacking is auto save (which doesn't work
> well with eieio-persistent). If you quit Gnus without saving, your
> intermediate state goes into the dribble file, and you can choose to
> load that again later. But the registry doesn't work with the dribble
> file (it would be a medium amount of work to make that happen) so you'd
> lose any edits to the registry made since the last save.

Another difference is that when you quit the summary with -Q, registry
marks are saved anyway.  But I like that :-)

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

On 12/04/19 22:51 PM, Michael Heerdegen wrote:
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> The only integration that's lacking is auto save (which doesn't work
>> well with eieio-persistent). If you quit Gnus without saving, your
>> intermediate state goes into the dribble file, and you can choose to
>> load that again later. But the registry doesn't work with the dribble
>> file (it would be a medium amount of work to make that happen) so you'd
>> lose any edits to the registry made since the last save.
>
> Another difference is that when you quit the summary with -Q, registry
> marks are saved anyway.  But I like that :-)

Yes, the registry actions are immediate, while most of the other summary
operations are "apply on exit". I don't think that matters, though --
the "apply on exit" actions are only related to manipulating marks,
which the registry doesn't care about. The immediate actions -- delete,
move, expire, spool, etc -- the registry already watches.

You made me nervous with this comment, but I'm going to assume
everything works the way it's supposed to :)

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> You made me nervous with this comment, but I'm going to assume
> everything works the way it's supposed to :)

Yes, I'm assuming that, too.  Didn't want to scare you.

So, are you going to install your patch?

Regards,

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> You made me nervous with this comment, but I'm going to assume
>> everything works the way it's supposed to :)
>
> Yes, I'm assuming that, too.  Didn't want to scare you.
>
> So, are you going to install your patch?

I'll do it now. The first time I applied it on my own setup it told me
it was reindexing the registry, which made me nervous again, but it only
did that once, and didn't lose any data. What I think actually happened
is that the fixup-registry routines ran and updated the :tracked slot
and saved it properly, which might not have been happening before.

Thanks for your continued patience with these problems. Don't
give up on the registry!

Eric

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Thanks for your continued patience with these problems. Don't
> give up on the registry!

Ok, I won't.

I wanted now to add a hack to my config to save a backup of the registry
save file every time it is saved.

But now I wonder if that would not be a good idea anyway?  I think it
could be done like in `gnus-save-newsrc-file' (look for
`backup-buffer').  Does that make sense?

I think I would then advice `gnus-registry-save' so that lots of backups
are kept, in a separate folder, to prevent future data loss.


Regards,

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

On 12/09/19 20:38 PM, Michael Heerdegen wrote:
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Thanks for your continued patience with these problems. Don't
>> give up on the registry!
>
> Ok, I won't.
>
> I wanted now to add a hack to my config to save a backup of the registry
> save file every time it is saved.
>
> But now I wonder if that would not be a good idea anyway?  I think it
> could be done like in `gnus-save-newsrc-file' (look for
> `backup-buffer').  Does that make sense?
>
> I think I would then advice `gnus-registry-save' so that lots of backups
> are kept, in a separate folder, to prevent future data loss.

There are plenty of places where you could do a backup. The registry
itself is saved on the `gnus-save-newsrc-hook', that's a fine spot.

I was going to say you could add a method to `eieio-persistent-save',
maybe an :after method, and do it there, but now I wonder if you might
get into some sort of recursive loop, since you'd want to use
`eieio-persistent-save' in there, as well. I was thinking you should do
it at the lowest level possible, but this might actually be too low.
Part of the reason for "going low" is that there are routines for making
sure that non-ascii group names are mangled properly before saving.

Ooh, I just realized -- Lars wasn't into me properly encoding Gnus group
names in the .newsrc.eld file, but there's no reason the registry has to
be under that restriction, as the data is meant to be transient at some
level. I could leave the decoding routines in place and get rid of the encoders.

Anyway! That's a different question.

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> I was going to say you could add a method to `eieio-persistent-save',
> maybe an :after method, and do it there, but now I wonder if you might
> get into some sort of recursive loop, since you'd want to use
> `eieio-persistent-save' in there, as well.

AFAIU that's not necessary or what I plan.  When `backup-buffer' creates
a backup for a file (named by buffer-file-name) it basically makes a
copy of the already existing file (and before any new contents are
saved) as newest backup.  Making a backup of the old version of the
registry save file is sufficient.

Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> I was going to say you could add a method to `eieio-persistent-save',
>> maybe an :after method, and do it there, but now I wonder if you might
>> get into some sort of recursive loop, since you'd want to use
>> `eieio-persistent-save' in there, as well.
>
> AFAIU that's not necessary or what I plan.  When `backup-buffer' creates
> a backup for a file (named by buffer-file-name) it basically makes a
> copy of the already existing file (and before any new contents are
> saved) as newest backup.  Making a backup of the old version of the
> registry save file is sufficient.

Oh, of course! Well in that case an :after method on
`eieio-persistent-save' might be just the thing.

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Oh, of course! Well in that case an :after method on
> `eieio-persistent-save' might be just the thing.

I'll use this for now - works so far:

#+begin_src emacs-lisp
(setf (alist-get (regexp-quote ".gnus.registry.eieio") file-name-handler-alist)
      (defun my-gnus-registry-save-file-handler (operation &rest args)
        (pcase-let ((`(,_start ,_end ,filename . ,_rest) args))
          (when (eq operation 'write-region)
            (let ((buffer-backed-up nil)
                  (buffer-file-name filename)
                  (file-precious-flag t)
                  (kept-new-versions 300))
              (backup-buffer))))
        (let ((inhibit-file-name-handlers
               (cons 'my-gnus-registry-save-file-handler
                     inhibit-file-name-handlers))
              (inhibit-file-name-operation operation))
          (apply operation args))))
#+end_src


Michael.

bug#33005: 27.0.50; Data loss with Gnus registry

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Oh, of course! Well in that case an :after method on
>> `eieio-persistent-save' might be just the thing.
>
> I'll use this for now - works so far:
>
> #+begin_src emacs-lisp
> (setf (alist-get (regexp-quote ".gnus.registry.eieio") file-name-handler-alist)
>       (defun my-gnus-registry-save-file-handler (operation &rest args)
>         (pcase-let ((`(,_start ,_end ,filename . ,_rest) args))
>           (when (eq operation 'write-region)
>             (let ((buffer-backed-up nil)
>                   (buffer-file-name filename)
>                   (file-precious-flag t)
>                   (kept-new-versions 300))
>               (backup-buffer))))
>         (let ((inhibit-file-name-handlers
>                (cons 'my-gnus-registry-save-file-handler
>                      inhibit-file-name-handlers))
>               (inhibit-file-name-operation operation))
>           (apply operation args))))
> #+end_src

Huh. Seems like a fair amount of work, but it's certainly low level!
Nice to have it integrated into all of Emacs' regular save-file
mechanisms.

bug#33005: 27.0.50; Data loss with Gnus registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> > #+begin_src emacs-lisp
> > (setf (alist-get (regexp-quote ".gnus.registry.eieio") file-name-handler-alist)
> >       (defun my-gnus-registry-save-file-handler (operation &rest args)
> >         (pcase-let ((`(,_start ,_end ,filename . ,_rest) args))
> >           (when (eq operation 'write-region)
> >             (let ((buffer-backed-up nil)
> >                   (buffer-file-name filename)
> >                   (file-precious-flag t)
> >                   (kept-new-versions 300))
> >               (backup-buffer))))
> >         (let ((inhibit-file-name-handlers
> >                (cons 'my-gnus-registry-save-file-handler
> >                      inhibit-file-name-handlers))
> >               (inhibit-file-name-operation operation))
> >           (apply operation args))))
> > #+end_src
>
> Huh. Seems like a fair amount of work, but it's certainly low level!
> Nice to have it integrated into all of Emacs' regular save-file
> mechanisms.

Caveat: if anybody wants to use this: please specify a `testfn' argument
(e.g. `equal') of `alist-get' to prevent repeated pushing to
`file-name-handler-alist' when the code is loaded repeatedly (which is
the case for the gnus init file AFAIK).

Michael.