Any way to control which articles Gnus summary shows by default?

Any way to control which articles Gnus summary shows by default?

[Michael Heerdegen]

Hi,

for example, I want to use registry custom marks to flag some articles
"important" (M M i).  When I enter a group, it would be nice if these
important articles would be shown, because they are, well, important to
me.  I don't want to tick or set unread articles only to make them been
shown by default.

I know about `gnus-alter-articles-to-read-function', but AFAICT that
does something different: something like

#+begin_src emacs-lisp
(add-function :around gnus-alter-articles-to-read-function
              (defun my-gnus-auto-show-registry-marked (f group-name article-list)
                (cl-union
                 (delq nil
                       (mapcar (lambda (id) (cdr (gnus-request-head id group-name)))
                               (cl-loop for key being the hash-keys of (oref gnus-registry-db data)
                                        using (hash-values v)
                                        when (assoc 'mark v)
                                        collect key)))
                 (funcall f group-name article-list))))
#+end_src

will only fetch all those articles (I guess), but won't make them been
shown by default.


Michael.

Re: Any way to control which articles Gnus summary shows by default?

[Emanuel Berg]

Michael Heerdegen wrote:

> for example, I want to use registry custom
> marks to flag some articles "important" (M
> M i). When I enter a group, it would be nice
> if these important articles would be shown,
> because they are, well, important to me.
> I don't want to tick or set unread articles
> only to make them been shown by default.

Interesting but, isn't this exactly what
happens when you tick an article? So is there
some side-effect that is undesired for this
purpose or why don't you want to use it?

What do you mean by "set" article?

BTW `M M i' is undefined for me so maybe it is
a new feature in Emacs 25 or did you configure
that? Or just thought that is what you'd assign
to the new "important" mark? You do this with
`gnus-summary-mark'?

> [snipped code] will only fetch all those
> articles (I guess), but won't make them been
> shown by default.

If you indeed got it to work, how about adding
it to some hook/advice each time you enter
a group?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Any way to control which articles Gnus summary shows by default?

[Emanuel Berg]

This is a really basic idea and the reason
I never thought about it or felt the need for
it is because I always used the tick for this.
But yes, I see the reason for having this (tick
for "important, read later", "important" for
"read, contains material I want to return to so
I'd like to keep it"), and it shouldn't be that
difficult to do.

On would need a variable
~"gnus-summary-always-show", and articles that
have a mark that is in that list, should by
shown by default.

Then one would have yet another variable,
~"gnus-summary-ignore-always-show" to inhibit
it :)

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Any way to control which articles Gnus summary shows by default?

[Michael Heerdegen]

Emanuel Berg <moasen@zoho.com> writes:

> Interesting but, isn't this exactly what happens when you tick an
> article? So is there some side-effect that is undesired for this
> purpose or why don't you want to use it?

I still try to find a good workflow for my Inbox.  In the past, I used
to tick nearly all articles, or make them unread, just to make them
appear the next time I select the group.

What I want in general is more control: I want to decide myself how Gnus
decides which articles it presents to me when I select a group.  In
particular, I want to have it depend on the information present in the
registry.


> What do you mean by "set" article?

I wrote "set unread" or so and meant "make the article unread".


> BTW `M M i' is undefined for me so maybe it is a new feature in Emacs
> 25 or did you configure that?

It's a feature when you enable the Gnus registry:

  (info "(gnus) Store custom flags and keywords")

Seems not many people use it so far, though.


> If you indeed got it to work, how about adding it to some hook/advice
> each time you enter a group?

I think you shouldn't need a hack to do this kind of thing.  A hack
would probably call `gnus-summary-limit', but it would be better if it
would be configurable out of the box.  Which I think is not the case.
The code responsible for group selection is quite complex, but I have
the impression that these things are hardcoded.


Thanks,

Michael.

Re: Any way to control which articles Gnus summary shows by default?

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Emanuel Berg <moasen@zoho.com> writes:
>
>> Interesting but, isn't this exactly what happens when you tick an
>> article? So is there some side-effect that is undesired for this
>> purpose or why don't you want to use it?
>
> I still try to find a good workflow for my Inbox.  In the past, I used
> to tick nearly all articles, or make them unread, just to make them
> appear the next time I select the group.
>
> What I want in general is more control: I want to decide myself how Gnus
> decides which articles it presents to me when I select a group.  In
> particular, I want to have it depend on the information present in the
> registry.
>
>
>> What do you mean by "set" article?
>
> I wrote "set unread" or so and meant "make the article unread".
>
>
>> BTW `M M i' is undefined for me so maybe it is a new feature in Emacs
>> 25 or did you configure that?
>
> It's a feature when you enable the Gnus registry:
>
>   (info "(gnus) Store custom flags and keywords")
>
> Seems not many people use it so far, though.
>
>
>> If you indeed got it to work, how about adding it to some hook/advice
>> each time you enter a group?
>
> I think you shouldn't need a hack to do this kind of thing.  A hack
> would probably call `gnus-summary-limit', but it would be better if it
> would be configurable out of the box.  Which I think is not the case.
> The code responsible for group selection is quite complex, but I have
> the impression that these things are hardcoded.

I agree that the process should be made more flexible, and would be
happy to help work on a better solution that we can add to Gnus. That
would happen at my typical pace of code contribution, though, which is
best measured in months :)

Some might think Gnus is over-customizable, but some aspects of
customization, like posting styles, is both flexible and usable. It
would be nice to provide similar access for group article selection,
which is another core aspect of Gnus usage behavior.

E

Re: Any way to control which articles Gnus summary shows by default?

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> I agree that the process should be made more flexible, and would be
> happy to help work on a better solution that we can add to Gnus. That
> would happen at my typical pace of code contribution, though, which is
> best measured in months :)

No problem.  I just filed a bug report (Emacs Bug#31057) as a reminder
that it would be nice to have this.


Regards,

Michael.

Re: Any way to control which articles Gnus summary shows by default?

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> I agree that the process should be made more flexible, and would be
>> happy to help work on a better solution that we can add to Gnus. That
>> would happen at my typical pace of code contribution, though, which is
>> best measured in months :)
>
> No problem.  I just filed a bug report (Emacs Bug#31057) as a reminder
> that it would be nice to have this.

And I have ticked the bug report article in Gnus :)

Re: Any way to control which articles Gnus summary shows by default?

[Emanuel Berg]

Eric Abrahamsen wrote:

> Some might think Gnus is over-customizable

I don't think a lot of people are thinking that
and besides is that even a word? :)

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Any way to control which articles Gnus summary shows by default?

[Emanuel Berg]

Michael Heerdegen wrote:

> I think you shouldn't need a hack to do this
> kind of thing.

Well, I agree it is optimal if it is provided
because it is such a natural feature, but using
a hook/advice I don't really consider a hack as
that is equally natural usage of *them*.

> A hack would probably call
> `gnus-summary-limit'

Are you sure? That one isn't documented, but
the interface doesn't have an argument or so to
imply one can process articles differently.

(gnus-summary-limit ARTICLES &optional POP)

> but it would be better if it would be
> configurable out of the box. Which I think is
> not the case. The code responsible for group
> selection is quite complex, but I have the
> impression that these things are hardcoded.

Did you manage to solve this the "hack"/hook
way? If so, even tho you consider it a hack, it
might be beneficial to the (official) developers
as I don't see why it can't just be moved from
your hook into where it fits into the Gnus
source just right.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Any way to control which articles Gnus summary shows by default?

[Michael Heerdegen]

Emanuel Berg <moasen@zoho.com> writes:

> > A hack would probably call `gnus-summary-limit'
>
> Are you sure? That one isn't documented, but the interface doesn't
> have an argument or so to imply one can process articles differently.

AFAICT most commands of kind "insert*articles" and "limit-to-*" are
implemented on top of it.

> Did you manage to solve this the "hack"/hook way?

No, I didn't try, I don't want to.  I'm quite sure doing it in a way
that it "works" somehow is easy.  Doing it right so that maintainers can
be sure it does the right thing under any circumstances is the hard
part.  The Gnus sources are complicated, so it's better you know what
you are doing.


Michael.

Re: Any way to control which articles Gnus summary shows by default?

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> And I have ticked the bug report article in Gnus :)

And that didn't happen automatically???

But well, bad news: I seemingly lost a custom mark.  Probably from
pruning, but I thought that can't happen when you have `mark' in
`gnus-registry-extra-entries-precious' (...?).

I then opened my .gnus.registry.eieio (with "kate", if you care), and I
again saw some lines with "quote" corruption (you know what I mean).
They can't be from before the fix because I deleted the old registry
file and started from scratch after rebuilding Emacs (I switched between
a master and an emacs-26 build some several times since then, but AFAIK
both versions included the fix).


Michael.

Re: Any way to control which articles Gnus summary shows by default?

[Michael Heerdegen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> But well, bad news: I seemingly lost a custom mark.  Probably from
> pruning, but I thought that can't happen when you have `mark' in
> `gnus-registry-extra-entries-precious' (...?).

Though, I had a crash of X today, maybe the custom mark hadn't yet been
saved.  OTOH I don't think the crash inserted the quotes.


Michael.

Re: Any way to control which articles Gnus summary shows by default?

[Emanuel Berg]

Michael Heerdegen wrote:

> No, I didn't try, I don't want to. I'm quite
> sure doing it in a way that it "works"
> somehow is easy. Doing it right so that
> maintainers can be sure it does the right
> thing under any circumstances is the
> hard part.

I don't think it'd be such a big difference,
and they could use the code as a starting point
and change the details if need be.

> The Gnus sources are complicated, so it's
> better you know what you are doing.

Gnus source is very difficult to read because
the functions are so long. I hear this is
because Elisp is slow, and Gnus is slow, so
making it (the source) more modular would slow
it down even further with lots of calls back
and forth.

Even so, this should all amount to, if this
feature is solved in a short way, this short
function or piece of code would just be
incorporated into a longer function that
already is, and the developers should know
already where to put it.

Gnus is complicated as a whole but boil it down
it is just the same Elisp all day every day
that does the job and I cannot see why it
should be anything else.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Any way to control which articles Gnus summary shows by default?

[Michael Heerdegen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> I then opened my .gnus.registry.eieio (with "kate", if you care), and I
> again saw some lines with "quote" corruption (you know what I mean).

My impression is that the Gnus registry itself contains some hashtables
as values, and the fix does only handle the top level hash values, while
the quote problem persists for the sub-hashtables.

Does that make sense?


Michael.

Re: Any way to control which articles Gnus summary shows by default?

[Eric Abrahamsen]

On 04/06/18 16:52 PM, Michael Heerdegen wrote:
> Michael Heerdegen <michael_heerdegen@web.de> writes:
>
>> I then opened my .gnus.registry.eieio (with "kate", if you care), and I
>> again saw some lines with "quote" corruption (you know what I mean).
>
> My impression is that the Gnus registry itself contains some hashtables
> as values, and the fix does only handle the top level hash values, while
> the quote problem persists for the sub-hashtables.
>
> Does that make sense?

Thanks for the hint. I was setting the cleaned values for the subtables,
but not then re-setting the tables themselves: they were getting fixed,
then thrown away. I can't believe how long this is taking to get
right... I'll post to gnus.general again once I'm confident (ha) I've
got it fixed.

Thanks,
Eric

Re: Any way to control which articles Gnus summary shows by default?

[Eric Abrahamsen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> On 04/06/18 16:52 PM, Michael Heerdegen wrote:
>> Michael Heerdegen <michael_heerdegen@web.de> writes:
>>
>>> I then opened my .gnus.registry.eieio (with "kate", if you care), and I
>>> again saw some lines with "quote" corruption (you know what I mean).
>>
>> My impression is that the Gnus registry itself contains some hashtables
>> as values, and the fix does only handle the top level hash values, while
>> the quote problem persists for the sub-hashtables.
>>
>> Does that make sense?
>
> Thanks for the hint. I was setting the cleaned values for the subtables,
> but not then re-setting the tables themselves: they were getting fixed,
> then thrown away. I can't believe how long this is taking to get
> right... I'll post to gnus.general again once I'm confident (ha) I've
> got it fixed.

Or maybe no, that's not what was happening. Does your quote corruption
look like multiple nested "quote"s, or does it look more like "quote
("val" quote ("val" quote...", like that? It's probably a bug in the
strip-quotes part of the fix.

Re: Any way to control which articles Gnus summary shows by default?

[Eric Abrahamsen]

Emanuel Berg <moasen@zoho.com> writes:

> Eric Abrahamsen wrote:
>
>> Some might think Gnus is over-customizable
>
> I don't think a lot of people are thinking that
> and besides is that even a word? :)

With enough hyphens, you can make anything a word :)

Re: Any way to control which articles Gnus summary shows by default?

[Eric Abrahamsen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> On 04/06/18 16:52 PM, Michael Heerdegen wrote:
>>> Michael Heerdegen <michael_heerdegen@web.de> writes:
>>>
>>>> I then opened my .gnus.registry.eieio (with "kate", if you care), and I
>>>> again saw some lines with "quote" corruption (you know what I mean).
>>>
>>> My impression is that the Gnus registry itself contains some hashtables
>>> as values, and the fix does only handle the top level hash values, while
>>> the quote problem persists for the sub-hashtables.
>>>
>>> Does that make sense?
>>
>> Thanks for the hint. I was setting the cleaned values for the subtables,
>> but not then re-setting the tables themselves: they were getting fixed,
>> then thrown away. I can't believe how long this is taking to get
>> right... I'll post to gnus.general again once I'm confident (ha) I've
>> got it fixed.
>
> Or maybe no, that's not what was happening. Does your quote corruption
> look like multiple nested "quote"s, or does it look more like "quote
> ("val" quote ("val" quote...", like that? It's probably a bug in the
> strip-quotes part of the fix.

Michael, would you give this new version a whirl (backup first,
obviously!)? For someone who writes as much lisp as I do, I'm actually
terrible at reasoning about list manipulation...

(defun gnus-registry-strip-quotes (lst)
  (let (acc)
    (when (consp lst)
      (while (eq (car lst) 'quote)
	(setq lst (cadr lst)))
      (while (consp lst)
	(if (eq (car lst) 'quote)
	    (setq lst (cadr lst))
	  (push (gnus-registry-strip-quotes (car lst)) acc)
	  (setq lst (cdr lst)))))
    (nconc (nreverse (delete-dups acc))
	   lst)))

(defun gnus-registry-fixit ()
  "Fix Gnus registry after eieio-persistent patches.
For use with Emacs master branch, after installing 0afb43eeb, or
the 26 branch, after daa9e853bd."
  (interactive)
  (unless (gnus-alive-p)
    (gnus))
  (when gnus-registry-enabled
    (with-slots (tracker data) gnus-registry-db
      (maphash
       (lambda (track-sym hsh)
	 (maphash
	  (lambda (k v)
	    (setf (gethash k hsh)
		  (gnus-registry-strip-quotes v)))
	  hsh))
       tracker)
      (maphash
       (lambda (k v)
	 (setf (gethash k data) (gnus-registry-strip-quotes v)))
       data))))

Re: Any way to control which articles Gnus summary shows by default?

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Or maybe no, that's not what was happening. Does your quote corruption
> look like multiple nested "quote"s, or does it look more like "quote
> ("val" quote ("val" quote...", like that? It's probably a bug in the
> strip-quotes part of the fix.

They look like nested quotes: '''''''''''''''''''''''''SOMETHING.

Michael.

Re: Any way to control which articles Gnus summary shows by default?

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Michael, would you give this new version a whirl (backup first,
> obviously!)? For someone who writes as much lisp as I do, I'm actually
> terrible at reasoning about list manipulation...

After calling this `gnus-registry-fixit', there are no nested quotes any
more in the file.  So far it seems it was successful!

But that does this also prevent that my database gets corrupted again?
As I said, I had started from scratch with a removed database file with
an Emacs version that had your fix already applied.


Thanks,

Michael.

Re: Any way to control which articles Gnus summary shows by default?

[Michael Heerdegen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> But that does this also prevent that my database gets corrupted again?

No, quotes still accumulate in the database file.


Michael.

Re: Any way to control which articles Gnus summary shows by default?

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> > Thanks for the hint. I was setting the cleaned values for the
> > subtables, but not then re-setting the tables themselves: they were
> > getting fixed, then thrown away. I can't believe how long this is
> > taking to get right... I'll post to gnus.general again once I'm
> > confident (ha) I've got it fixed.
>
> Or maybe no, that's not what was happening.

I'm not sure if that's not what is happening.

AFAIU `eieio-persistent-read' reads the whole thing (database) at once.
That function calls `eieio-persistent-convert-list-to-object' on the
read value, which loops `eieio-persistent-validate/fix-slot-value' over
the slot values.  The latter may only transform (quote SOMETHING) to
SOMETHING but doesn't look inside SOMETHING so the whole cleanup thing
isn't recursive.  But AFAIU the Gnus registry is recursive enough to be
problematic in this regard.

BTW, why does writing to a file involve adding quotes before a list at
all?  The reader doesn't need the lists been quoted, so why do we add
the quotes to the file just to remove them troublesomely after reading?


Michael.

Re: Any way to control which articles Gnus summary shows by default?

[Eric Abrahamsen]

On 04/08/18 16:17 PM, Michael Heerdegen wrote:
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> > Thanks for the hint. I was setting the cleaned values for the
>> > subtables, but not then re-setting the tables themselves: they were
>> > getting fixed, then thrown away. I can't believe how long this is
>> > taking to get right... I'll post to gnus.general again once I'm
>> > confident (ha) I've got it fixed.
>>
>> Or maybe no, that's not what was happening.
>
> I'm not sure if that's not what is happening.
>
> AFAIU `eieio-persistent-read' reads the whole thing (database) at once.
> That function calls `eieio-persistent-convert-list-to-object' on the
> read value, which loops `eieio-persistent-validate/fix-slot-value' over
> the slot values.  The latter may only transform (quote SOMETHING) to
> SOMETHING but doesn't look inside SOMETHING so the whole cleanup thing
> isn't recursive.  But AFAIU the Gnus registry is recursive enough to be
> problematic in this regard.

Yes, the whole problem arose from the fact that the write/restore
process isn't recursive. In Emacs 26, the process needed to "look
deeper" inside the objects, and I thought that in doing that I'd been
careful to balance the write and restore process. Apparently I've still
failed to get it right -- though now I'm only seeing quote accumulation
in the secondary tables, not the main :data table.

> BTW, why does writing to a file involve adding quotes before a list at
> all?  The reader doesn't need the lists been quoted, so why do we add
> the quotes to the file just to remove them troublesomely after reading?

The whole process is ad hoc and largely unnecessary. In #29541 I've
started working on a more general solution that would be actually
recursive, and also not work so hard. I've had second thoughts about the
patch I posted there, but I think the general direction is correct.

Anyway, the real solution is just to re-write the whole system, though I
guess I'll need to fix this current bug, hopefully for a 26.X release.

Eric

Re: Any way to control which articles Gnus summary shows by default?

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Yes, the whole problem arose from the fact that the write/restore
> process isn't recursive. In Emacs 26, the process needed to "look
> deeper" inside the objects, and I thought that in doing that I'd been
> careful to balance the write and restore process. Apparently I've still
> failed to get it right -- though now I'm only seeing quote accumulation
> in the secondary tables, not the main :data table.

Same here: not in :data, but in :tracker.  But the :tracker data
contains sub-hashtables, while :data doesn't.

> The whole process is ad hoc and largely unnecessary. In #29541 I've
> started working on a more general solution that would be actually
> recursive, and also not work so hard. I've had second thoughts about the
> patch I posted there, but I think the general direction is correct.

What would happen if we just don't add quotes when writing lists?


Michael.

Re: Any way to control which articles Gnus summary shows by default?

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Yes, the whole problem arose from the fact that the write/restore
>> process isn't recursive. In Emacs 26, the process needed to "look
>> deeper" inside the objects, and I thought that in doing that I'd been
>> careful to balance the write and restore process. Apparently I've still
>> failed to get it right -- though now I'm only seeing quote accumulation
>> in the secondary tables, not the main :data table.
>
> Same here: not in :data, but in :tracker.  But the :tracker data
> contains sub-hashtables, while :data doesn't.
>
>> The whole process is ad hoc and largely unnecessary. In #29541 I've
>> started working on a more general solution that would be actually
>> recursive, and also not work so hard. I've had second thoughts about the
>> patch I posted there, but I think the general direction is correct.
>
> What would happen if we just don't add quotes when writing lists?

I don't really know. I've been trying to make the smallest possible fix
for the meantime, while simultaneously working on a more thorough
solution. I can experiment with not writing a quote, but I keep screwing
this up _because_ the system is so convoluted, and poking something over
here results in things falling apart over there. Apparently the
regression tests aren't good enough yet, either. Anyway, I'll try to
come up with something later today.

Eric

Re: Any way to control which articles Gnus summary shows by default?

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Yes, the whole problem arose from the fact that the write/restore
>> process isn't recursive. In Emacs 26, the process needed to "look
>> deeper" inside the objects, and I thought that in doing that I'd been
>> careful to balance the write and restore process. Apparently I've still
>> failed to get it right -- though now I'm only seeing quote accumulation
>> in the secondary tables, not the main :data table.
>
> Same here: not in :data, but in :tracker.  But the :tracker data
> contains sub-hashtables, while :data doesn't.

Okay, the immediate problem is that `eieio-override-prin1' is recursive
inside hash tables (it adds quotes to hash table values to an arbitrary
depth), but the mirror-image recovery process inside
`eieio-persistent-validate/fix-slot-value' is not recursive, and only
examines one layer.

>> The whole process is ad hoc and largely unnecessary. In #29541 I've
>> started working on a more general solution that would be actually
>> recursive, and also not work so hard. I've had second thoughts about the
>> patch I posted there, but I think the general direction is correct.
>
> What would happen if we just don't add quotes when writing lists?

The solution will either be not adding quotes at all, or making the
read/restore/recovery process equally recursive. Stand by...

Re: Any way to control which articles Gnus summary shows by default?

[Eric Abrahamsen]

[-- Attachment #1: Type: text/plain, Size: 2111 bytes --]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Michael Heerdegen <michael_heerdegen@web.de> writes:
>
>> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>>
>>> Yes, the whole problem arose from the fact that the write/restore
>>> process isn't recursive. In Emacs 26, the process needed to "look
>>> deeper" inside the objects, and I thought that in doing that I'd been
>>> careful to balance the write and restore process. Apparently I've still
>>> failed to get it right -- though now I'm only seeing quote accumulation
>>> in the secondary tables, not the main :data table.
>>
>> Same here: not in :data, but in :tracker.  But the :tracker data
>> contains sub-hashtables, while :data doesn't.
>
> Okay, the immediate problem is that `eieio-override-prin1' is recursive
> inside hash tables (it adds quotes to hash table values to an arbitrary
> depth), but the mirror-image recovery process inside
> `eieio-persistent-validate/fix-slot-value' is not recursive, and only
> examines one layer.
>
>>> The whole process is ad hoc and largely unnecessary. In #29541 I've
>>> started working on a more general solution that would be actually
>>> recursive, and also not work so hard. I've had second thoughts about the
>>> patch I posted there, but I think the general direction is correct.
>>
>> What would happen if we just don't add quotes when writing lists?
>
> The solution will either be not adding quotes at all, or making the
> read/restore/recovery process equally recursive. Stand by...

And the solution needs to be making the process equally recursive,
because right now it's entirely possible that other packages could start
using eieio-persistent with weird data structures, and fall afoul of
this asymmetry. It's not just about quotes: if another package starts
stashing EIEIO objects inside nested hash tables, they will be written
correctly, but read incorrectly.

My next stab at a fix is in the attached patch. Michael, assuming I
haven't exhausted your patience, would you mind giving this a try? I do
believe this is the right solution, or at least the best minimal
solution.

Eric


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: anotherfix.diff --]
[-- Type: text/x-patch, Size: 2320 bytes --]

diff --git a/lisp/emacs-lisp/eieio-base.el b/lisp/emacs-lisp/eieio-base.el
index 9f9f870a75..75709ddc0a 100644
--- a/lisp/emacs-lisp/eieio-base.el
+++ b/lisp/emacs-lisp/eieio-base.el
@@ -360,32 +360,30 @@ eieio-persistent-validate/fix-slot-value
 		  proposed-value))))
         ;; For hash-tables and vectors, the top-level `read' will not
         ;; "look inside" member values, so we need to do that
-        ;; explicitly.
+        ;; explicitly.  Because `eieio-override-prin1' is recursive in
+        ;; the case of hash-tables and vectors, we recurse
+        ;; `eieio-persistent-validate/fix-slot-value' here as well.
         ((hash-table-p proposed-value)
          (maphash
           (lambda (key value)
-            (cond ((class-p (car-safe value))
-                   (setf (gethash key proposed-value)
-                         (eieio-persistent-convert-list-to-object
-                          value)))
-                  ((and (consp value)
-                        (eq (car value) 'quote))
-                   (setf (gethash key proposed-value)
-                         (cadr value)))))
+            (setf (gethash key proposed-value)
+                  (if (class-p (car-safe value))
+                      (eieio-persistent-convert-list-to-object
+                       value)
+                    (eieio-persistent-validate/fix-slot-value
+                     class slot value))))
           proposed-value)
          proposed-value)
 
         ((vectorp proposed-value)
          (dotimes (i (length proposed-value))
            (let ((val (aref proposed-value i)))
-            (cond ((class-p (car-safe val))
-                   (aset proposed-value i
-                         (eieio-persistent-convert-list-to-object
-                          (aref proposed-value i))))
-                  ((and (consp val)
-                        (eq (car val) 'quote))
-                   (aset proposed-value i
-                         (cadr val))))))
+             (aset proposed-value i
+                   (if (class-p (car-safe val))
+                       (eieio-persistent-convert-list-to-object
+                        val)
+                     (eieio-persistent-validate/fix-slot-value
+                      class slot val)))))
          proposed-value)
 
 	 ((stringp proposed-value)

Re: Any way to control which articles Gnus summary shows by default?

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> And the solution needs to be making the process equally recursive,
> because right now it's entirely possible that other packages could start
> using eieio-persistent with weird data structures, and fall afoul of
> this asymmetry. It's not just about quotes: if another package starts
> stashing EIEIO objects inside nested hash tables, they will be written
> correctly, but read incorrectly.

I think I would make it so that what is written in the save file is an
expression that, when read _and_ evaluated, would return the recovered
object.  The approach on the object-write side would not be much
different than now, but the restoring part would be trivial, so code
changes would become less problematic.


> My next stab at a fix is in the attached patch. Michael, assuming I
> haven't exhausted your patience,

My patience here lasts as long as necessary, don't worry :-)

> would you mind giving this a try? I do believe this is the right
> solution, or at least the best minimal solution.

So far this fixes quote accumulation completely for me - Great!  I
didn't yet try to use the registry explicitly, so I can't say whether
your fix works correctly.  Anyway - thanks so far!

FWIW, when I read the sources, I got the impression that all conses are
handled as if they were true lists.  Trying to save conses whose cdr is
not consp will probably error, since the current code always seems to
try to iterate over list elements.


Michael.

Re: Any way to control which articles Gnus summary shows by default?

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> And the solution needs to be making the process equally recursive,
>> because right now it's entirely possible that other packages could start
>> using eieio-persistent with weird data structures, and fall afoul of
>> this asymmetry. It's not just about quotes: if another package starts
>> stashing EIEIO objects inside nested hash tables, they will be written
>> correctly, but read incorrectly.
>
> I think I would make it so that what is written in the save file is an
> expression that, when read _and_ evaluated, would return the recovered
> object.  The approach on the object-write side would not be much
> different than now, but the restoring part would be trivial, so code
> changes would become less problematic.

Part of the principle of the thing is that `eval' is never called, as
a nod to code safety. Object creation is the only actual code to run,
and it should be done by feeding the serialization to EIEIO functions,
rather than `eval'ling anything.

>> My next stab at a fix is in the attached patch. Michael, assuming I
>> haven't exhausted your patience,
>
> My patience here lasts as long as necessary, don't worry :-)

That's good!

>> would you mind giving this a try? I do believe this is the right
>> solution, or at least the best minimal solution.
>
> So far this fixes quote accumulation completely for me - Great!  I
> didn't yet try to use the registry explicitly, so I can't say whether
> your fix works correctly.  Anyway - thanks so far!

That's good to hear, though I've thought this was fixed a couple of
times already, so I'm not celebrating yet!

> FWIW, when I read the sources, I got the impression that all conses are
> handled as if they were true lists.  Trying to save conses whose cdr is
> not consp will probably error, since the current code always seems to
> try to iterate over list elements.

You mean values like '(bob)? This seems to work okay:

(defclass persist-test (eieio-persistent)
  ((slot1 :initarg :slot1 :type list)))

(setq test (make-instance
	    'persist-test
	    :file (concat user-emacs-directory "persist-test.dat")
	    :slot1 '(bob)))

(eieio-persistent-save test)

(setq test2 (eieio-persistent-read
	     (concat user-emacs-directory "persist-test.dat")))

(slot-value test2 'slot1)

Did you mean something else?

Eric

Re: Any way to control which articles Gnus summary shows by default?

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> > FWIW, when I read the sources, I got the impression that all conses are
> > handled as if they were true lists.  Trying to save conses whose cdr is
> > not consp will probably error, since the current code always seems to
        ^^^^^
        listp

Sorry, that was a typo.

> You mean values like '(bob)? This seems to work okay:

No, I mean something that is not a regular list like (bob . alice), or
an alist with atomic values, like

  ((a . 1) (b . 2))

Haven't tried it, though.


Michael.

Re: Any way to control which articles Gnus summary shows by default?

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> > FWIW, when I read the sources, I got the impression that all conses are
>> > handled as if they were true lists.  Trying to save conses whose cdr is
>> > not consp will probably error, since the current code always seems to
>         ^^^^^
>         listp
>
> Sorry, that was a typo.
>
>> You mean values like '(bob)? This seems to work okay:
>
> No, I mean something that is not a regular list like (bob . alice), or
> an alist with atomic values, like
>
>   ((a . 1) (b . 2))
>
> Haven't tried it, though.

The same test code switching '(bob) to '(bob . alice) also works just
fine.

But do keep in mind that I'm not trying to find more bugs in the
current implementation -- there are all kinds of values that won't work
correctly. All I'm trying to do with the current version is get it
balanced between writing and restoring. The next step is to toss it out
completely and write something simpler.

The process as I'd like to have it looks more like (with code for
backwards compatibility removed):

(defun eieio-persistent-fix-value (value)
  (let (result)
    (when (consp value)
      (if (class-p (car value))
          (setq result
                (eieio-persistent-make-instance (car value) (cdr value))
                value nil)
        (while (consp value)
          (push (eieio-persistent-fix-value (car value)) result)
          (setq value (cdr value)))))
    (if (eieio-object-p result)
        result
      (nconc (nreverse result)
             (if (stringp value)
                 (substring-no-properties value)
               value)))))

That should be all that's needed. One of the discussions now is whether
it's okay to just check (class-p (car value)), or whether there should
be a more explicit flag saying "this list should actually be turned into
an object".

Eric

Re: Any way to control which articles Gnus summary shows by default?

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> The process as I'd like to have it looks more like (with code for
> backwards compatibility removed):
>
> (defun eieio-persistent-fix-value (value)
>   (let (result)
>     (when (consp value)
>       (if (class-p (car value))
>           (setq result
>                 (eieio-persistent-make-instance (car value) (cdr value))
>                 value nil)
>         (while (consp value)
>           (push (eieio-persistent-fix-value (car value)) result)
>           (setq value (cdr value)))))
>     (if (eieio-object-p result)
>         result
>       (nconc (nreverse result)
>              (if (stringp value)
>                  (substring-no-properties value)
>                value)))))

There must be something missing: VALUEs that are not consp are never
handled by this.

A minor problem with that approach is that `eieio-persistent-fix-value'
(I would rather call it "eieio-persistent-restore-value") can't
distinguish between objects and data that looks like an object.

Say the object write part converts some OBJECT to some list
REPRESENTATION.  Now it may occur that the user wants to save some data
that by accident is a list that looks like REPRESENTATION.  When the
saved data is to be restored, there is no mean to decide whether what
has been save was the OBJECT or a the list REPRESENTATION.  With other
words, there is a problem if the object-write function is not injective.
One can use special names so that collisions are less likely to happen,
but who knows in which ways people will use this stuff in the future.

Solving that problem would imply that lists are not represented by
themselves when saving.


Michael.

Re: Any way to control which articles Gnus summary shows by default?

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> The process as I'd like to have it looks more like (with code for
>> backwards compatibility removed):
>>
>> (defun eieio-persistent-fix-value (value)
>>   (let (result)
>>     (when (consp value)
>>       (if (class-p (car value))
>>           (setq result
>>                 (eieio-persistent-make-instance (car value) (cdr value))
>>                 value nil)
>>         (while (consp value)
>>           (push (eieio-persistent-fix-value (car value)) result)
>>           (setq value (cdr value)))))
>>     (if (eieio-object-p result)
>>         result
>>       (nconc (nreverse result)
>>              (if (stringp value)
>>                  (substring-no-properties value)
>>                value)))))
>
> There must be something missing: VALUEs that are not consp are never
> handled by this.

If VALUE isn't a consp, it falls through to the equivalent of:

(nconc (nreverse nil) value)

It might not be the most efficient (I'd really like to work on making
this whole process more efficient), but it comes out correctly. This
entire process shouldn't do anything but reconstruct objects, and strip
properties from strings.

> A minor problem with that approach is that `eieio-persistent-fix-value'
> (I would rather call it "eieio-persistent-restore-value") can't
> distinguish between objects and data that looks like an object.

That's the current state of the debate! There is a thread on emacs.devel
(or maybe a bug report, now I can't find it) from a while back where I
was discussing this with Stefan and others. We got as far as "there
should be no ambiguity/collisions about what's an object constructor and
what's a list." Then I got sidetracked by the continuing breakage of the
Gnus registry.

If you've got some ideas about this (very welcome!), maybe we should
move the discussion there.

Eric

Re: Any way to control which articles Gnus summary shows by default?

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> If VALUE isn't a consp, it falls through to the equivalent of:
>
> (nconc (nreverse nil) value)

I would expect that this would make my computer explode, but ok I see,
it returns VALUE.

> If you've got some ideas about this (very welcome!), maybe we should
> move the discussion there.

Ok, will try to find the discussion, thanks.


Michael.

Re: Any way to control which articles Gnus summary shows by default?

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> If VALUE isn't a consp, it falls through to the equivalent of:
>>
>> (nconc (nreverse nil) value)
>
> I would expect that this would make my computer explode, but ok I see,
> it returns VALUE.

I ran that test first so that you don't have to :)

>> If you've got some ideas about this (very welcome!), maybe we should
>> move the discussion there.
>
> Ok, will try to find the discussion, thanks.

Let me add a new patch to that bug discussion; I've been thinking about
this more since making the initial report.

Eric

Re: Any way to control which articles Gnus summary shows by default?

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> >> If VALUE isn't a consp, it falls through to the equivalent of:
> >>
> >> (nconc (nreverse nil) value)
> >
> > I would expect that this would make my computer explode, but ok I see,
> > it returns VALUE.
>
> I ran that test first so that you don't have to :)

Thanks.  Though, it's quite logical: the expression returns a chain of
zero conses with the last cdr being VALUE.  Or something like that...


Michael.

Re: Any way to control which articles Gnus summary shows by default?

[Robert Girault]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

[...]

> Some might think Gnus is over-customizable, but some aspects of
> customization, like posting styles, is both flexible and usable. It
> would be nice to provide similar access for group article selection,
> which is another core aspect of Gnus usage behavior.

I might second that motion.  But I'd first ask for help because I'm not
sure I'm in a position to ask for code writing.  I'm currently
interested in being shown ways of handling Gnus in a standard way when
it comes to finding a routine.  Please point me documents I can read to
discover more.  I've read An Introduction to Emacs Lisp by Robert
Chassel.  I can read.  Thank you.

For instance, after I send this message, I'm highly interested in
getting very clear sight of what happened to it.  I want to see my own
messages, spot typos and imperfections and definitely see when someone
has replied to it.  In fact, the whole thread.  Forever.  I love to see
my history.  I'd be very happy to always see backlog of all the threads
I've been involved with (forever) and those that I found interesting.  I
believe Gnus can do that today easily.