unofficial mirror of help-gnu-emacs@gnu.org
 help / color / mirror / Atom feed
* Gnus: caching message headers?
@ 2020-09-07 15:23 Gregory Heytings via Users list for the GNU Emacs text editor
  2020-09-07 16:26 ` Eric Abrahamsen
  0 siblings, 1 reply; 28+ messages in thread
From: Gregory Heytings via Users list for the GNU Emacs text editor @ 2020-09-07 15:23 UTC (permalink / raw)
  To: help-gnu-emacs


Hi list,

I'm trying to configure Gnus to access my email through IMAP.

I'm too accustomed to traditional MUAs to switch to the "Gnus way" in one 
step, so I have:

(setq gnus-permanently-visible-groups ".*")
(setq gnus-parameters
    '((".*"
       (gnus-show-threads nil)
       (gnus-use-scoring nil)
       (gnus-article-sort-functions '(gnus-article-sort-by-number))
       (display . all))))

I would like to have, in the *Summary*, a complete list of the emails 
contained in a folder when I hit RET on its label.  Each time I do this 
however, Gnus asks me how many articles I want to retrieve, and issues a 
"UID FETCH 1:N (UID RFC822.SIZE BODYSTRUCTURE BODY.PEEK[HEADER.FIELDS 
(...)])" IMAP command, which can take quite some time to complete when N 
is large.

Is there a way to convince Gnus to cache the result of that command 
(without caching all emails), and to issue a command only for the new 
UIDs?  Caching the result of that command should not eat too much disk 
space.

I apologize in advance if that question is naïve, or if I missed some part 
of the manual.

Gregory


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
  2020-09-07 15:23 Gregory Heytings via Users list for the GNU Emacs text editor
@ 2020-09-07 16:26 ` Eric Abrahamsen
  2020-09-07 17:10   ` Gregory Heytings via Users list for the GNU Emacs text editor
  0 siblings, 1 reply; 28+ messages in thread
From: Eric Abrahamsen @ 2020-09-07 16:26 UTC (permalink / raw)
  To: help-gnu-emacs

Gregory Heytings via Users list for the GNU Emacs text editor
<help-gnu-emacs@gnu.org> writes:

> Hi list,
>
> I'm trying to configure Gnus to access my email through IMAP.
>
> I'm too accustomed to traditional MUAs to switch to the "Gnus way" in
> one step, so I have:
>
> (setq gnus-permanently-visible-groups ".*")
> (setq gnus-parameters
>    '((".*"
>       (gnus-show-threads nil)
>       (gnus-use-scoring nil)
>       (gnus-article-sort-functions '(gnus-article-sort-by-number))
>       (display . all))))
>
> I would like to have, in the *Summary*, a complete list of the emails
> contained in a folder when I hit RET on its label.  Each time I do
> this however, Gnus asks me how many articles I want to retrieve, and
> issues a "UID FETCH 1:N (UID RFC822.SIZE BODYSTRUCTURE
> BODY.PEEK[HEADER.FIELDS (...)])" IMAP command, which can take quite
> some time to complete when N is large.
>
> Is there a way to convince Gnus to cache the result of that command
> (without caching all emails), and to issue a command only for the new
> UIDs?  Caching the result of that command should not eat too much disk
> space.

No, not at the moment. You can avoid the prompt by customizing the value
of `gnus-large-newsgroup', but it's still going to retrieve all the
headers for the group.

There's currently no way around this, as Gnus only lets you have one
active Summary buffer at a time, which means all the old data is dumped
every time you switch groups. In the back of my head I have some ideas
for removing this restriction, but it will take a long time to get
there.

Eric




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
  2020-09-07 16:26 ` Eric Abrahamsen
@ 2020-09-07 17:10   ` Gregory Heytings via Users list for the GNU Emacs text editor
  2020-09-07 17:37     ` Eric Abrahamsen
  0 siblings, 1 reply; 28+ messages in thread
From: Gregory Heytings via Users list for the GNU Emacs text editor @ 2020-09-07 17:10 UTC (permalink / raw)
  To: help-gnu-emacs


Hi Eric,

Many thanks for your answer.

>> I would like to have, in the *Summary*, a complete list of the emails 
>> contained in a folder when I hit RET on its label.  Each time I do this 
>> however, Gnus asks me how many articles I want to retrieve, and issues 
>> a "UID FETCH 1:N (UID RFC822.SIZE BODYSTRUCTURE BODY.PEEK[HEADER.FIELDS 
>> (...)])" IMAP command, which can take quite some time to complete when 
>> N is large.
>>
>> Is there a way to convince Gnus to cache the result of that command 
>> (without caching all emails), and to issue a command only for the new 
>> UIDs?  Caching the result of that command should not eat too much disk 
>> space.
>
> No, not at the moment. You can avoid the prompt by customizing the value 
> of `gnus-large-newsgroup', but it's still going to retrieve all the 
> headers for the group.
>
> There's currently no way around this, as Gnus only lets you have one 
> active Summary buffer at a time, which means all the old data is dumped 
> every time you switch groups. In the back of my head I have some ideas 
> for removing this restriction, but it will take a long time to get 
> there.
>

I'm clearly not an expert, but would it not be enough to save and retrieve 
the contents of nntp-server-buffer in nnimap-retrieve-headers?  Or are you 
thinking about a more generic solution that would work with all backends?



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
  2020-09-07 17:10   ` Gregory Heytings via Users list for the GNU Emacs text editor
@ 2020-09-07 17:37     ` Eric Abrahamsen
  2020-09-07 17:49       ` wgreenhouse
  2020-09-07 17:50       ` Gregory Heytings via Users list for the GNU Emacs text editor
  0 siblings, 2 replies; 28+ messages in thread
From: Eric Abrahamsen @ 2020-09-07 17:37 UTC (permalink / raw)
  To: help-gnu-emacs

Gregory Heytings via Users list for the GNU Emacs text editor
<help-gnu-emacs@gnu.org> writes:

> Hi Eric,
>
> Many thanks for your answer.
>
>>> I would like to have, in the *Summary*, a complete list of the
>>> emails contained in a folder when I hit RET on its label.  Each
>>> time I do this however, Gnus asks me how many articles I want to
>>> retrieve, and issues a "UID FETCH 1:N (UID RFC822.SIZE
>>> BODYSTRUCTURE BODY.PEEK[HEADER.FIELDS (...)])" IMAP command, which
>>> can take quite some time to complete when N is large.
>>>
>>> Is there a way to convince Gnus to cache the result of that command
>>> (without caching all emails), and to issue a command only for the
>>> new UIDs?  Caching the result of that command should not eat too
>>> much disk space.
>>
>> No, not at the moment. You can avoid the prompt by customizing the
>> value of `gnus-large-newsgroup', but it's still going to retrieve
>> all the headers for the group.
>>
>> There's currently no way around this, as Gnus only lets you have one
>> active Summary buffer at a time, which means all the old data is
>> dumped every time you switch groups. In the back of my head I have
>> some ideas for removing this restriction, but it will take a long
>> time to get there.
>>
>
> I'm clearly not an expert, but would it not be enough to save and
> retrieve the contents of nntp-server-buffer in
> nnimap-retrieve-headers?  Or are you thinking about a more generic
> solution that would work with all backends?

We could do something ad-hoc for nnimap, but yes I'm thinking of
something more generic. All the header data (what's used to create the
Summary display) is held in variables that are local to the Summary
buffer, so in principle there's no reason we couldn't just leave the
local data in place when we leave the buffer. There are plenty of
obstacles to making it work correctly, but in principle I don't see why not.




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
  2020-09-07 17:37     ` Eric Abrahamsen
@ 2020-09-07 17:49       ` wgreenhouse
  2020-09-08 13:19         ` Eric S Fraga
  2020-09-08 13:37         ` Gregory Heytings via Users list for the GNU Emacs text editor
  2020-09-07 17:50       ` Gregory Heytings via Users list for the GNU Emacs text editor
  1 sibling, 2 replies; 28+ messages in thread
From: wgreenhouse @ 2020-09-07 17:49 UTC (permalink / raw)
  To: help-gnu-emacs

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Gregory Heytings via Users list for the GNU Emacs text editor
> <help-gnu-emacs@gnu.org> writes:
>
>> Hi Eric,
>>
>> Many thanks for your answer.
>>
>>>> I would like to have, in the *Summary*, a complete list of the
>>>> emails contained in a folder when I hit RET on its label.  Each
>>>> time I do this however, Gnus asks me how many articles I want to
>>>> retrieve, and issues a "UID FETCH 1:N (UID RFC822.SIZE
>>>> BODYSTRUCTURE BODY.PEEK[HEADER.FIELDS (...)])" IMAP command, which
>>>> can take quite some time to complete when N is large.
>>>>
>>>> Is there a way to convince Gnus to cache the result of that command
>>>> (without caching all emails), and to issue a command only for the
>>>> new UIDs?  Caching the result of that command should not eat too
>>>> much disk space.

[...]

> We could do something ad-hoc for nnimap, but yes I'm thinking of
> something more generic. All the header data (what's used to create the
> Summary display) is held in variables that are local to the Summary
> buffer, so in principle there's no reason we couldn't just leave the
> local data in place when we leave the buffer. There are plenty of
> obstacles to making it work correctly, but in principle I don't see why not.

The `agent' functionality might help here, in that it (among other
things) creates an on-disk cache of headers seen in a group handled by
the agent, so that it only needs to download headers again for articles
that have not been seen yet. See (info "(gnus) Agent as Cache") for more
on this.

--wgreenhouse




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
  2020-09-07 17:37     ` Eric Abrahamsen
  2020-09-07 17:49       ` wgreenhouse
@ 2020-09-07 17:50       ` Gregory Heytings via Users list for the GNU Emacs text editor
  2020-09-07 17:53         ` Eric Abrahamsen
  1 sibling, 1 reply; 28+ messages in thread
From: Gregory Heytings via Users list for the GNU Emacs text editor @ 2020-09-07 17:50 UTC (permalink / raw)
  To: help-gnu-emacs


>> I'm clearly not an expert, but would it not be enough to save and 
>> retrieve the contents of nntp-server-buffer in nnimap-retrieve-headers? 
>> Or are you thinking about a more generic solution that would work with 
>> all backends?
>
> We could do something ad-hoc for nnimap, but yes I'm thinking of 
> something more generic. All the header data (what's used to create the 
> Summary display) is held in variables that are local to the Summary 
> buffer, so in principle there's no reason we couldn't just leave the 
> local data in place when we leave the buffer. There are plenty of 
> obstacles to making it work correctly, but in principle I don't see why 
> not.
>

I would be happy with an ad-hoc solution for nnimap, which I suppose is 
the most used backend these days (but I could be wrong).  In fact I wasn't 
thinking of just leaving the local data in place when the summary buffer 
is left (which could eat a lot of memory), but instead to store them 
permanently in a cache on disk, and to read that file again instead of 
doing a "UID FETCH" on the server.



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
  2020-09-07 17:50       ` Gregory Heytings via Users list for the GNU Emacs text editor
@ 2020-09-07 17:53         ` Eric Abrahamsen
  0 siblings, 0 replies; 28+ messages in thread
From: Eric Abrahamsen @ 2020-09-07 17:53 UTC (permalink / raw)
  To: help-gnu-emacs

Gregory Heytings via Users list for the GNU Emacs text editor
<help-gnu-emacs@gnu.org> writes:

>>> I'm clearly not an expert, but would it not be enough to save and
>>> retrieve the contents of nntp-server-buffer in
>>> nnimap-retrieve-headers? Or are you thinking about a more generic
>>> solution that would work with all backends?
>>
>> We could do something ad-hoc for nnimap, but yes I'm thinking of
>> something more generic. All the header data (what's used to create
>> the Summary display) is held in variables that are local to the
>> Summary buffer, so in principle there's no reason we couldn't just
>> leave the local data in place when we leave the buffer. There are
>> plenty of obstacles to making it work correctly, but in principle I
>> don't see why not.
>>
>
> I would be happy with an ad-hoc solution for nnimap, which I suppose
> is the most used backend these days (but I could be wrong).  In fact I
> wasn't thinking of just leaving the local data in place when the
> summary buffer is left (which could eat a lot of memory), but instead
> to store them permanently in a cache on disk, and to read that file
> again instead of doing a "UID FETCH" on the server.

Check out wgreenhouse's suggestion!




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
  2020-09-07 17:49       ` wgreenhouse
@ 2020-09-08 13:19         ` Eric S Fraga
  2020-09-08 13:37         ` Gregory Heytings via Users list for the GNU Emacs text editor
  1 sibling, 0 replies; 28+ messages in thread
From: Eric S Fraga @ 2020-09-08 13:19 UTC (permalink / raw)
  To: help-gnu-emacs

On Monday,  7 Sep 2020 at 13:49, wgreenhouse wrote:
> The `agent' functionality might help here, in that it (among other

It will indeed and I second this suggestion!

I used to use the agent functionality for off-line reading of my emails
while commuting on train.  Now, with hotspot availability through my
phone, I've not used it for this purpose in several years.  However,
I've kept the groups under agent control for efficient handling of large
groups (news and imap) even if I don't necessarily download all the
articles.

-- 
Eric S Fraga via Emacs 28.0.50 & org 9.3.7 on Debian bullseye/sid




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
  2020-09-07 17:49       ` wgreenhouse
  2020-09-08 13:19         ` Eric S Fraga
@ 2020-09-08 13:37         ` Gregory Heytings via Users list for the GNU Emacs text editor
  2020-09-08 14:34           ` Stefan Monnier
  2020-09-08 16:12           ` wgreenhouse
  1 sibling, 2 replies; 28+ messages in thread
From: Gregory Heytings via Users list for the GNU Emacs text editor @ 2020-09-08 13:37 UTC (permalink / raw)
  To: wgreenhouse; +Cc: help-gnu-emacs


>
> The `agent' functionality might help here, in that it (among other 
> things) creates an on-disk cache of headers seen in a group handled by 
> the agent, so that it only needs to download headers again for articles 
> that have not been seen yet. See (info "(gnus) Agent as Cache") for more 
> on this.
>

That did the trick, thank you.  I had some difficulty to set it up, but 
now it seems to work.  Is there a way to tell gnus-agent to create only a 
cache of headers, that is, to not store a local copy of the emails that 
have been read?

Gregory



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
  2020-09-08 13:37         ` Gregory Heytings via Users list for the GNU Emacs text editor
@ 2020-09-08 14:34           ` Stefan Monnier
  2020-09-08 17:29             ` Eric Abrahamsen
  2020-09-08 21:52             ` Gregory Heytings via Users list for the GNU Emacs text editor
  2020-09-08 16:12           ` wgreenhouse
  1 sibling, 2 replies; 28+ messages in thread
From: Stefan Monnier @ 2020-09-08 14:34 UTC (permalink / raw)
  To: help-gnu-emacs

> it seems to work.  Is there a way to tell gnus-agent to create only a cache
> of headers, that is, to not store a local copy of the emails that have
> been read?

Interesting.  IMO nowadays MUAs should work by systematically
maintaining a full local copy of your mailboxes/folders.

[ And ideally, the IMAP protocol would be replaced by something like the
  Git protocol for that, so something like `git pull`, and `git push` would
  be used to fetch new mail and update flags, delete messages on the
  remote server, ...  ]


-- Stefan




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
  2020-09-08 13:37         ` Gregory Heytings via Users list for the GNU Emacs text editor
  2020-09-08 14:34           ` Stefan Monnier
@ 2020-09-08 16:12           ` wgreenhouse
  2020-09-08 20:39             ` Gregory Heytings via Users list for the GNU Emacs text editor
  1 sibling, 1 reply; 28+ messages in thread
From: wgreenhouse @ 2020-09-08 16:12 UTC (permalink / raw)
  To: Gregory Heytings; +Cc: help-gnu-emacs

Gregory Heytings <ghe@sdf.org> writes:

[...]

> That did the trick, thank you.  I had some difficulty to set it up,
> but now it seems to work.  Is there a way to tell gnus-agent to create
> only a cache of headers, that is, to not store a local copy of the
> emails that have been read?
>
> Gregory

I'm really not sure. It might be that you could set agent parameters in
such a way that no articles were eligible for downloading. But I'm not
sure the "agent cache" obeys the parameters; I have a feeling it just
downloads every message you read unconditionally.



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
  2020-09-08 14:34           ` Stefan Monnier
@ 2020-09-08 17:29             ` Eric Abrahamsen
  2020-09-08 21:52             ` Gregory Heytings via Users list for the GNU Emacs text editor
  1 sibling, 0 replies; 28+ messages in thread
From: Eric Abrahamsen @ 2020-09-08 17:29 UTC (permalink / raw)
  To: help-gnu-emacs

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> it seems to work.  Is there a way to tell gnus-agent to create only a cache
>> of headers, that is, to not store a local copy of the emails that have
>> been read?
>
> Interesting.  IMO nowadays MUAs should work by systematically
> maintaining a full local copy of your mailboxes/folders.

Setting `gnus-use-cache' to t will save everything. I can't remember if
there are any knobs for limiting the total disk space used, etc.

> [ And ideally, the IMAP protocol would be replaced by something like the
>   Git protocol for that, so something like `git pull`, and `git push` would
>   be used to fetch new mail and update flags, delete messages on the
>   remote server, ...  ]

Now this is just crazy talk :) But have you looked a JMAP?

https://jmap.io/spec.html




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
  2020-09-08 16:12           ` wgreenhouse
@ 2020-09-08 20:39             ` Gregory Heytings via Users list for the GNU Emacs text editor
  0 siblings, 0 replies; 28+ messages in thread
From: Gregory Heytings via Users list for the GNU Emacs text editor @ 2020-09-08 20:39 UTC (permalink / raw)
  To: wgreenhouse; +Cc: help-gnu-emacs


>
>> That did the trick, thank you.  I had some difficulty to set it up, but 
>> now it seems to work.  Is there a way to tell gnus-agent to create only 
>> a cache of headers, that is, to not store a local copy of the emails 
>> that have been read?
>
> I'm really not sure. It might be that you could set agent parameters in 
> such a way that no articles were eligible for downloading. But I'm not 
> sure the "agent cache" obeys the parameters; I have a feeling it just 
> downloads every message you read unconditionally.
>

A simple and direct way to do what I wanted is:

(defun gnus-agent-do-not-store-article (fun &rest args) "Do nothing") 
(advice-add 'gnus-agent-store-article :around #'gnus-agent-do-not-store-article)



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
  2020-09-08 14:34           ` Stefan Monnier
  2020-09-08 17:29             ` Eric Abrahamsen
@ 2020-09-08 21:52             ` Gregory Heytings via Users list for the GNU Emacs text editor
  2020-09-08 22:33               ` Stefan Monnier
  1 sibling, 1 reply; 28+ messages in thread
From: Gregory Heytings via Users list for the GNU Emacs text editor @ 2020-09-08 21:52 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: help-gnu-emacs


>
>> Is there a way to tell gnus-agent to create only a cache of headers, 
>> that is, to not store a local copy of the emails that have been read?
>
> Interesting.  IMO nowadays MUAs should work by systematically 
> maintaining a full local copy of your mailboxes/folders.
>

I (respectfully!) disagree.  This means that each MUA has to reinvent what 
already exists on the server, that is, to reinvent its own IMAP-like 
implementation, to store emails and to locate information in emails.  It 
is also a waste of resources, given that only a very small fraction of the 
emails are read more than once.  A "pure IMAP" MUA only has to query the 
server through the IMAP protocol, and only manages (1) a list of folders 
and (2) a list of email summaries in each folder.  In fact it could even 
manage nothing, and query the list of folders each time it is started, and 
create a summary of the N most recent emails each time a folder is opened.



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
  2020-09-08 21:52             ` Gregory Heytings via Users list for the GNU Emacs text editor
@ 2020-09-08 22:33               ` Stefan Monnier
  0 siblings, 0 replies; 28+ messages in thread
From: Stefan Monnier @ 2020-09-08 22:33 UTC (permalink / raw)
  To: Gregory Heytings; +Cc: help-gnu-emacs

>>> Is there a way to tell gnus-agent to create only a cache of headers, that
>>> is, to not store a local copy of the emails that have been read?
>> Interesting.  IMO nowadays MUAs should work by systematically maintaining
>> a full local copy of your mailboxes/folders.
> I (respectfully!) disagree.  This means that each MUA has to reinvent what
> already exists on the server, that is, to reinvent its own IMAP-like
> implementation, to store emails and to locate information in emails.

If your MUA caches some email messages locally across sessions, then it
probably already has all that complexity ;-)


        Stefan




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
@ 2020-09-10  8:38 Ozhap
  2020-09-10  9:00 ` Gregory Heytings via Users list for the GNU Emacs text editor
  0 siblings, 1 reply; 28+ messages in thread
From: Ozhap @ 2020-09-10  8:38 UTC (permalink / raw)
  To: ghe; +Cc: help-gnu-emacs

> Is there a way to tell gnus-agent to create only a cache of headers,
> that is, to not store a local copy of the emails that have been read?

I don't think it does. From info '(gnus)Agent Caveats':

> “If I read an article while plugged, do they get entered into the Agent?”
> 
>      *No*.  If you want this behavior, add
>      ‘gnus-agent-fetch-selected-article’ to ‘gnus-select-article-hook’.

--
Ozhap



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
  2020-09-10  8:38 Gnus: caching message headers? Ozhap
@ 2020-09-10  9:00 ` Gregory Heytings via Users list for the GNU Emacs text editor
  2020-09-10  9:34   ` Ozhap
  2020-09-10 13:16   ` wgreenhouse
  0 siblings, 2 replies; 28+ messages in thread
From: Gregory Heytings via Users list for the GNU Emacs text editor @ 2020-09-10  9:00 UTC (permalink / raw)
  To: Ozhap; +Cc: help-gnu-emacs


>
>> Is there a way to tell gnus-agent to create only a cache of headers, 
>> that is, to not store a local copy of the emails that have been read?
>
> I don't think it does. From info '(gnus)Agent Caveats':
>
>> “If I read an article while plugged, do they get entered into the 
>> Agent?”
>>
>> *No*.  If you want this behavior, add 
>> ‘gnus-agent-fetch-selected-article’ to ‘gnus-select-article-hook’.
>

That's what the doc says, and I'm not sure what it means exactly, but with 
the current version of Gnus, and with `gnus-select-article-hook' set to 
nil (the default), a local copy of the emails is indeed entered into the 
agent cache, by `gnus-agent-store-article'.  The backtrace is

gnus-agent-store-article
gnus-request-article-this-buffer 
gnus-article-prepare
gnus-summary-display-article 
gnus-summary-select-article


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
  2020-09-10  9:00 ` Gregory Heytings via Users list for the GNU Emacs text editor
@ 2020-09-10  9:34   ` Ozhap
  2020-09-10  9:45     ` Ozhap
  2020-09-10 13:16   ` wgreenhouse
  1 sibling, 1 reply; 28+ messages in thread
From: Ozhap @ 2020-09-10  9:34 UTC (permalink / raw)
  To: Gregory Heytings; +Cc: help-gnu-emacs

Gregory Heytings <ghe@sdf.org> writes:

>>> Is there a way to tell gnus-agent to create only a cache of headers, 
>>> that is, to not store a local copy of the emails that have been read?
>>
>> I don't think it does. From info '(gnus)Agent Caveats':
>>
>>> “If I read an article while plugged, do they get entered into the 
>>> Agent?”
>>>
>>> *No*.  If you want this behavior, add 
>>> ‘gnus-agent-fetch-selected-article’ to ‘gnus-select-article-hook’.
>>
>
> That's what the doc says, and I'm not sure what it means exactly, but with 
> the current version of Gnus, and with `gnus-select-article-hook' set to 
> nil (the default), a local copy of the emails is indeed entered into the 
> agent cache, by `gnus-agent-store-article'.  The backtrace is

You should reset the default agent category predicate from 'short' to
'false' if you havent. See '(gnus)Category Syntax' for details.

Used to work properly for me when I had it set up like that. I'm using 26.3.

--
Ozhap



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
  2020-09-10  9:34   ` Ozhap
@ 2020-09-10  9:45     ` Ozhap
  0 siblings, 0 replies; 28+ messages in thread
From: Ozhap @ 2020-09-10  9:45 UTC (permalink / raw)
  To: Gregory Heytings; +Cc: help-gnu-emacs

Ozhap <ozhap@vollbio.de> writes:

> Gregory Heytings <ghe@sdf.org> writes:
>
>>>> Is there a way to tell gnus-agent to create only a cache of headers, 
>>>> that is, to not store a local copy of the emails that have been read?
>>>
>>> I don't think it does. From info '(gnus)Agent Caveats':
>>>
>>>> “If I read an article while plugged, do they get entered into the 
>>>> Agent?”
>>>>
>>>> *No*.  If you want this behavior, add 
>>>> ‘gnus-agent-fetch-selected-article’ to ‘gnus-select-article-hook’.
>>>
>>
>> That's what the doc says, and I'm not sure what it means exactly, but with 
>> the current version of Gnus, and with `gnus-select-article-hook' set to 
>> nil (the default), a local copy of the emails is indeed entered into the 
>> agent cache, by `gnus-agent-store-article'.  The backtrace is
>
> You should reset the default agent category predicate from 'short' to
> 'false' if you havent. See '(gnus)Category Syntax' for details.

Scratch that. I guess it's only for gnus-agent-fetch-session.

> Used to work properly for me when I had it set up like that. I'm using 26.3.

In fact I was once trying to get the opposite behaviour to your desired
one - to store read emails, which was not happening by default. And
finally achieved it with the above option.

--
Ozhap



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
  2020-09-10  9:00 ` Gregory Heytings via Users list for the GNU Emacs text editor
  2020-09-10  9:34   ` Ozhap
@ 2020-09-10 13:16   ` wgreenhouse
  1 sibling, 0 replies; 28+ messages in thread
From: wgreenhouse @ 2020-09-10 13:16 UTC (permalink / raw)
  To: help-gnu-emacs

Gregory Heytings via Users list for the GNU Emacs text editor
<help-gnu-emacs@gnu.org> writes:

[...]

> That's what the doc says, and I'm not sure what it means exactly, but
> with the current version of Gnus, and with `gnus-select-article-hook'
> set to nil (the default), a local copy of the emails is indeed entered
> into the agent cache, by `gnus-agent-store-article'.  The backtrace is
>
> gnus-agent-store-article
> gnus-request-article-this-buffer gnus-article-prepare
> gnus-summary-display-article gnus-summary-select-article

That Info page could be more explicit that `gnus-agent-cache' means that
articles *will* be added to the agent as they are read.




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
@ 2020-09-10 23:41 Ozhap
  2020-09-11 22:59 ` Eric Abrahamsen
  0 siblings, 1 reply; 28+ messages in thread
From: Ozhap @ 2020-09-10 23:41 UTC (permalink / raw)
  To: wgreenhouse; +Cc: help-gnu-emacs

>> “If I read an article while plugged, do they get entered into the 
>> Agent?”
>>
>> *No*.  If you want this behavior, add 
>> ‘gnus-agent-fetch-selected-article’ to ‘gnus-select-article-hook’.

> That Info page could be more explicit that `gnus-agent-cache' means that
> articles *will* be added to the agent as they are read.

That's how it stands now, but in info '(gnus)Agent as Cache' it says:

"..Gnus normally only downloads *headers* once, and stores them in the
Agent"

"Articles are *not* cached in the Agent by default though (that would
potentially consume lots of disk space), but if you have *already
downloaded an article into the Agent*, Gnus will not download the
article from the server again but use the locally stored copy instead."

I did some digging, and this behaviour was changed due to bug#8502 where
Lars concludes (erroneously, on the face of it):

> Let's see...  caching...  so if you're online (in an agentised group)
> and select an article, you want the article to be saved in the Agent
> directory?  That sounds eminently reasonable, and I thought that was
> supposed to happen, but I can't find any code for doing that...  hm...
> Nope.

Ref:
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=8502

--
Ozhap



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
  2020-09-10 23:41 Ozhap
@ 2020-09-11 22:59 ` Eric Abrahamsen
  2020-09-12 23:08   ` Ozhap
  0 siblings, 1 reply; 28+ messages in thread
From: Eric Abrahamsen @ 2020-09-11 22:59 UTC (permalink / raw)
  To: Ozhap; +Cc: wgreenhouse, help-gnu-emacs

Ozhap <ozhap@vollbio.de> writes:

>>> “If I read an article while plugged, do they get entered into the 
>>> Agent?”
>>>
>>> *No*.  If you want this behavior, add 
>>> ‘gnus-agent-fetch-selected-article’ to ‘gnus-select-article-hook’.
>
>> That Info page could be more explicit that `gnus-agent-cache' means that
>> articles *will* be added to the agent as they are read.
>
> That's how it stands now, but in info '(gnus)Agent as Cache' it says:
>
> "..Gnus normally only downloads *headers* once, and stores them in the
> Agent"
>
> "Articles are *not* cached in the Agent by default though (that would
> potentially consume lots of disk space), but if you have *already
> downloaded an article into the Agent*, Gnus will not download the
> article from the server again but use the locally stored copy instead."
>
> I did some digging, and this behaviour was changed due to bug#8502 where
> Lars concludes (erroneously, on the face of it):
>
>> Let's see...  caching...  so if you're online (in an agentised group)
>> and select an article, you want the article to be saved in the Agent
>> directory?  That sounds eminently reasonable, and I thought that was
>> supposed to happen, but I can't find any code for doing that...  hm...
>> Nope.
>
> Ref:
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=8502

If I were you I would do this up as a proper bug report. Lars is now an
Emacs maintainer, so he'll be obliged to look at it :)

One way or another there is a mismatch between documentation,
customization options, and actual observed behavior. I think if you can
make it clear what that mismatch is, we stand a good chance of getting
it fixed.



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
  2020-09-11 22:59 ` Eric Abrahamsen
@ 2020-09-12 23:08   ` Ozhap
  2020-09-12 23:29     ` Eric Abrahamsen
  0 siblings, 1 reply; 28+ messages in thread
From: Ozhap @ 2020-09-12 23:08 UTC (permalink / raw)
  To: Eric Abrahamsen; +Cc: wgreenhouse, help-gnu-emacs

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Ozhap <ozhap@vollbio.de> writes:
>
>>>> “If I read an article while plugged, do they get entered into the 
>>>> Agent?”
>>>>
>>>> *No*.  If you want this behavior, add 
>>>> ‘gnus-agent-fetch-selected-article’ to ‘gnus-select-article-hook’.
>>
>>> That Info page could be more explicit that `gnus-agent-cache' means that
>>> articles *will* be added to the agent as they are read.
>>
>> That's how it stands now, but in info '(gnus)Agent as Cache' it says:
>>
>> "..Gnus normally only downloads *headers* once, and stores them in the
>> Agent"
>>
>> "Articles are *not* cached in the Agent by default though (that would
>> potentially consume lots of disk space), but if you have *already
>> downloaded an article into the Agent*, Gnus will not download the
>> article from the server again but use the locally stored copy instead."
>>
>> I did some digging, and this behaviour was changed due to bug#8502 where
>> Lars concludes (erroneously, on the face of it):
>>
>>> Let's see...  caching...  so if you're online (in an agentised group)
>>> and select an article, you want the article to be saved in the Agent
>>> directory?  That sounds eminently reasonable, and I thought that was
>>> supposed to happen, but I can't find any code for doing that...  hm...
>>> Nope.
>>
>> Ref:
>> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=8502
>
> If I were you I would do this up as a proper bug report. Lars is now an
> Emacs maintainer, so he'll be obliged to look at it :)
>
> One way or another there is a mismatch between documentation,
> customization options, and actual observed behavior. I think if you can
> make it clear what that mismatch is, we stand a good chance of getting
> it fixed.

Here you go. I think I did good :)
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=43356

--
Ozhap



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
  2020-09-12 23:08   ` Ozhap
@ 2020-09-12 23:29     ` Eric Abrahamsen
  2020-10-04  9:12       ` Madhu
  0 siblings, 1 reply; 28+ messages in thread
From: Eric Abrahamsen @ 2020-09-12 23:29 UTC (permalink / raw)
  To: Ozhap; +Cc: wgreenhouse, help-gnu-emacs

Ozhap <ozhap@vollbio.de> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Ozhap <ozhap@vollbio.de> writes:
>>
>>>>> “If I read an article while plugged, do they get entered into the 
>>>>> Agent?”
>>>>>
>>>>> *No*.  If you want this behavior, add 
>>>>> ‘gnus-agent-fetch-selected-article’ to ‘gnus-select-article-hook’.
>>>
>>>> That Info page could be more explicit that `gnus-agent-cache' means that
>>>> articles *will* be added to the agent as they are read.
>>>
>>> That's how it stands now, but in info '(gnus)Agent as Cache' it says:
>>>
>>> "..Gnus normally only downloads *headers* once, and stores them in the
>>> Agent"
>>>
>>> "Articles are *not* cached in the Agent by default though (that would
>>> potentially consume lots of disk space), but if you have *already
>>> downloaded an article into the Agent*, Gnus will not download the
>>> article from the server again but use the locally stored copy instead."
>>>
>>> I did some digging, and this behaviour was changed due to bug#8502 where
>>> Lars concludes (erroneously, on the face of it):
>>>
>>>> Let's see...  caching...  so if you're online (in an agentised group)
>>>> and select an article, you want the article to be saved in the Agent
>>>> directory?  That sounds eminently reasonable, and I thought that was
>>>> supposed to happen, but I can't find any code for doing that...  hm...
>>>> Nope.
>>>
>>> Ref:
>>> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=8502
>>
>> If I were you I would do this up as a proper bug report. Lars is now an
>> Emacs maintainer, so he'll be obliged to look at it :)
>>
>> One way or another there is a mismatch between documentation,
>> customization options, and actual observed behavior. I think if you can
>> make it clear what that mismatch is, we stand a good chance of getting
>> it fixed.
>
> Here you go. I think I did good :)
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=43356

I saw it, I agree it's good!



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
  2020-09-12 23:29     ` Eric Abrahamsen
@ 2020-10-04  9:12       ` Madhu
  0 siblings, 0 replies; 28+ messages in thread
From: Madhu @ 2020-10-04  9:12 UTC (permalink / raw)
  To: help-gnu-emacs

* Eric Abrahamsen <87lfheefq4.fsf@ericabrahamsen.net> :
Wrote on Sat, 12 Sep 2020 16:29:39 -0700:

> Ozhap <ozhap@vollbio.de> writes:
>
>> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>>
>>> Ozhap <ozhap@vollbio.de> writes:
>>>
>>>>>> "If I read an article while plugged, do they get entered into the
>>>>>> Agent?"
>>>>>>
>>>>>> *No*.  If you want this behavior, add
>>>>>> `gnus-agent-fetch-selected-article' to `gnus-select-article-hook'.
>>>>
>>>>> That Info page could be more explicit that `gnus-agent-cache' means that
>>>>> articles *will* be added to the agent as they are read.
>>>>
>>>> That's how it stands now, but in info '(gnus)Agent as Cache' it says:
>>>>
>>>> "..Gnus normally only downloads *headers* once, and stores them in the
>>>> Agent"
>>>>
>>>> "Articles are *not* cached in the Agent by default though (that
>>>> would potentially consume lots of disk space), but if you have
>>>> *already downloaded an article into the Agent*, Gnus will not
>>>> download the article from the server again but use the locally
>>>> stored copy instead."
>>>>
>>>> I did some digging, and this behaviour was changed due to bug#8502
>>>> where Lars concludes (erroneously, on the face of it):
>>>>
>>>>> Let's see...  caching...  so if you're online (in an agentised
>>>>> group) and select an article, you want the article to be saved in
>>>>> the Agent directory?  That sounds eminently reasonable, and I
>>>>> thought that was supposed to happen, but I can't find any code for
>>>>> doing that...  hm...  Nope.
>>>>
>>>> Ref:
>>>> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=8502
>>>
>>> If I were you I would do this up as a proper bug report. Lars is now
>>> an Emacs maintainer, so he'll be obliged to look at it :)
>>>
>>> One way or another there is a mismatch between documentation,
>>> customization options, and actual observed behavior. I think if you
>>> can make it clear what that mismatch is, we stand a good chance of
>>> getting it fixed.
>>
>> Here you go. I think I did good :)
>> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=43356
>
> I saw it, I agree it's good!

Lars followed the instructions like a good boy and and has obsoleted
gnus-agent-store-article and removed that functionality.

| commit def34a20766ea5179afd5e5ed090a2b86fcccb5e
| Author:     Lars Ingebrigtsen <larsi@gnus.org>
| AuthorDate: Sun Sep 13 16:46:11 2020 +0200
|
|     "If I read an article while plugged, do they get entered into the
|     Agent?"
|
|          *No*. If you want this behavior, add
|          `gnus-agent-fetch-selected-article' to
|          `gnus-select-article-hook'.
|


However the overhead of calling gnus-agent-fetch-selected-article is too
much in my 100s of (NNTP) news groups and the several thousands of
articles with complicated reading history. "Yuuge lists of gnus cons
data structures".  The overhead is not acceptable.

The earlier behaviour was exactly what I wanted for NNTP (I dont use
gnus for mail) and I have been using the feature on selective servers
ever since it was added, relying on gnus to back up my memory (If I read
an article gnus can help me remember where I read it). I don't give a
shit about gmail users and I'd suggest reverting it unobsoleting
gnus-agent-store-article and yet another switch for those who want to
keep the original very useful behaviour code path. For those who prefer
to control their own data instead of selling it (along with their access
patterns and their souls) to google.




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
@ 2020-10-07 11:39 Ozhap
  0 siblings, 0 replies; 28+ messages in thread
From: Ozhap @ 2020-10-07 11:39 UTC (permalink / raw)
  To: enometh; +Cc: help-gnu-emacs

> However the overhead of calling gnus-agent-fetch-selected-article is too
> much

Whoops! Omitting the relevant xkcd :) I should learn to curb my
enthusiasm...

Perhaps you could try advising gnus-request-article to call
gnus-agent-store-article after, when it has a non-nil 3rd argument (it
doesn't seem to be used with it anywhere else).

But probably there should be a new hook. Or of course, reverting it and
rewriting the manual instead.

--
Ozhap



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
@ 2020-10-08  1:59 Ozhap
  0 siblings, 0 replies; 28+ messages in thread
From: Ozhap @ 2020-10-08  1:59 UTC (permalink / raw)
  To: enometh; +Cc: help-gnu-emacs

> But probably there should be a new hook. Or of course, reverting it and
> rewriting the manual instead.

An easier way would be to use gnus-agent-store-article instead of
gnus-agent-fetch-articles (which is designed for many messages) in
gnus-agent-fetch-selected-article. Please open a bug report so that we
can fix it there.

(A related problem seems to be the download-mark not being updated in
the previous arrangement)

--
Ozhap



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Gnus: caching message headers?
@ 2020-10-08  3:28 Ozhap
  0 siblings, 0 replies; 28+ messages in thread
From: Ozhap @ 2020-10-08  3:28 UTC (permalink / raw)
  To: enometh; +Cc: help-gnu-emacs

Ozhap wrote:

>> But probably there should be a new hook. Or of course, reverting it and
>> rewriting the manual instead.
>
> An easier way would be to use gnus-agent-store-article instead of
> gnus-agent-fetch-articles (which is designed for many messages) in
> gnus-agent-fetch-selected-article. Please open a bug report so that we
> can fix it there.

That should read: Check for stored article as in
gnus-agent-store-article in -agent-fetch-selected-article, before
fetching. Well, the previous code (or a new hook) seems better :|



^ permalink raw reply	[flat|nested] 28+ messages in thread

end of thread, other threads:[~2020-10-08  3:28 UTC | newest]

Thread overview: 28+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-09-10  8:38 Gnus: caching message headers? Ozhap
2020-09-10  9:00 ` Gregory Heytings via Users list for the GNU Emacs text editor
2020-09-10  9:34   ` Ozhap
2020-09-10  9:45     ` Ozhap
2020-09-10 13:16   ` wgreenhouse
  -- strict thread matches above, loose matches on Subject: below --
2020-10-08  3:28 Ozhap
2020-10-08  1:59 Ozhap
2020-10-07 11:39 Ozhap
2020-09-10 23:41 Ozhap
2020-09-11 22:59 ` Eric Abrahamsen
2020-09-12 23:08   ` Ozhap
2020-09-12 23:29     ` Eric Abrahamsen
2020-10-04  9:12       ` Madhu
2020-09-07 15:23 Gregory Heytings via Users list for the GNU Emacs text editor
2020-09-07 16:26 ` Eric Abrahamsen
2020-09-07 17:10   ` Gregory Heytings via Users list for the GNU Emacs text editor
2020-09-07 17:37     ` Eric Abrahamsen
2020-09-07 17:49       ` wgreenhouse
2020-09-08 13:19         ` Eric S Fraga
2020-09-08 13:37         ` Gregory Heytings via Users list for the GNU Emacs text editor
2020-09-08 14:34           ` Stefan Monnier
2020-09-08 17:29             ` Eric Abrahamsen
2020-09-08 21:52             ` Gregory Heytings via Users list for the GNU Emacs text editor
2020-09-08 22:33               ` Stefan Monnier
2020-09-08 16:12           ` wgreenhouse
2020-09-08 20:39             ` Gregory Heytings via Users list for the GNU Emacs text editor
2020-09-07 17:50       ` Gregory Heytings via Users list for the GNU Emacs text editor
2020-09-07 17:53         ` Eric Abrahamsen

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).