all messages for Emacs-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
From: Eric Abrahamsen <eric@ericabrahamsen.net>
To: Lars Ingebrigtsen <larsi@gnus.org>
Cc: 38011@debbugs.gnu.org
Subject: bug#38011: 27.0.50; [PATCH] WIP on allowing Gnus backends to return header data directly
Date: Sun, 03 Jan 2021 11:53:03 -0800	[thread overview]
Message-ID: <87zh1pkdb4.fsf@ericabrahamsen.net> (raw)
In-Reply-To: <877doua2g5.fsf@gnus.org> (Lars Ingebrigtsen's message of "Sun, 03 Jan 2021 08:45:30 +0100")


On 01/03/21 08:45 AM, Lars Ingebrigtsen wrote:
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Sure thing. It's in girzel/gnus-headers now. I made a few more sneaky
>> last minute changes, so yes... testing is in order.
>
> I'm now running this branch, and good news: No breakages so far.  :-)

Great!

>> It gets complicated because the cache and the agent need to mix their
>> saved headers into whatever newly-fetched headers we get from the
>> server. So instead of having them call `gnus-retrieve-headers' and
>> mixing their cached text into the nntp-server-buffer, they now call
>> `gnus-fetch-headers' on the server, which actually returns real headers.
>
> Yes, that was the main bit I was unsure of.  The braiding stuff tries to
> be efficient and avoid parsing things twice (or more) -- you usually get
> a bunch of headers from the NNTP server, and then you have even more
> headers in the agent/cache, and it stitches them all together as text,
> and parses the resulting mess.  (If I remember correctly; it's been at
> least a decade since I looked at that code.)
>
> Are headers parsed more than once now and then merged?

I certainly hope they're not parsed more than once, but it would
definitely be good to have more eyes on that part of the code. For
instance, `gnus-cache-retrieve-headers' finds cached articles in the
group, then uses (gnus-sorted-difference articles cached) to find
uncached articles. The uncached articles are sent to
`gnus-fetch-headers', and the cached articles are parsed from the cache
file using:

(gnus-get-newsgroup-headers-xover
 (gnus-sorted-difference
 cached uncached-articles))

Then the two sets of headers are merged and sorted. The agent does
something similar. I don't think the parsing should do any duplicate
work, though it's possible that adding the newly-received headers back
into the cache file is not optimal: it appends the new headers at the end
of the buffer, then re-sorts the whole buffer. 

Crud, I just realized that the agent will re-sort its cache file using gnus-agent-check-overview-buffer





  reply	other threads:[~2021-01-03 19:53 UTC|newest]

Thread overview: 29+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-10-31 21:34 bug#38011: 27.0.50; [PATCH] WIP on allowing Gnus backends to return header data directly Eric Abrahamsen
2019-11-01 14:12 ` Lars Ingebrigtsen
2019-11-01 18:41   ` Eric Abrahamsen
2019-11-01 20:52     ` Eric Abrahamsen
2019-11-02 14:49     ` Lars Ingebrigtsen
2019-11-07 23:21       ` Eric Abrahamsen
2019-11-08 21:03         ` Lars Ingebrigtsen
2019-11-08 21:43           ` Eric Abrahamsen
2019-11-08 21:58             ` Lars Ingebrigtsen
2020-03-29 19:50               ` Eric Abrahamsen
2020-04-30  4:50                 ` Lars Ingebrigtsen
2020-09-27  4:13                   ` Eric Abrahamsen
2020-09-27 12:16                     ` Lars Ingebrigtsen
2020-09-27 23:41                       ` Eric Abrahamsen
2021-01-02  3:18                         ` Eric Abrahamsen
2021-01-02  5:59                           ` Lars Ingebrigtsen
2021-01-02 20:49                             ` Eric Abrahamsen
2021-01-03  7:45                               ` Lars Ingebrigtsen
2021-01-03 19:53                                 ` Eric Abrahamsen [this message]
2021-01-04  9:05                                   ` Lars Ingebrigtsen
2021-01-04 18:09                                     ` Eric Abrahamsen
2021-01-05  8:47                                       ` Lars Ingebrigtsen
2021-01-05 17:02                                         ` Eric Abrahamsen
2021-01-17  5:00                                           ` Eric Abrahamsen
2021-01-18 10:48                                             ` Robert Pluim
2021-01-18 21:12                                               ` Eric Abrahamsen
2021-01-18 16:37                                             ` Lars Ingebrigtsen
2021-01-03 19:54                                 ` Eric Abrahamsen
2021-01-03 21:38                                   ` Eric Abrahamsen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87zh1pkdb4.fsf@ericabrahamsen.net \
    --to=eric@ericabrahamsen.net \
    --cc=38011@debbugs.gnu.org \
    --cc=larsi@gnus.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/emacs.git
	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.