bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom)

[Eric Abrahamsen <eric@ericabrahamsen.net>]

Daniel Semyonov via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs@gnu.org> writes:

> Sorry, I missed this email (I didn't realize debbugs doesn't forward all
> messages to the author of the "bug report").

No worries, usually I do a reply all, but sometimes forget.

>>>>>> Eric Abrahamsen writes:
>
>     > Huh! In all my years of using and working on Gnus I've never used
>     > a "foreign" server, nor have I really understood what it means. At
>     > some point it would be good to make sure this works via other
>     > entrypoints as well, but so far so good.
>
> AFAIK it should work for any entry point which allows you to define a
> new server or select method (Atom feeds are represented as servers with
> a single group, so there is no way to create a group without a
> pre-existing corresponding server).
> I'll be honest though, I only ever use this method and
> 'gnus-secondary-select-methods', so I'm not really sure which other
> entry points there are.

What I usually start with is "G m", for `gnus-group-make-group'. That
prompts for a group name, and also a backend. The viable list of
backends includes all your currently-defined servers, plus the generic
symbols for all defined backends, in this case a plain 'nnatom.

As you point out, that doesn't work for nnatom because each group will
need its own server. What *does* work is to create a group this way,
then put point on the new group and hit "M-e", for
`gnus-group-edit-group-method', put in the actual URL, then "C-c C-c".

Obviously this is horribly baroque and it would be better if
`gnus-group-make-group' could prompt for the URL. No matter what, it
would be good to document the various ways to do this.

>     > Regarding your earlier question about having this backend handle
>     > RSS too, I'm not aware of any significant difference between the
>     > two beyond the format of the XML. Is that true?
>
> Yes, even the XML format is very similar.
>
>     > If so, it seems like it would make most sense to merge the
>     > code. Have you looked at nnrss? It would be good to know if there
>     > was anything in there worth stealing for nnatom -- if one of them
>     > has a faster parser than the other, for instance, or better logic
>     > for keeping updates efficient.
>
> The issue with merging the two is that nnrss saves feed data differently
> (both on disk and in memory), and also represents each feed as a group,
> with a virtual "server" holding all groups.
> I'm not sure if it's possible/a good idea to migrate feed data from
> nnrss to a hypothetical merged backend, at least not automatically.
>
> Stefan floated the idea of adding RSS support, deprecating nnrss and
> creating an interactive migration command - so users who wish to migrate
> will have to do so manually (which should also potentially allow asking
> the user some questions if the migration includes some non-trivial
> steps).
>
> nnrss does do some cool stuff that nnatom doesn't, though (for example,
> it tries very hard to find an RSS feed when you provide it with a link
> to a website, while nnatom currently requires a direct link to a feed).

Right, I should have been more explicit here -- what I was thinking is
what Stefan suggests: just deprecate nnrss altogether. If it does
anything cool like feed discovery, just steal that code. If you're
inclined to be nice enough to provide a migration function, that's a
bonus.

>     > I just subscribed to a feed with nnrss, and noticed that after I
>     > marked all the items in the feed as read, I couldn't re-enter the
>     > group and see the old items. It gave me "Can't select group". So
>     > that's not very encouraging.
>
> Honestly, from my experience nnrss has many small issues (although I
> never encountered this exact issue).  It is partly why I developed
> nnatom (previously I used a hack documented on the Emacs wiki which
> converted Atom feeds to RSS feeds on the fly).
>
>     > If you do want to expand this to be a general "feed" backend, we
>     > might want to do some boring things like rename it nnfeed.el,
>
> This is the name I thought of too, and I guess if two people thought of
> it independently it's probably fine.

Good!
  
>     > and add support for ridiculous things like JSON feed[0] (why?!?). I
>     > assume a derived backend could handle JSON feeds by setting the
>     > appropriate values for the `nnatom-read-*-function' deffoos?
>
> I actually attached a derivative "nnjsonfeed" backend I made as an
> experiment to one of my previous messages (it doesn't work with the
> current version of the patch, but it won't be hard to fix).
>
> It wasn't 100% conforming to the standard (JSON feeds support some weird
> features like pagination, which can actually be supported very well in
> theory by nnatom, but I didn't feel like doing that), but it worked and
> it was very easy to make.

I missed nnjsonfeed, sorry. That's great you're already working in this
direction. I don't think we need to support absolutely everything it
does (what would pagination look like in Gnus?), just the basics to get
started with.

>     > One of the awkward things about nnrss is that it's never really
>     > fit well into Gnus' one-server-many-groups paradigm, which you
>     > allude to in the nnatom Info section. Do you have any further
>     > ideas in that direction?
>
> Well, nnatom theoretically supports this paradigm, but it doesn't do
> this with Atom feeds, since it doesn't really make sense IMO.
> However, there is a standardized way to include links to Atom feeds in
> HTML documents, so it might be a good idea to support adding them as
> servers which show any linked Atom feeds as groups.

It's also perfectly possible that a single website would publish
multiple Atom feeds, right? Like weather.gov, for instance. That would
have an added benefit of letting us simplify the server name from the
full URL (https://alerts.weather.gov/cap/wa.php?x=0) which looks ugly in
the Group buffer, to just eg alerts.weather.gov.

But I guess I don't know why it would matter, really. The only practical
use for wanting multiple feeds under a single server would be to set
some configuration at the server level, that would apply to all feeds
from that server. And at the moment I don't think there are many such
settings (right?).

> I also have a (very experimental) derivative backend using the API of
> some website, which exposes various categories of content, which I
> expose as groups (this is what I use to test support for multiple groups
> in a single server).
>
> As a side note: I had hoped to publish an updated version of the patch
> by now, but unfortunately I was a bit under the weather lately, so I
> didn't feel like working on it.

No rush!

Eric