bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom)

[Daniel Semyonov via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>]

[-- Attachment #1: Type: text/plain, Size: 1265 bytes --]

>>>>> Daniel Semyonov writes:

    > I've procrastinated on this for a while and then due to unforeseen
    > circumstances I've been able to use Emacs on more than my phone
    > only a few times in the past two months or so.  I don't think
    > these circumstances will change soon, so I'm not sure when I'll be
    > able to work on this again unfortunately...

Well as luck would have it I had a rare opportunity to spend a few hours
at home today with no prior business I had to take care of (and this was
also fresh on my mind for a change) so I decided to try finish this
while I could while I can use a normal computer.

Attached is a new version of the patch which is identical in
functionality (more or less) to the last one but nnatom is implemented
on top of an abstract nnfeed backend.  I made a lot of effort to
document nnfeed, hopefully everything is clear, but I'm honestly very
tired and might have made some mistakes.  The docs in nnfeed.el are
supposed to be all that's needed for someone basing a backend on it, so
I'd like to know if it's clear.

I just realized this patch still can't be merged though, NEWS and the
manual need updated.  I'll do it on my phone tomorrow if I have the
time.  Hopefully this can be merged soon finally though


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch --]
[-- Type: text/x-patch, Size: 48167 bytes --]

From dbb989d9584bdefab2470cbe8ef86103e2d263aa Mon Sep 17 00:00:00 2001
From: Daniel Semyonov <daniel@dsemy.com>
Date: Wed, 21 Jun 2023 10:05:04 +0300
Subject: [PATCH] Gnus: Add back end for Atom feeds (nnatom)

* lisp/gnus/gnus.el (gnus-valid-select-methods): Add entry for nnatom.
* lisp/gnus/nnatom.el:
* lisp/gnus/nnfeed.el: New file.
* doc/misc/gnus.texi:
* etc/NEWS: Document nnatom
---
 doc/misc/gnus.texi  |  58 ++++
 etc/NEWS            |   7 +
 lisp/gnus/gnus.el   |   1 +
 lisp/gnus/nnatom.el | 277 ++++++++++++++++++
 lisp/gnus/nnfeed.el | 683 ++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 1026 insertions(+)
 create mode 100644 lisp/gnus/nnatom.el
 create mode 100644 lisp/gnus/nnfeed.el

diff --git a/doc/misc/gnus.texi b/doc/misc/gnus.texi
index 586e4b94ba1..6691b0fba7b 100644
--- a/doc/misc/gnus.texi
+++ b/doc/misc/gnus.texi
@@ -715,6 +715,7 @@ Top
 
 * Web Searches::                Creating groups from articles that match a string.
 * RSS::                         Reading RDF site summary.
+* Atom::                        Reading Atom Syndication Format feeds.
 
 Other Sources
 
@@ -17250,6 +17251,7 @@ Browsing the Web
 @menu
 * Web Searches::                Creating groups from articles that match a string.
 * RSS::                         Reading RDF site summary.
+* Atom::                        Reading Atom Syndication Format feeds.
 @end menu
 
 The main caveat with all these web sources is that they probably won't
@@ -17494,6 +17496,62 @@ RSS
 @end lisp
 
 
+@node Atom
+@subsection Atom
+@cindex nnatom
+@cindex Atom
+
+Some web sites provide an Atom Syndication Format feed.  Atom is a web
+feed format similar in function to RDF Site Summary (@pxref{RSS}).
+
+The @code{nnatom} back end allows you to add HTTP or local Atom feeds
+as Gnus servers (with a single group), as you would with any other
+method, by supplying the location of the feed as the server address.
+Note, however, that the server address shouldn't be prefixed with
+@url{http://} or @url{https://}.
+
+Features of @code{nnatom} include:
+
+@itemize @bullet
+
+@item
+Server data is saved per-server in the @file{atom} sub-directory of
+@file{gnus-directory}.
+
+@item
+An article part is generated for both the summary and the content for
+each entry in the feed.  Content of all MIME types should be displayed
+correctly through Gnus (as long as the feed specifies a MIME type),
+even base64 encoded files (like images for example), which are
+supported by Atom.
+
+@item
+Article modification and publish dates are tracked, and articles are
+updated if changed.
+
+@item
+Designed with modularity in mind, the back end theoretically supports
+features which aren't available in Atom (and thus aren't actually
+implemented in practice).  For example, the back end operates under
+the assumption that ``servers'' (Atom feeds) can have multiple groups,
+when in reality the parsing functions will only ever return servers
+with a single group.  In addition, all parsing steps and some of the
+printing is done in functions stored in server variables, to allow
+very easily defining new back ends for different kinds of feeds by
+inheriting from @code{nnatom}; these are called
+@code{nnatom-read-*-function} and
+@code{nnatom-print-content-function}, and their requirements are
+detailed in their docstrings.  A macro is provided to ease creation of
+new inheriting backends.
+
+@defmac nnatom-define-basic-backend-interface backend
+Define server variables expected by @code{nnatom} and import its back
+end functions for @var{backend}.  This macro (or code similar to it) should
+be used by any inheriting back end.
+@end defmac
+
+@end itemize
+
 @node Other Sources
 @section Other Sources
 
diff --git a/etc/NEWS b/etc/NEWS
index 03a40c11c5b..47ecbf3f8e0 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -882,6 +882,13 @@ distracting and easily confused with actual code, or a significant
 early aid that relieves you from moving the buffer or reaching for the
 mouse to consult an error message.
 
+** Gnus
+
++++
+*** New back end 'nnatom'.
+This allow users to add Atom Syndication Format feeds to Gnus as
+servers.
+
 ** JS Mode
 The binding 'M-.' has been removed from the major mode keymaps in
 'js-mode' and 'js-ts-mode', having it default to the global binding
diff --git a/lisp/gnus/gnus.el b/lisp/gnus/gnus.el
index 6bf66233101..de5567d1fc0 100644
--- a/lisp/gnus/gnus.el
+++ b/lisp/gnus/gnus.el
@@ -1348,6 +1348,7 @@ gnus-valid-select-methods
     ("nnimap" post-mail address prompt-address physical-address respool
      server-marks cloud)
     ("nnmaildir" mail respool address server-marks)
+    ("nnatom" address)
     ("nnnil" none))
   "An alist of valid select methods.
 The first element of each list lists should be a string with the name
diff --git a/lisp/gnus/nnatom.el b/lisp/gnus/nnatom.el
new file mode 100644
index 00000000000..e92a6f09baa
--- /dev/null
+++ b/lisp/gnus/nnatom.el
@@ -0,0 +1,277 @@
+;;; nnatom.el --- Atom backend for Gnus -*- lexical-binding: t -*-
+
+;; Copyright (C) 2023 Free Software Foundation, Inc.
+;; Author: Daniel Semyonov <daniel@dsemy.com>
+
+;; This file is part of GNU Emacs.
+
+;; nnatom is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; nnatom is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with nnatom.  If not, see <https://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;; Gnus backend for HTTP or local feeds following the
+;; Atom Syndication Format <https://www.ietf.org/rfc/rfc4287>.
+;; Other types of feeds may be supported by providing custom parsing
+;; functions.
+
+;;; Code:
+
+(eval-when-compile
+  (require 'cl-lib)
+  (require 'subr-x))
+
+(require 'nnfeed)
+(require 'mm-url)
+(require 'dom)
+
+(defgroup nnatom nil
+  "Atom backend for Gnus."
+  :group 'nnfeed)
+
+(nnoo-declare nnatom nnfeed)
+
+(nnfeed-define-basic-backend-interface nnatom)
+
+;;;; Atom feed parser:
+
+(defun nnatom--read-feed (feed _)
+  "Return a list structure representing FEED, or nil."
+  (if (string-match-p "\\`https?://" feed)
+      (nnheader-report
+       nnatom-backend
+       "Address shouldn't start with \"http://\" or \"https://\"")
+    (with-temp-buffer
+      (condition-case e
+          (if (file-name-absolute-p feed)
+              (insert-file-contents feed)
+            (mm-url-insert-file-contents (concat "https://" feed)))
+        (file-error (nnheader-report nnatom-backend (cdr e)))
+        (:success (when-let ((data (if (libxml-available-p)
+                                       (libxml-parse-xml-region
+                                        (point-min) (point-max))
+                                     (car (xml-parse-region
+                                           (point-min) (point-max)))))
+                             (auth (list 'authors)))
+                    (when (eq (car data) 'top)
+                      (setq data (assq 'feed data)))
+                    (dom-add-child-before data auth)
+                    (catch :stop ; Collect feed authors, stop at first entry.
+                      (dolist (child (cdddr data))
+                        (let ((tag (car-safe child)))
+                          (if (eq tag 'entry)
+                              (throw :stop data)
+                            (and (or (eq tag 'author)
+                                     (eq tag 'contributor))
+                                 (dom-add-child-before auth child)))))
+                      data)))))))
+(defvoo nnatom-read-feed-function #'nnatom--read-feed
+  nil nnfeed-read-feed-function)
+
+(defun nnatom--read-group (data)
+  "Return the next group and the remaining DATA in a cons cell, or nil."
+  `(,data))
+(defvoo nnatom-read-group-function #'nnatom--read-group
+  nil nnfeed-read-group-function)
+
+(defun nnatom--read-article (data _)
+  "Return the next article and the remaining DATA in a cons cell, or nil."
+  (when (eq (car data) 'feed) (setq data (cddr data)))
+  (while (and data (not (eq (car-safe (car data)) 'entry))) (pop data))
+  (when-let ((article (car data))
+             (auths (list 'authors)) (links (list 'links)))
+    (dom-add-child-before article links)
+    (dom-add-child-before article auths)
+    (dolist (child (cddddr article) `(,article . ,(cdr data)))
+      (pcase (car-safe child)                ; Authors and links can appear
+        ((or 'author 'contributor)           ; anywhere in the entry so we
+         (dom-add-child-before auths child)  ; collect them all here to
+         (dom-add-child-before links child)) ; avoid looping over the
+        ((or 'link                           ; entry multiple times later.
+             (and 'content (guard (assq 'src (dom-attributes child)))))
+         (dom-add-child-before links child))))))
+(defvoo nnatom-read-article-function #'nnatom--read-article
+  nil nnfeed-read-article-function)
+
+(defun nnatom--read-title (group)
+  "Return the title of GROUP, or nil."
+  (dom-text (dom-child-by-tag group 'title)))
+(defvoo nnatom-read-title-function #'nnatom--read-title
+  nil nnfeed-read-title-function)
+
+(defun nnatom--read-description (group)
+  "Return the description of GROUP, or nil."
+  (dom-text (dom-child-by-tag group 'subtitle)))
+(defvoo nnatom-read-description-function #'nnatom--read-description
+  nil nnfeed-read-description-function)
+
+(defun nnatom--read-article-or-group-authors (article-or-group)
+  "Return the authors of ARTICLE-OR-GROUP, or nil."
+  (when-let
+      ((a (string-trim-right
+           (mapconcat (lambda (author)
+                        (let* ((name
+                                (dom-text (dom-child-by-tag author 'name)))
+                               (name (unless (string-blank-p name) name))
+                               (email
+                                (dom-text (dom-child-by-tag author 'email)))
+                               (email (unless (string-blank-p email) email)))
+                          (concat
+                           (or (and name email (format "%s <%s>" name email))
+                               name email)
+                           ", ")))
+                      (dom-children
+                       (dom-child-by-tag article-or-group 'authors)))
+           ", "))
+       ((not (string-blank-p a))))
+    a))
+(defvoo nnatom-read-author-function #'nnatom--read-article-or-group-authors
+  nil nnfeed-read-author-function)
+(defvoo nnatom-read-group-author-function #'nnatom--read-article-or-group-authors
+  nil nnfeed-read-group-author-function)
+
+(defun nnatom--read-subject (article)
+  "Return the subject of ARTICLE, or nil."
+  (dom-text (dom-child-by-tag article 'title)))
+(defvoo nnatom-read-subject-function #'nnatom--read-subject
+  nil nnfeed-read-subject-function)
+
+(defun nnatom--read-id (article)
+  "Return the ID of ARTICLE.
+If the ARTICLE doesn't contain an ID but it does contain a subject,
+return the subject.  Otherwise, return nil."
+  (or (dom-text (dom-child-by-tag article 'id))
+      (nnatom--read-subject article)))
+(defvoo nnatom-read-id-function #'nnatom--read-id
+  nil nnfeed-read-id-function)
+
+(defun nnatom--read-publish (article)
+  "Return the date and time ARTICLE was published, or nil."
+  (when-let (d (dom-child-by-tag article 'published))
+    (date-to-time (dom-text d))))
+(defvoo nnatom-read-publish-date-function #'nnatom--read-publish
+  nil nnfeed-read-publish-date-function)
+
+(defun nnatom--read-update (article)
+  "Return the date and time of the last update to ARTICLE, or nil."
+  (when-let (d (dom-child-by-tag article 'updated))
+    (date-to-time (dom-text d))))
+(defvoo nnatom-read-update-date-function #'nnatom--read-update
+  nil nnfeed-read-update-date-function)
+
+(defun nnatom--read-links (article)
+  "Return all links contained in ARTICLE, or nil."
+  (let ((alt 0) (rel 0) (sel 0) (enc 0) (via 0) (aut 0))
+    (mapcan
+     (lambda (link)
+       (when-let ((l (car-safe link)))
+         (or
+          (when-let (((eq l 'content))
+                     (src (dom-attr link 'src))
+                     (label (concat "Link"
+                                    (and (< 1 (cl-incf alt))
+                                         (format " %s" alt)))))
+            `(((("text/plain") . ,(format "%s: %s\n" label src))
+               (("text/html") . ,(format "<a href=\"%s\">[%s]</a> "
+                                         src label)))))
+          (when-let (((or (eq l 'author) (eq l 'contributor)))
+                     (name (dom-text (dom-child-by-tag link 'name)))
+                     (name (if (string-blank-p name)
+                               (concat "Author"
+                                       (and (< 1 (cl-incf aut))
+                                            (format " %s" aut)))
+                             name))
+                     (uri (dom-text (dom-child-by-tag link 'uri)))
+                     ((not (string-blank-p uri))))
+            `(((("text/plain") . ,(format "%s: %s\n" name uri))
+               (("text/html") . ,(format "<a href=\"%s\">[%s]</a> "
+                                         uri name)))))
+          (when-let (((eq l 'link))
+                     (attrs (dom-attributes link))
+                     (label (or (cdr (assq 'title attrs))
+                                (pcase (cdr (assq 'rel attrs))
+                                  ("related"
+                                   (concat "Related"
+                                           (and (< 1 (cl-incf rel))
+                                                (format " %s" rel))))
+                                  ("self"
+                                   (concat "More"
+                                           (and (< 1 (cl-incf sel))
+                                                (format " %s" sel))))
+                                  ("enclosure"
+                                   (concat "Enclosure"
+                                           (and (< 1 (cl-incf enc))
+                                                (format " %s" enc))))
+                                  ("via"
+                                   (concat "Source"
+                                           (and (< 1 (cl-incf via))
+                                                (format " %s" via))))
+                                  (_ (if-let
+                                         ((lang (cdr (assq 'hreflang link))))
+                                         (format "Link (%s)" lang)
+                                       (concat
+                                        "Link"
+                                        (and (< 1 (cl-incf alt))
+                                             (format " %s" alt))))))))
+                     (link (cdr (assq 'href attrs))))
+            `(((("text/plain") . ,(format "%s: %s\n" label link))
+               (("text/html") . ,(format "<a href=\"%s\">[%s]</a> "
+                                         link label))))))))
+     (dom-children (dom-child-by-tag article 'links)))))
+(defvoo nnatom-read-links-function #'nnatom--read-links
+  nil nnfeed-read-links-function)
+
+(defun nnatom--read-part (part type)
+  (let* ((atypes '("html" "plain"))
+         (mtypes '(("xhtml" . "text/html") ("text" . "text/plain")))
+         (xsuff (concat "[+/]xml\\(-\\(dtd\\|external-parsed-entity\\)\\)?\\'"
+                        "\\|^text"))
+         (part (if (string= type "xhtml")
+                   (with-temp-buffer
+                     (dom-print (dom-child-by-tag part 'div) nil t)
+                     (buffer-substring-no-properties
+                      (point-min) (point-max)))
+                 (dom-text part)))
+         (type (if (member type atypes) (concat "text/" type) type))
+         (type (or (cdr (assoc type mtypes)) type)))
+    (unless (string-blank-p part)
+      `(,part (Content-Type . ,(or type (setq type "text/plain")))
+              ,(and (not (string-match-p xsuff type))
+                    '(Content-Transfer-Encoding . "base64"))))))
+
+(defun nnatom--read-parts (article)
+  "Return all parts contained in ARTICLE, or an empty HTML part with links."
+  (let* ((summary (dom-child-by-tag article 'summary))
+         (stype (cdr (assq 'type (dom-attributes summary))))
+         (summary (nnatom--read-part summary stype))
+         (content (dom-child-by-tag article 'content))
+         (ctype (cdr (assq 'type (dom-attributes content))))
+         (content (nnatom--read-part content ctype))
+         (st (string= stype ctype))
+         parts)
+    (cond ((and summary content)
+           (and st (push summary parts))
+           (push (append content '(links)) parts)
+           (or st (push summary parts))
+           parts)
+          ((setq parts (or summary content))
+           `(,(append (or summary content) '(links))))
+          (t '((nil (Content-Type . "text/html") links))))))
+(defvoo nnatom-read-parts-function #'nnatom--read-parts
+  nil nnfeed-read-parts-function)
+
+(gnus-declare-backend (symbol-name nnatom-backend) 'address)
+
+(provide 'nnatom)
+
+;;; nnatom.el ends here
diff --git a/lisp/gnus/nnfeed.el b/lisp/gnus/nnfeed.el
new file mode 100644
index 00000000000..021fc6bbb32
--- /dev/null
+++ b/lisp/gnus/nnfeed.el
@@ -0,0 +1,683 @@
+;;; nnfeed.el --- Generic feed backend for Gnus -*- lexical-binding: t -*-
+
+;; Copyright (C) 2023 Free Software Foundation, Inc.
+;; Author: Daniel Semyonov <daniel@dsemy.com>
+
+;; This file is part of GNU Emacs.
+
+;; nnfeed is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; nnfeed is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with nnfeed.  If not, see <https://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;; Generic Gnus backend (intended) for implementing backends for web
+;; feeds (Atom, RSS).
+;;
+;; This backend is abstract - it doesn't implement a parser for any
+;; specific web feed type, and so can't be used independently.
+;;
+;; Instead, it implements a generic parser, feed data storage and most
+;; of the Gnus backend interface; the intended use for this backend is
+;; to be a source of inheritance for backends supporting new web feed
+;; types.
+;;
+;; To implement new backends, use `nnfeed-define-basic-backend-interface':
+;;
+;;    ...
+;;    (require 'nnfeed)
+;;
+;;    (nnoo-declare nnfoo nnfeed)
+;;
+;;    (nnfeed-define-basic-backend-interface nnfoo)
+;;    ...
+;;    [  definitions of parsing functions, see the "Feed parser interface"
+;;       section for more information.  ]
+;;
+;;    (defvoo nnfoo-read-feed-function #'nnfoo--read-feed
+;;     nil nnfeed-read-feed-function)
+;;    ...
+;;    (gnus-declare-backend (symbol-name nnfeed-backend) 'address)
+;;
+;;    (provide 'nnfoo)
+;;
+;;; Code:
+
+(eval-when-compile
+  (require 'subr-x))
+
+(require 'gnus)
+(require 'nnoo)
+
+(defgroup nnfeed nil
+  "Generic feed backend for Gnus."
+  :group 'gnus)
+
+(nnoo-declare nnfeed)
+
+(defvoo nnfeed-backend nil
+  "Symbol which identifies this backend.")
+
+(defvoo nnfeed-status-string nil
+  "Last status message reported by this backend.")
+
+;; This macro should be used to define inheriting backends.
+
+(defmacro nnfeed-define-basic-backend-interface (backend)
+  "Define a basic set of functions and variables for BACKEND."
+  `(progn
+     (defvoo ,(nnoo-symbol backend 'backend) ',backend nil nnfeed-backend)
+     (defvoo ,(nnoo-symbol backend 'status-string)
+         nil nil nnfeed-status-string)
+     (defvoo ,(nnoo-symbol backend 'group) nil nil nnfeed-group)
+     (defvoo ,(nnoo-symbol backend 'servers)
+         (make-hash-table :test 'equal) nil nnfeed-servers)
+     (defvoo ,(nnoo-symbol backend 'group-article-ids)
+         (make-hash-table :test 'equal) nil nnfeed-group-article-ids)
+     (defvoo ,(nnoo-symbol backend 'group-articles)
+         (make-hash-table :test 'eql) nil nnfeed-group-articles)
+     (defvoo ,(nnoo-symbol backend 'group-article-max-num) 0 nil
+             nnfeed-group-article-max-num)
+     (defvoo ,(nnoo-symbol backend 'group-article-mix-num) 1 nil
+             nnfeed-group-article-min-num)
+     ,@(mapcar (lambda (fun)
+                 `(deffoo ,(nnoo-symbol backend fun) (&optional server)
+	            (,(nnoo-symbol 'nnoo fun) ',backend server)))
+	       '(server-opened status-message))
+     (deffoo ,(nnoo-symbol backend 'open-server) (server &optional defs)
+       (nnfeed-open-server server defs ',backend))
+     (nnoo-import ,backend (nnfeed))))
+
+;;;; Feed parser interface:
+
+;; The following set of server variables define a parser for a
+;; specific web feed type.
+;; An inheriting backend should define all of these functions (ideally).
+;; Apart from this set of variables there is also
+;; `nnfeed-print-content-function' which can be optionally be defined
+;; by an inheriting backend to allow more advanced control over the
+;; printing of articles.
+
+(defvoo nnfeed-read-feed-function #'ignore
+  "Function returning a Lisp object representing a feed (or part of it).
+
+It should accept two arguments, the address of a feed and the name of
+a group (or nil).
+If a group name is supplied, it should return a representation of only
+the group (as if it was extracted from the feed with
+`nnfeed-read-group-function').")
+
+(defvoo nnfeed-read-group-function #'ignore
+  "Function returning a cons cell of a group and remaining data from a feed.
+
+The returned group can be represented by any Lisp object.
+It should accept a single argument, a Lisp object representing a feed
+\(as can be returned by this function or `nnfeed-read-feed-function').")
+
+(defvoo nnfeed-read-article-function #'ignore
+  "Function returning a cons cell of an article and remaining data from a group.
+
+The returned article can be represented by any Lisp object.
+It should accept two arguments, a Lisp object representing a group
+\(as can be returned by this function or `nnfeed-read-group-function'),
+and a flag indicating whether the last article was not new or updated.")
+
+(defvoo nnfeed-read-title-function #'ignore
+  "Function returning the title of a group (a string).
+
+It should accept a single argument, a Lisp object representing a group
+\(as returned by `nnfeed-read-group-function').")
+
+(defvoo nnfeed-read-description-function #'ignore
+  "Function returning the description of a group (a string), or nil.
+
+It should accept a single argument, a Lisp object representing a group
+\(as returned by `nnfeed-read-group-function').")
+
+(defvoo nnfeed-read-group-author-function #'ignore
+  "Function returning the author of a group (a string), or nil.
+
+It should accept a single argument, a Lisp object representing a group
+\(as returned by `nnfeed-read-group-function')..")
+
+(defvoo nnfeed-read-id-function #'ignore
+  "Function returning the ID of an article.
+
+It should accept a single argument, a Lisp object representing an article
+\(as returned by `nnfeed-read-article-function').")
+
+(defvoo nnfeed-read-subject-function #'ignore
+  "Function returning the subject of an article (a string), or nil.
+
+It should accept a single argument, a Lisp object representing an article
+\(as returned by `nnfeed-read-article-function').")
+
+(defvoo nnfeed-read-publish-date-function #'ignore
+  "Function returning the publish date of an article (a time value), or nil.
+
+It should accept a single argument, a Lisp object representing an article
+\(as returned by `nnfeed-read-article-function').")
+
+(defvoo nnfeed-read-update-date-function #'ignore
+  "Function returning the update date of an article (a time value), or nil.
+
+It should accept a single argument, a Lisp object representing an article
+\(as returned by `nnfeed-read-article-function').")
+
+(defvoo nnfeed-read-author-function #'ignore
+  "Function returning the author of an article (a string), or nil.
+
+It should accept a single argument, a Lisp object representing an article
+\(as returned by `nnfeed-read-article-function').")
+
+(defvoo nnfeed-read-headers-function #'ignore
+  "Function returning an alist of article-wide MIME headers.
+
+Each element of this alist maps a MIME header (a symbol,
+i.e. `Content-Type') to its value.  As a special case, `:boundary'
+maps to a string which will serve as the boundary between article
+parts.  This must be supplied if a custom boundary is used in a
+multipart content type header.  The default boundary is \"-_nnfeed_-\",
+and is automatically modified to match the name of the back end.
+It should accept a single argument, a Lisp object representing an article
+\(as returned by `nnfeed-read-article-function').")
+
+;; As mentioned in their docstrings, the last two parsing functions
+;; can optionally return any Lisp representation they want, provided
+;; an appropriate `nnfeed-print-content-function' is defined.
+
+(defvoo nnfeed-read-links-function #'ignore
+  "Function returning all links contained in an article.
+
+With the default `nnfeed-print-content-function', it should return a
+list of links, where each link is an alist mapping MIME content types
+to links formatted for display in a part of that type.  Each content
+type may also be a list of content types.
+Otherwise, it could return any Lisp object.
+It should accept a single argument, a Lisp object representing an article
+\(as returned by `nnfeed-read-article-function').")
+
+(defvoo nnfeed-read-parts-function #'ignore
+  "Function returning an alist associating parts of an article to their headers.
+
+With the default `nnfeed-print-content-function', each part should be a
+string.  Otherwise, it can be any Lisp object.  The \"headers\" of
+each part should be a list where each element is either a cons of a
+MIME header (a symbol, i.e. `Content-Type') and its value (a string),
+or any other Lisp object.  MIME headers will be printed, the rest will
+be passed on to `nnfeed-print-content-function', which recognizes the
+following extra data by default:
+- `links', if present, will cause links to be printed in the part.
+It should accept a single argument, a Lisp object representing an article
+\(as returned by `nnfeed-read-article-function').")
+
+;;;; Feed data storage:
+
+;; This section defines the data types used to store feed data, and
+;; All variables in this section are automatically defined by
+;; `nnfeed-define-basic-backend-interface'.
+
+(defvoo nnfeed-servers (make-hash-table :test 'equal)
+  "Hash table mapping known servers to their groups.
+
+Each value in this table should itself be a hash table mapping known
+group names to their data, which should be a vector of the form
+[GROUP IDS ARTICLES MAX MIN DESCRIPTION], where:
+- GROUP is the \"real\" group name (the name known to the server).
+- IDS is a hash table mapping article IDs to their numbers.
+- ARTICLES is a hash table mapping article numbers to articles and
+  their attributes (see `nnfeed-group-articles').
+- MAX is the maximum article number.
+- MIN is the minimum article number.
+- DESCRIPTION is the group description.")
+
+(defvoo nnfeed-group-names (make-hash-table :test 'equal)
+  "Hash table mapping real group names to their custom name.")
+
+(defun nnfeed--server-file (server)
+  "Return the file containing data for SERVER."
+  (expand-file-name (format "%s/%s.eld"
+                            (string-trim (symbol-name nnfeed-backend)
+                                         "nn")
+                            (gnus-newsgroup-savable-name server))
+                    gnus-directory))
+
+(defun nnfeed--read-server (server)
+  "Read SERVER's information storage."
+  (if-let ((f (nnfeed--server-file server))
+           ((file-readable-p f)))
+      (with-temp-buffer
+        (insert-file-contents f)
+        (goto-char (point-min))
+        (puthash server (read (current-buffer)) nnfeed-servers))
+    (nnheader-report nnfeed-backend "Can't read %s" server)))
+
+(defun nnfeed--write-server (server)
+  "Write SERVER's information to storage."
+  (if-let ((f (nnfeed--server-file server))
+           ((file-writable-p f)))
+      (if-let ((s (gethash server nnfeed-servers))
+               ((hash-table-p s)))
+          (with-temp-file f
+            (insert ";;;; -*- mode: lisp-data -*- DO NOT EDIT\n")
+            (prin1 s (current-buffer))
+            (insert "\n")
+            t)
+        t)
+    (nnheader-report nnfeed-backend "Can't write %s" f)))
+
+(defun nnfeed--server-address (server)
+  "Return SERVER's real address."
+  (if (string-suffix-p "-ephemeral" server)
+      (setq server (or (cadr (assq (nnoo-symbol nnfeed-backend 'address)
+                                   (cddr (gnus-server-to-method
+                                          (concat
+                                           (symbol-name nnfeed-backend) ":"
+                                           server)))))
+                       server))
+    server))
+
+;; This is the function which actually uses the functions defined in
+;; the last section.  None of those functions are called after parsing
+;; a feed.
+
+(defun nnfeed--parse-feed (feed &optional group)
+  "Parse FEED into a new or existing server.
+Optionally, only parse GROUP."
+  (let* ((feed (nnfeed--server-address feed))
+         (s (or (gethash feed nnfeed-servers) (nnfeed--read-server feed)))
+         (g (or (and (hash-table-p s) (gethash group s)) `[,group]))
+         (name group) ; (Maybe) fake name
+         (group (or (aref g 0) name)) ; Real name
+         data)
+    (when (setq data (funcall nnfeed-read-feed-function feed group)
+                s (or (gethash feed nnfeed-servers)
+                      (make-hash-table :test 'equal)))
+      (while-let ((cg (or (and name `(,data))
+                          (funcall nnfeed-read-group-function data)))
+                  (cg (prog1 (car cg) (setq data (cdr cg)))))
+        (let* ((name (funcall nnfeed-read-title-function cg)) ; Real name
+               (group (gethash name nnfeed-group-names name)) ; (Maybe) fake name
+               (info (gnus-get-info
+                      (concat (symbol-name nnfeed-backend) "+" feed ":" group)))
+               (g (or (gethash group s)
+                      `[ ,name ,(make-hash-table :test 'equal)
+                         ,(make-hash-table :test 'eql) nil 1 ""]))
+               (desc (funcall nnfeed-read-description-function cg))
+               (ids (aref g 1))
+               (articles (aref g 2))
+               (max (aref g 3))
+               (max (if max max
+                      (setq max 0) ; Find max article number
+                      (dolist      ; remembered by Gnus.
+                          ( r (cons (gnus-info-read info)
+                                    (gnus-info-marks info))
+                            max)
+                        (mapc (lambda (x)
+                                (let ((x (if (consp x)
+                                             (if (< (car x) (cdr x))
+                                                 (cdr x) (car x))
+                                           x)))
+                                  (when (< max x) (setq max x))))
+                              (if (symbolp (car r)) (cdr r) r)))))
+               (group-author (funcall nnfeed-read-group-author-function cg))
+               stale)
+          (and desc (aset g 5 desc))
+          (while-let ((article (funcall nnfeed-read-article-function cg stale))
+                      (article (prog1 (car article) (setq cg (cdr article)))))
+            (when-let ((id (funcall nnfeed-read-id-function article))
+                       (id (format "<%s@%s.%s>" id name nnfeed-backend)))
+              (let* ((num (gethash id ids))
+                     (update (funcall nnfeed-read-update-date-function article))
+                     (prev-update (aref (gethash num articles
+                                                 '[nil nil nil nil nil])
+                                        4)))
+                (if (or (null num) ; New article ID.
+                        (and (null prev-update) update)
+                        (and prev-update update
+                             (time-less-p prev-update update)))
+                    (let* ((num (or num (aset g 3 (setq max (1+ max)))))
+                           (publish (funcall nnfeed-read-publish-date-function
+                                             article)))
+                      (setf
+                       (gethash id (aref g 1)) num
+                       (gethash num (aref g 2))
+                       `[ ,id
+                          ,(or (funcall nnfeed-read-author-function article)
+                               group-author group)
+                          ,(or (funcall nnfeed-read-subject-function article)
+                               "no subject")
+                          ,(or publish update '(0 0)) ; published
+                          ,(or update publish '(0 0)) ; updated
+                          ,(funcall nnfeed-read-links-function article)
+                          ,(funcall nnfeed-read-parts-function article)
+                          ,(funcall nnfeed-read-headers-function article)]
+                       stale nil))
+                  (setq stale t)))))
+          (puthash group g s)))
+      (puthash feed s nnfeed-servers))))
+
+;;;; Gnus backend functions:
+
+;; The following two sections define a Gnus backend interface based on
+;; the parsed data from the last section.
+;; All server variables in this section are automatically defined by
+;; `nnfeed-define-basic-backend-interface'.
+;; For more information about these functions see the "Back End
+;; Interface" section of the Gnus manual.
+
+(defvoo nnfeed-group nil
+  "Name of the current group.")
+
+(defvoo nnfeed-group-article-ids (make-hash-table :test 'equal)
+  "Hash table mapping article IDs to their article number.")
+
+(defvoo nnfeed-group-articles (make-hash-table :test 'eql)
+  "Hash table mapping article numbers to articles and their attributes.
+
+Each value in this table should be a vector of the form
+[ID FROM SUBJECT DATE UPDATED LINKS PARTS HEADERS], where:
+- ID is the ID of the article.
+- FROM is the author of the article or group.
+- SUBJECT is the subject of the article.
+- DATE is the date the article was published, or last updated (time value).
+- UPDATE is the date the article was last updated, or published (time value).
+- LINKS is a collection of links (any Lisp object).
+- PARTS is an alist associating the content of each part of the
+  article to its headers.
+- HEADERS is an alist associating article-wide MIME headers to their value.")
+
+(defvoo nnfeed-group-article-max-num 0
+  "Maximum article number for the current group.")
+
+(defvoo nnfeed-group-article-min-num 1
+  "Minimum article number for the current group.")
+
+(defvar nnfeed-date-format "%F %X"
+  "Format of displayed dates (see `format-time-string').")
+
+(nnoo-define-basics nnfeed)
+
+(defun nnfeed--current-server-no-prefix ()
+  "Remove the \"<backend>+\" prefix from the current server."
+  (string-remove-prefix (concat (symbol-name nnfeed-backend) "+")
+                        (nnoo-current-server nnfeed-backend)))
+
+(defun nnfeed--group-data (group &optional server)
+  "Get parsed data for GROUP from SERVER (or the current server if omitted)."
+  (let ((s (gethash server nnfeed-servers)) c)
+    (or (and (hash-table-p s) (gethash group s))
+        (and (setq c (nnfeed--current-server-no-prefix))
+             (setq s (gethash c nnfeed-servers))
+             (hash-table-p s) (gethash group s))
+        (catch :stop (maphash (lambda (n s)
+                                (or (string= n server)
+                                    (string= n c)
+                                    (when-let (((hash-table-p s))
+                                               (g (gethash group s)))
+                                      (throw :stop g))))
+                              nnfeed-servers)))))
+
+(defun nnfeed-retrieve-article (article group)
+  "Retrieve headers for ARTICLE from GROUP."
+  (if-let ((a (gethash article (aref group 2))))
+      (insert (format "221 %s Article retrieved.
+From: %s\nSubject: %s\nDate: %s\nMessage-ID: %s\n.\n"
+                      article
+                      (aref a 1)
+                      (aref a 2)
+                      (format-time-string nnfeed-date-format (aref a 3))
+                      (aref a 0)))
+    (insert "404 Article not found.\n.\n")))
+
+(deffoo nnfeed-retrieve-headers (articles &optional group server _fetch-old)
+  (if-let ((server (or server (nnfeed--current-server-no-prefix)))
+           (g (or (nnfeed--group-data group server)
+                  `[ nil ,nnfeed-group-article-ids ,nnfeed-group-articles
+                     nil nil nil])))
+      (with-current-buffer nntp-server-buffer
+        (erase-buffer)
+        (or (and (stringp (car articles))
+                 (mapc (lambda (a)
+                         (nnfeed-retrieve-article
+                          (gethash a (aref g 2)) g))
+                       articles))
+            (and (numberp (car articles))
+                 (range-map (lambda (a) (nnfeed-retrieve-article a g))
+                            articles)))
+        'headers)
+    (nnheader-report nnfeed-backend "Group %s not found" (or group ""))))
+
+(deffoo nnfeed-open-server (server &optional defs backend)
+  (let ((backend (or backend 'nnfeed))
+        (a (nnfeed--server-address server))
+        s)
+    (nnoo-change-server backend server defs)
+    (when (setq s (or (gethash server nnfeed-servers)
+                      (nnfeed--read-server a)))
+      (maphash (lambda (group g)
+                 (setq g (aref g 0))
+                 (unless (string= group g)
+                   (puthash group g nnfeed-group-names)))
+               s))
+    (or s (file-writable-p (nnfeed--server-file a))
+        (and (nnoo-close-server nnfeed-backend server)
+             (nnheader-report
+              nnfeed-backend "Server file %s not readable or writable"
+              server)))))
+
+(deffoo nnfeed-request-close ()
+  (maphash (lambda (server _)
+             (nnfeed--write-server
+              (nnfeed--server-address server)))
+           nnfeed-servers)
+  (setq nnfeed-servers (make-hash-table :test 'equal)
+        nnfeed-status-string nil)
+  t)
+
+;; The default content printing function, which should be suitable for
+;; most inheriting backends.
+
+(defun nnfeed--print-content (content attributes links)
+  "Return CONTENT formatted according to ATTRIBUTES, with LINKS possibly added."
+  (let ((links (and (memq 'links attributes) links)))
+    (when (or content links)
+      (concat
+       (and content (format "%s\n\n" content))
+       (mapconcat (lambda (link)
+                    (cdr (assoc (cdr (assq 'Content-Type attributes)) link
+                                (lambda (types type)
+                                  (if (stringp types) (string= types type)
+                                    (member type types))))))
+                  links)))))
+
+(defvoo nnfeed-print-content-function #'nnfeed--print-content
+  "Function returning a single piece of content for an article (a string).
+
+It should accept three arguments, a part and its attributes (as returned
+by `nnfeed-read-parts-function'), and links (as returned by
+`nnfeed-read-links-function').")
+
+(defun nnfeed--print-part (content headers mime links)
+  "Print part of an article using its CONTENT, HEADERS, and LINKS.
+Only HEADERS of a type included in MIME are considered."
+  (concat
+   (mapconcat (lambda (header)
+                (when-let ((m (car-safe header))
+                           ((member m mime)))
+                  (format "%s: %s\n" m (cdr header))))
+              headers)
+   "\n"
+   (funcall nnfeed-print-content-function content headers links)))
+
+(deffoo nnfeed-request-article (article &optional group server to-buffer)
+  (if-let ((server (or server (nnfeed--current-server-no-prefix)))
+           (g (or (nnfeed--group-data group server)
+                  (and (setq group nnfeed-group)
+                       `[ nil ,nnfeed-group-article-ids
+                          ,nnfeed-group-articles
+                          ,nnfeed-group-article-max-num
+                          ,nnfeed-group-article-min-num nil])))
+           (num (or (and (stringp article)
+                         (gethash article (aref g 1)))
+                    (and (numberp article) article)))
+           ((and (<= num (aref g 3))
+                 (>= num (aref g 4))))
+           (a (gethash num (aref g 2))))
+      (with-current-buffer (or to-buffer nntp-server-buffer)
+        (erase-buffer)
+        (let* ((links (aref a 5))
+               (parts (aref a 6))
+               (headers (aref a 7))
+               (boundary (or (cdr (assq :boundary headers))
+                             (format "-_%s_-" nnfeed-backend)))
+               (multi (length> parts 1))
+               (mime '( Content-Type Content-Disposition
+                        Content-Transfer-Encoding)))
+          (insert (format
+                   "Subject: %s\nFrom: %s\nDate: %s\nMessage-ID: %s\n"
+                   (aref a 2) (aref a 1)
+                   (format-time-string
+                    nnfeed-date-format (or (aref a 3) '(0 0)))
+                   (aref a 0))
+                  (if (assq 'MIME-Version headers) "" "MIME-Version: 1.0\n")
+                  (mapconcat (lambda (header)
+                               (unless (keywordp (car header))
+                                 (format "%s: %s\n" (car header) (cdr header))))
+                             headers)
+                  (if multi
+                      (if (assq 'Content-Type headers) ""
+                        (format
+                         "Content-Type: multipart/alternative; boundary=%s\n"
+                         boundary))
+                    (prog1 (nnfeed--print-part
+                            (caar parts) (cdar parts) mime links)
+                      (setq parts nil)))
+                  (mapconcat (lambda (part)
+                               (format "--%s\n%s\n" boundary
+                                       (nnfeed--print-part
+                                        (car part) (cdr part) mime links)))
+                             parts)
+                  (if multi (format "--%s--" boundary) "\n")))
+        `(,group . ,num))
+    (nnheader-report nnfeed-backend "No such article")))
+
+(deffoo nnfeed-request-group (group &optional server fast _info)
+  (with-current-buffer nntp-server-buffer
+    (erase-buffer)
+    (if-let ((server (or server (nnfeed--current-server-no-prefix)))
+             (g (or (if fast (nnfeed--group-data group server)
+                      (setq server (nnfeed--parse-feed server group))
+                      (and (hash-table-p server) (gethash group server)))
+                    `[ ,group ,(make-hash-table :test 'equal)
+                       ,(make-hash-table :test 'eql) 0 1 ""])))
+        (progn
+          (setq nnfeed-group group
+                nnfeed-group-article-ids (aref g 1)
+                nnfeed-group-articles (aref g 2)
+                nnfeed-group-article-max-num (aref g 3)
+                nnfeed-group-article-min-num (aref g 4))
+          (insert (format "211 %s %s %s \"%s\""
+                          (hash-table-count nnfeed-group-article-ids)
+                          nnfeed-group-article-min-num
+                          nnfeed-group-article-max-num group))
+          t)
+      (insert "404 group not found")
+      (nnheader-report nnfeed-backend "Group %s not found" group))))
+
+(deffoo nnfeed-close-group (group &optional server)
+  (and (string= group nnfeed-group)
+       (setq nnfeed-group nil
+             nnfeed-group-article-ids (make-hash-table :test 'equal)
+             nnfeed-group-articles (make-hash-table :test 'eql)
+             nnfeed-group-article-max-num 0
+             nnfeed-group-article-min-num 1))
+  (setq server (or server (nnfeed--current-server-no-prefix)))
+  (nnfeed--write-server server))
+
+(deffoo nnfeed-request-list (&optional server)
+  (with-current-buffer nntp-server-buffer
+    (erase-buffer)
+    (when-let ((p (point))
+               (s (nnfeed--parse-feed
+                   (or server (nnfeed--current-server-no-prefix))))
+               ((hash-table-p s)))
+      (maphash (lambda (group g)
+                 (insert (format "\"%s\" %s %s y\n"
+                                 group (aref g 3) (aref g 4))))
+               s)
+      (not (= (point) p)))))
+
+(deffoo nnfeed-request-post (&optional _server)
+  (nnheader-report nnfeed-backend "%s is a read only backend" nnfeed-backend))
+
+;;;; Optional back end functions:
+
+(deffoo nnfeed-retrieve-groups (_groups &optional server)
+  (nnfeed-request-list (or server (nnfeed--current-server-no-prefix)))
+  'active)
+
+(deffoo nnfeed-request-type (_group &optional _article)
+  'unknown)
+
+(deffoo nnfeed-request-group-description (group &optional server)
+  (when-let ((server (or server (nnfeed--current-server-no-prefix)))
+             (g (nnfeed--group-data group server)))
+    (with-current-buffer nntp-server-buffer
+      (erase-buffer)
+      (insert group "	" (aref g 5) "\n"))))
+
+(deffoo nnfeed-request-list-newsgroups (&optional server)
+  (when-let ((server (or server (nnfeed--current-server-no-prefix)))
+             (s (gethash server nnfeed-servers))
+             ((hash-table-p s)))
+    (with-current-buffer nntp-server-buffer
+      (erase-buffer)
+      (maphash (lambda (group g)
+                 (insert group "	" (aref g 5) "\n"))
+               s))))
+
+(deffoo nnfeed-request-rename-group (group new-name &optional server)
+  (when-let ((server (or server (nnfeed--current-server-no-prefix)))
+             (s (or (gethash server nnfeed-servers)
+                    (and ; Open the server to add it to `nnfeed-servers'
+                     (save-match-data
+                       (nnfeed-open-server
+                        server
+                        (cdr ; Get defs and backend.
+                         (assoc server (cdr (assq 'nnfeed nnoo-state-alist))
+                                (lambda (car key)
+                                  (and (stringp car)
+                                       (string-match
+                                        (concat
+                                         "\\(\\(nn[[:alpha:]]+\\)\\+\\)?"
+                                         "\\(" (regexp-quote key) "\\)")
+                                        car)
+                                       (setq server car)))))
+                        (prog1 (if (stringp (match-string 1 server))
+                                   (intern (match-string 2 server))
+                                 'nnfeed)
+                          (setq server (match-string 3 server))))
+                       (gethash server nnfeed-servers)))))
+             (g (or (nnfeed--group-data group server)
+                    `[ ,group ,(make-hash-table :test 'equal)
+                       ,(make-hash-table :test 'eql) nil 1 ""])))
+    (puthash new-name g s)
+    (puthash group new-name nnfeed-group-names)
+    (remhash group s)
+    (and (string= group nnfeed-group)
+         (setq nnfeed-group new-name))
+    t))
+
+(provide 'nnfeed)
+
+;;; nnfeed.el ends here
-- 
2.43.0