From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Tim Landscheidt Newsgroups: gmane.emacs.bugs Subject: bug#45477: 27.1; RFE: Make full RSS fragments available for nnrss servers Date: Wed, 30 Dec 2020 08:22:15 +0000 Organization: http://www.tim-landscheidt.de/ Message-ID: <87czyrn1oo.fsf@passepartout.tim-landscheidt.de> References: <874kk7os1v.fsf@passepartout.tim-landscheidt.de> <87lfdi6bhi.fsf@gnus.org> <87wnx0mu9s.fsf@passepartout.tim-landscheidt.de> <87wnx0q9ls.fsf@gnus.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="39226"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) Cc: 45477@debbugs.gnu.org To: Lars Ingebrigtsen Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Wed Dec 30 09:23:12 2020 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kuWlC-000A47-NT for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 30 Dec 2020 09:23:10 +0100 Original-Received: from localhost ([::1]:40316 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kuWlB-0005XS-0j for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 30 Dec 2020 03:23:09 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:44358) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kuWl3-0005XI-Vc for bug-gnu-emacs@gnu.org; Wed, 30 Dec 2020 03:23:01 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:57505) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kuWl3-000339-Om for bug-gnu-emacs@gnu.org; Wed, 30 Dec 2020 03:23:01 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kuWl3-0004Yg-K1; Wed, 30 Dec 2020 03:23:01 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Tim Landscheidt Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org, bugs@gnus.org Resent-Date: Wed, 30 Dec 2020 08:23:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 45477 X-GNU-PR-Package: emacs,gnus Original-Received: via spool by 45477-submit@debbugs.gnu.org id=B45477.160931654317470 (code B ref 45477); Wed, 30 Dec 2020 08:23:01 +0000 Original-Received: (at 45477) by debbugs.gnu.org; 30 Dec 2020 08:22:23 +0000 Original-Received: from localhost ([127.0.0.1]:40818 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kuWkR-0004Xh-7V for submit@debbugs.gnu.org; Wed, 30 Dec 2020 03:22:23 -0500 Original-Received: from andalucia.tim-landscheidt.de ([116.203.78.250]:34556) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kuWkM-0004XW-4R for 45477@debbugs.gnu.org; Wed, 30 Dec 2020 03:22:22 -0500 Original-Received: from dslb-090-186-126-124.090.186.pools.vodafone-ip.de ([90.186.126.124]:42182 helo=passepartout.tim-landscheidt.de) by andalucia.tim-landscheidt.de with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1kuWkK-0000WC-D4; Wed, 30 Dec 2020 09:22:16 +0100 In-Reply-To: <87wnx0q9ls.fsf@gnus.org> (Lars Ingebrigtsen's message of "Wed, 30 Dec 2020 04:02:55 +0100") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:197010 Archived-At: Lars Ingebrigtsen wrote: >> Therefore, I need the data, and I need it in a format that >> can be processed further, and there will be a need for a >> custom user function to process the data because each use >> case will be different. > Sure, that sounds reasonable. However: >> (If there was a major overhaul of nnrss, it could be inter- >> esting to forego the intermediate nnrss-group-data saved in >> ~/News/rss/* and either store the feeds as pure XML files, >> re-parsed on demand and available for further processing, or >> write out all the articles as mbox files after parsing the >> feeds, with the entries' fragments as MIME parts.) > I've not used nnrss myself, but reading the code, it seems like it's > storing all the data needed for Gnus to read an nnrss group in > `nnrss-group-data', so storing all the XML data in case somebody is > going to use it would require orders of magnitude more storage? In my practice (so far), not even one magnitude. Random on- disk sample: | -rw-r--r--. 1 root root 132680 Dec 30 06:56 Conan O=E2=80=99Brien Need= s A Friend.el | -rw-r--r--. 1 root root 471799 Dec 30 07:28 Conan O=E2=80=99Brien Need= s A Friend.xml | -rw-r--r--. 1 root root 72249 Dec 30 06:56 Doug Loves Movies.el | -rw-r--r--. 1 root root 245312 Dec 30 07:01 Doug Loves Movies.xml | -rw-r--r--. 1 root root 630495 Dec 30 06:56 ID10T with Chris Hardwick.= el | -rw-r--r--. 1 root root 2100500 Dec 27 21:30 ID10T with Chris Hardwick.= xml | -rw-r--r--. 1 root root 21754 Dec 30 06:56 Sprechen wir =C3=BCber Mor= d?! Der SWR2 True Crime Podcast.el | -rw-r--r--. 1 root root 47741 Dec 30 06:36 Sprechen wir =C3=BCber Mor= d?! Der SWR2 True Crime Podcast.xml | -rw-r--r--. 1 root root 93927 Dec 30 06:56 Stone Clearing With Richar= d Herring.el | -rw-r--r--. 1 root root 221040 Dec 30 07:25 Stone Clearing With Richar= d Herring.xml | -rw-r--r--. 1 root root 17002 Dec 30 06:56 Taskmaster The Podcast.el | -rw-r--r--. 1 root root 53080 Dec 30 07:28 Taskmaster The Podcast.xml | -rw-r--r--. 1 root root 265970 Dec 30 06:56 You Made It Weird with Pet= e Holmes.el | -rw-r--r--. 1 root root 650710 Dec 30 04:07 You Made It Weird with Pet= e Holmes.xml Even if the XML gets bloated when saved in nnrss-group-data (it holds one feed at most), IMHO almost all feeds will be small enough to be negligible in a typical Emacs/Gnus setup (the largest feed above holds data from February 2010 till now; usually feeds only contain the most recent x entries). > I think a way to implement this would be to add an nnrss variable that > says what "extra" XML fields to store -- like (nnrss-extra-fields > '(itunes:episodeType ...)). That would allow my use case. (In a major overhaul, another way to approach this could be a hook/function variable (con- figurable per group) that gets called in addition/in lieu of nnrss-request-article with the raw XML data and then has free rein to format the Gnus article as it wishes to.)