bug#28159: Updater needs to support HTTP(S) servers

[ludo@gnu.org (Ludovic Courtès)]

Hartmut Goebel <h.goebel@crazy-compilers.com> skribis:

> Am 22.08.2017 um 10:57 schrieb Ludovic Courtès:
>> So I would suggest picking one updater, say kde, and implementing it
>> using whatever metadata can be retrieved from kde.org.
>
> I'm not sure if I understood what you mean with "whatever metadata can
> be retrieved from kde.org".

I mean using package metadata provided by kde.org (maybe they have a
JSON representation of the package graph or something?), or the ‘ls-lR’
files at worst.

> By change, download.kde.org indeed provides a "ls-lR" and "ls-lR.bz"
> file at the top-level. I was not aware of this up to just now. Using
> this might be an option (It is lagging a bit, though I think this is
> acceptable. From what I've ssen I guess it is generated each hour if
> some file changed.)

Sounds good.

> So for kde we might find a simpler solution. But in the long-run IMHO we
> need a simple html parser.

In some cases yes, but maybe not in all cases.  I also suspect that
something that attempts to extract the latest release number from a home
page may be brittle.

> I'm not skilled enough in scheme/guile to write such a parser, sorry.

This can be done along these lines:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,use(sxml simple)
scheme@(guile-user)> ,use(web client)
scheme@(guile-user)> ,use(sxml match)
scheme@(guile-user)> (define page (xml->sxml (call-with-values 
						 (lambda ()
						   (http-get "http://www.gnu.org/software/guix/guix.html" #:streaming? #t))
					       (lambda (response port)
						 port))))
--8<---------------cut here---------------end--------------->8---

… where ‘page’ is the SXML representation of the web page.  The
difficulty is to browse this page (using ‘match’ or ‘sxml-match’.)

HTH,
Ludo’.