From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Subject: bug#28159: Updater needs to support HTTP(S) servers Date: Wed, 23 Aug 2017 23:30:35 +0200 Message-ID: <87ziaqj7k4.fsf@gnu.org> References: <2c2838f3-24d6-5010-faf6-49e70f85e963@crazy-compilers.com> <87poboasjz.fsf@gnu.org> <570534f1-58d2-6db5-b5c2-b9e5276c5974@crazy-compilers.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:54234) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dkdEt-0002hK-6V for bug-guix@gnu.org; Wed, 23 Aug 2017 17:31:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dkdEs-0004K0-6U for bug-guix@gnu.org; Wed, 23 Aug 2017 17:31:03 -0400 Received: from debbugs.gnu.org ([208.118.235.43]:43120) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dkdEs-0004Jt-2u for bug-guix@gnu.org; Wed, 23 Aug 2017 17:31:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1dkdEr-0005eu-SP for bug-guix@gnu.org; Wed, 23 Aug 2017 17:31:01 -0400 Sender: "Debbugs-submit" Resent-Message-ID: In-Reply-To: <570534f1-58d2-6db5-b5c2-b9e5276c5974@crazy-compilers.com> (Hartmut Goebel's message of "Wed, 23 Aug 2017 12:20:02 +0200") List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: "bug-Guix" To: Hartmut Goebel Cc: 28159@debbugs.gnu.org Hartmut Goebel skribis: > Am 22.08.2017 um 10:57 schrieb Ludovic Court=C3=A8s: >> So I would suggest picking one updater, say kde, and implementing it >> using whatever metadata can be retrieved from kde.org. > > I'm not sure if I understood what you mean with "whatever metadata can > be retrieved from kde.org". I mean using package metadata provided by kde.org (maybe they have a JSON representation of the package graph or something?), or the =E2=80=98ls= -lR=E2=80=99 files at worst. > By change, download.kde.org indeed provides a "ls-lR" and "ls-lR.bz" > file at the top-level. I was not aware of this up to just now. Using > this might be an option (It is lagging a bit, though I think this is > acceptable. From what I've ssen I guess it is generated each hour if > some file changed.) Sounds good. > So for kde we might find a simpler solution. But in the long-run IMHO we > need a simple html parser. In some cases yes, but maybe not in all cases. I also suspect that something that attempts to extract the latest release number from a home page may be brittle. > I'm not skilled enough in scheme/guile to write such a parser, sorry. This can be done along these lines: --8<---------------cut here---------------start------------->8--- scheme@(guile-user)> ,use(sxml simple) scheme@(guile-user)> ,use(web client) scheme@(guile-user)> ,use(sxml match) scheme@(guile-user)> (define page (xml->sxml (call-with-values=20 (lambda () (http-get "http://www.gnu.org/software/guix/guix.html" #:streaming= ? #t)) (lambda (response port) port)))) --8<---------------cut here---------------end--------------->8--- =E2=80=A6 where =E2=80=98page=E2=80=99 is the SXML representation of the we= b page. The difficulty is to browse this page (using =E2=80=98match=E2=80=99 or =E2=80= =98sxml-match=E2=80=99.) HTH, Ludo=E2=80=99.