From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id ANuhKLdcO2HgdQEAgWs5BA (envelope-from ) for ; Fri, 10 Sep 2021 15:25:11 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id oBo1JLdcO2GnVgAAbx9fmQ (envelope-from ) for ; Fri, 10 Sep 2021 13:25:11 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 271F0C779 for ; Fri, 10 Sep 2021 15:25:11 +0200 (CEST) Received: from localhost ([::1]:45690 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mOgWk-0004CD-1T for larch@yhetil.org; Fri, 10 Sep 2021 09:25:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50030) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mOgWc-000499-RP for guix-patches@gnu.org; Fri, 10 Sep 2021 09:25:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:53947) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mOgWc-0001uI-KA for guix-patches@gnu.org; Fri, 10 Sep 2021 09:25:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mOgWb-0006pq-OV for guix-patches@gnu.org; Fri, 10 Sep 2021 09:25:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50359] [PATCH] import: Add 'generic-git' updater. Resent-From: Xinglu Chen Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 10 Sep 2021 13:25:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50359 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Sarah Morgensen , 50359@debbugs.gnu.org Received: via spool by 50359-submit@debbugs.gnu.org id=B50359.163128025626212 (code B ref 50359); Fri, 10 Sep 2021 13:25:01 +0000 Received: (at 50359) by debbugs.gnu.org; 10 Sep 2021 13:24:16 +0000 Received: from localhost ([127.0.0.1]:37260 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mOgVs-0006oh-CH for submit@debbugs.gnu.org; Fri, 10 Sep 2021 09:24:16 -0400 Received: from h87-96-130-155.cust.a3fiber.se ([87.96.130.155]:32840 helo=mail.yoctocell.xyz) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mOgVl-0006oN-Dg for 50359@debbugs.gnu.org; Fri, 10 Sep 2021 09:24:14 -0400 From: Xinglu Chen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yoctocell.xyz; s=mail; t=1631280241; bh=dA/Av/IPp+kgHuUFg4yuSP5fWmSIuSvL7H81Ahln9XU=; h=From:To:Cc:Subject:In-Reply-To:References:Date; b=SqoVnPZ8gwW5ffYbExWKbL2MUVKaazlkzjU5Oqcir88ToTs3VH3ROEeZYwcDFM4tQ n9stD1FGmfyHwPqviIMKHcY/Y6WJUBwYNrub5jkxXUqNyTe9l/UZbiI+ecL8uTFHDj EbH4cWejKq+d3yVbfVcGLvXFDmKvqb+1HFcDLFDY= In-Reply-To: <87o890ddz7.fsf_-_@gnu.org> References: <86k0jvkh5v.fsf@mgsn.dev> <87h7ez48d3.fsf@yoctocell.xyz> <86y28ai7ns.fsf@mgsn.dev> <87y28928vh.fsf@yoctocell.xyz> <86pmtli4hn.fsf@mgsn.dev> <87wnnsyzal.fsf@yoctocell.xyz> <87mtomzzu1.fsf@yoctocell.xyz> <87o890ddz7.fsf_-_@gnu.org> Date: Fri, 10 Sep 2021 15:23:46 +0200 Message-ID: <87bl50zhr1.fsf@yoctocell.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1631280311; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post: dkim-signature; bh=EWDsfn3QCuhgVwYApChDI9lEyE9PqHoiKkx4la33SEU=; b=IztBZjpfchvfVtJKuDs8cLynMBu3SWzR7GMe0D4WbNipJAEMRH3fwSykymNZ/VWYMDSIFU 7fR4PQdxGF9ZSHMVDbO5sLqtVqwOdO3MQ7+vIn1meL7mzxAN+OAovG/LIXZeIoNeb0xfFQ LHngrO8lM9g7rYcHJPvVyBKETIV3SjdrlDf8pP79yXnDO8Tr4ckL5tzgkd0esu83XnhnUv KaYnvAsIpMEEiRGaFVm1FDmvZh31tt5URnh0sJFnhaCEhub7OIa6tgvNFLjJEndsZ/0AAb aTumTg2EzC93A0kAOLT2Dw5JbpHVPlH2kTfEosYJhEVvcvZIEiG1LHsA5cw6kg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1631280311; a=rsa-sha256; cv=none; b=Lydl+KBIuW0l6c4avXhUhmKjLeVg17PjI+kSjESDadBMLO8o2lCytEp7T1PDOA/3kw9CCc E8Q+zqaMXqsEjJOPl+LCrQP47XcxEXUjmZqVscXfFUr8KKYPV7+XyxHcDY6KEAacnPIJAG hAc+3hLJ6tdP48IXXHNXmVN03WZLd/uy1Q66dWz7dO7vPPfAeqQZ99nqEBWNYK9D+sOcTa IsaqTx9abEqUJKqVLLOFwowcrblEWfHIsnkKZ7tpSHo1FxWrQOXqJLxTZLbN42Q4dR247R onYDQDJFpCvljxwZaQIwv48b42LTUA9N7DlN/4nP6eK46oS+TC0VmUoaCqQjFg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=yoctocell.xyz header.s=mail header.b=SqoVnPZ8; dmarc=fail reason="SPF not aligned (relaxed)" header.from=yoctocell.xyz (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Spam-Score: -1.91 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=yoctocell.xyz header.s=mail header.b=SqoVnPZ8; dmarc=fail reason="SPF not aligned (relaxed)" header.from=yoctocell.xyz (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: 271F0C779 X-Spam-Score: -1.91 X-Migadu-Scanner: scn0.migadu.com X-TUID: UJDL4XsWucz/ --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On Fri, Sep 10 2021, Ludovic Court=C3=A8s wrote: > Hello, > > This looks very cool! Thanks for taking a look! It=E2=80=99s still a WIP, but I think it=E2=80=99s getting there. :-) > Xinglu Chen skribis: > >> From f924dbb835425f6b9a5796918125592870391405 Mon Sep 17 00:00:00 2001 >> Message-Id: >> From: Xinglu Chen >> Date: Fri, 3 Sep 2021 17:50:56 +0200 >> Subject: [PATCH] import: Add 'generic-git' updater. >> >> * guix/import/git.scm: New file. >> * doc/guix.texi (Invoking guix refresh): Document it. >> * Makefile.am (MODULES): Register it. >> * guix/git.scm (ls-remote-refs): New procedure. >> >> Co-authored-by: Sarah Morgensen > > Overall LGTM; comments below. > >> diff --git a/doc/guix.texi b/doc/guix.texi >> index 36a0c7f5ec..26afb1607a 100644 >> --- a/doc/guix.texi >> +++ b/doc/guix.texi >> @@ -11920,6 +11920,33 @@ the updater for @uref{https://launchpad.net, La= unchpad} packages. >> @item generic-html >> a generic updater that crawls the HTML page where the source tarball of >> the package is hosted, when applicable. >> +@item generic-git > > Please add a newline above. Noted. >> +@lisp >> +(package >> + (name "foo") >> + ;; ... >> + (properties >> + '((tag-prefix . "^release0-") >> + (tag-suffix . "[a-z]?$") >> + (tag-version-delimiter . ":")))) >> +@end lisp=20=20=20=20=20=20 > > Very nice. Perhaps s/tag-/release-tag-/ for clarity? Good idea. >> +(define* (ls-remote-refs url #:key tags?) >> + "Return the list of references advertised at Git repository URL. If = TAGS? >> +is true, limit to only refs/tags." > > To remain consistent with existing naming conventions, I=E2=80=99d call it > =E2=80=98remote-refs=E2=80=99. > >> + (with-libgit2 >> + (call-with-temporary-directory >> + (lambda (cache-directory) >> + (let* ((repository (repository-init cache-directory)) >> + ;; Create an in-memory remote so we don't touch disk. >> + (remote (remote-create-anonymous repository url))) > > Too bad we need to create an empty repo; hopefully it costs next to > nothing though. > >> + (remote-connect remote) >> + (remote-disconnect remote) >> + (repository-close! repository) >> + >> + (filter include? (map remote-head-name (remote-ls remote)))))))) > > Use =E2=80=98filter-map=E2=80=99. > >> +(define* (get-version-mapping tags #:key prefix suffix delim pre-releas= es?) > > Please add a docstring and remove =E2=80=98get-=E2=80=99 from the name. = :-) > >> + (define (guess-delim) >> + (let ((total (length tags)) >> + (dots (reduce + 0 (map (cut string-count <> #\.) tags))) >> + (dashes (reduce + 0 (map (cut string-count <> #\-) tags))) >> + (underscores (reduce + 0 (map (cut string-count <> #\_) tags)= ))) >> + (display (format #t "total: ~d, dots: ~d, dashes ~d, underscores = ~d~%" >> + total dots dashes underscores)) > > Leftover? (Also display + format.) Yep. :-) > Please spell out =E2=80=98delimiter=E2=80=99 (info "(guix) Formatting Cod= e"). > >> + (cond >> + ((>=3D dots (* total 0.35)) ".") >> + ((>=3D dashes (* total 0.8)) "-") >> + ((>=3D underscores (* total 0.8)) "_") >> + (else "")))) > > That=E2=80=99s a fancy heuristic. :-) Yeah, it was suggested by Sarah, and in my testing it seems to work pretty well. :-) >> + (let ((mapping (fold alist-cons '() (map get-version tags) tags))) >> + (stable-sort! (filter car mapping) entry > It=E2=80=99s perhaps clearer written like this: > > (stable-sort (filter-map (lambda (tag) > (let ((version (get-version tag))) > (and version (cons version tag)))) > tags) > entry> +(define* (get-latest-tag url #:key prefix suffix delim pre-releases?) >> + "Return the latest tag available from the Git repository at URL." > > Maybe =E2=80=9Cthe tag corresponding to the latest version=E2=80=9D. Yeah, as the latest tag might not correspond to a release... > s/get-latest-tag/latest-tag/ > >> + (define (pre-release? tag) >> + (any (lambda (rx) (regexp-exec (make-regexp rx regexp/icase) tag)) >> + %pre-release-words)) > > Better call =E2=80=98make-regexp=E2=80=99 only once; so you could change > =E2=80=98%pre-release-words=E2=80=99 to be a list of regexp objects inste= ad of a list of > strings. Noted. >> +(define (latest-git-tag-version package tag-prefix tag-suffix >> + tag-version-delimiter refresh-pre-relea= ses?) >> + "Given a PACKAGE, the TAG-PREFIX, TAG-SUFFIX, TAG-VERSION-DELIMITER, = and >> +REFRESH-PRE-RELEASES? properties of PACKAGE, returns the latest versio= n of >> +PACKAGE." > > Maybe s/refresh-pre-releases?/accept-pre-preleases?/ =E2=80=98accept-pre-releases?=E2=80=99 ;-) > Since this procedure takes a package, it probably doesn=E2=80=99t need th= e other > arguments: it can extract them from the package properties, rather than > doing it at the call site. Good point. >> +(define (latest-git-release package) >> + "Return the latest release of PACKAGE." >> + (let* ((name (package-name package)) >> + (properties (package-properties package)) >> + (tag-prefix (assq-ref properties 'tag-prefix)) >> + (tag-suffix (assq-ref properties 'tag-suffix)) >> + (tag-version-delimiter (assq-ref properties 'tag-version-delim= iter)) >> + (refresh-pre-releases? (assq-ref properties 'refresh-pre-relea= ses?)) >> + (old-version (package-version package)) >> + (url (git-reference-url (origin-uri (package-source package)))) >> + (new-version (latest-git-tag-version package >> + tag-prefix >> + tag-suffix >> + tag-version-delimiter >> + refresh-pre-releases?))) >> + >> + (if new-version >> + (upstream-source >> + (package name) >> + (version new-version) >> + (urls (list url))) >> + ;; No new release or no tags available. >> + #f))) > > Simply: (and new-version (upstream-source =E2=80=A6)). > > It would have been nice to have tests. I think testing > =E2=80=98latest-git-release=E2=80=99 should be feasible without too much = hassle using > the (guix tests git) infrastructure, as is done in tests/git.scm, with a > package referring to a locally-created repo using a git-reference with a > file:// URL. Thanks for the pointers! I will look into it. > Could you send an updated patch? Sure! Thanks for the review! :-) --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQJJBAEBCAAzFiEEAVhh4yyK5+SEykIzrPUJmaL7XHkFAmE7XGMVHHB1YmxpY0B5 b2N0b2NlbGwueHl6AAoJEKz1CZmi+1x5WHkQALL76RuYAn3al9OovUTR6OM0PY80 icGunDKkIgsAPba0SWL+3SmohhwfKwtaKlOn7+LSCdTNmV9D9USS/58oGCwQ8z+o HX3SCHLGhAP66pHPbT/LKv5AZL2yvt/bPP6N8VKGuwoihWF3zVMmH2yjDl8ux8se hN4Orie8H1b8u2g+edApe7CpZWIX8K2caxy7yDKuyh7YeMOMtPgroy/03o2ucA0m odIRZXuTr4m/YD6O6gqKn7tRaIfzFPIMfF2FiZqHAB+g+pdRMdLFxLFA5QaW+3+W J3fRe4YQA08lA/YOobb1sOVT357pT4jVpAK9N3E6yulNil+D/jQfCkAhDulCDRLP aEO6SWntd8lBBDOvWnbUSuvZvQOKNED9qIOj/OXYmcfNBRze2YUpmchLiGv12Mai xrQfp6xOplPtDTci5ZYYSp/Y8ygMn+a6ClzA6e6rdru2BMYVjpvGe1fjvzz7laeP dFBQOpiblgL7K/xVF/xia0LvHmjFKDpm6zwccGZ9p+F/cNUeU4Mh+so3Y4V/Zjvk PWjktr9Yx+yK5Ue9bbLkEyXPHUAzDWRzz8p85FcKcIOkWUTKFHRWdqZK3wlB8hJ/ 9MastbBgN3dIIC9fBTWA0MSPOQN0VtUt8pkFxSxuZiYzVsADTblUbASYB/zWKeAY 92cDAVngc5i36LDp =RW8F -----END PGP SIGNATURE----- --=-=-=--