From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 4IDyBjQZO2Fe4gAAgWs5BA (envelope-from ) for ; Fri, 10 Sep 2021 10:37:08 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id kLyhAjQZO2HdeAAAB5/wlQ (envelope-from ) for ; Fri, 10 Sep 2021 08:37:08 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 8C1758859 for ; Fri, 10 Sep 2021 10:37:07 +0200 (CEST) Received: from localhost ([::1]:43328 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mOc1y-0008Re-MP for larch@yhetil.org; Fri, 10 Sep 2021 04:37:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45722) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mOc1u-0008RL-Fz for guix-patches@gnu.org; Fri, 10 Sep 2021 04:37:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:53552) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mOc1u-00079s-8Q for guix-patches@gnu.org; Fri, 10 Sep 2021 04:37:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mOc1u-0004co-5M for guix-patches@gnu.org; Fri, 10 Sep 2021 04:37:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50359] [PATCH] import: Add 'generic-git' updater. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 10 Sep 2021 08:37:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50359 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Xinglu Chen Cc: Sarah Morgensen , 50359@debbugs.gnu.org Received: via spool by 50359-submit@debbugs.gnu.org id=B50359.163126298917734 (code B ref 50359); Fri, 10 Sep 2021 08:37:02 +0000 Received: (at 50359) by debbugs.gnu.org; 10 Sep 2021 08:36:29 +0000 Received: from localhost ([127.0.0.1]:36865 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mOc1M-0004by-Mm for submit@debbugs.gnu.org; Fri, 10 Sep 2021 04:36:29 -0400 Received: from eggs.gnu.org ([209.51.188.92]:56760) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mOc1K-0004bk-Rd for 50359@debbugs.gnu.org; Fri, 10 Sep 2021 04:36:27 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:37190) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mOc1A-0006OB-P7; Fri, 10 Sep 2021 04:36:17 -0400 Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=47218 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mOc1A-0004Ru-1e; Fri, 10 Sep 2021 04:36:16 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <86k0jvkh5v.fsf@mgsn.dev> <87h7ez48d3.fsf@yoctocell.xyz> <86y28ai7ns.fsf@mgsn.dev> <87y28928vh.fsf@yoctocell.xyz> <86pmtli4hn.fsf@mgsn.dev> <87wnnsyzal.fsf@yoctocell.xyz> <87mtomzzu1.fsf@yoctocell.xyz> Date: Fri, 10 Sep 2021 10:36:12 +0200 In-Reply-To: <87mtomzzu1.fsf@yoctocell.xyz> (Xinglu Chen's message of "Wed, 08 Sep 2021 20:28:38 +0200") Message-ID: <87o890ddz7.fsf_-_@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1631263027; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=8Prz49DT83+FAhCmY1XrcyM/dBgkpKrtAKEuN/0Wug8=; b=HBlFH55GFPbgj8JFFmJcBlGTIg1qYADrdkgpwR9ld2apbl3iA7+RrIIs67AoiJgt6MAoIU 15bg8lCl861c0TirX1OG3OxMNKoGMlp+bGj5+NjdM6QPkGOo0/YF8buXy7sabZ4fIRPq9b WnHQerUzWKke7OEGa2FpooR9RS12sD7fpo1UwQexR1LFzm9qomHoIBXUjQMrd0SQWY3JVO AWFdOFUAr/pupGdwHBOkkvm7V1Ki4XPXdBUwmh4pfTJa+nhYFa93zY+cAMGzuTMJcRLZ41 esSOLfBLz3h05YB0qCZWErxVvXwYVh0P21DcQ4rflvdyXirswX+GEj+LUdUWyg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1631263027; a=rsa-sha256; cv=none; b=SlglPo/ue+ixA25QxVqWDL4UB3yMN37jBRf1RbddvfNAVCFGHLW4BzplycaAJjIUNSkAw2 7/7RpEhNpMUDb4OuQOcA70hHQGNN2YjPjwPQJDXtkERaiqbr1dOMkC31FXrPQ1Hw8wKsWW CyYFlWmjzhK8Gu4msS4Rqe6oP1zH2NVe0UvUSz9pWiwy8UdgBG+V+in8J10ExaokWo01U6 ieC/7mnIsYjptEpLIvrkaPHs3MG84LMrP/HbnSXFDx5d9VFuf2rYoFQoWJGaAfSefvTyX3 dRHPE33875JdIzrRxZXqSVAfjT63bL1mrPZYNOTl+ePE/kE2SS1Iv2QPszt5ng== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Spam-Score: -2.91 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: 8C1758859 X-Spam-Score: -2.91 X-Migadu-Scanner: scn0.migadu.com X-TUID: NOiGAdZA++rm Hello, This looks very cool! Xinglu Chen skribis: > From f924dbb835425f6b9a5796918125592870391405 Mon Sep 17 00:00:00 2001 > Message-Id: > From: Xinglu Chen > Date: Fri, 3 Sep 2021 17:50:56 +0200 > Subject: [PATCH] import: Add 'generic-git' updater. > > * guix/import/git.scm: New file. > * doc/guix.texi (Invoking guix refresh): Document it. > * Makefile.am (MODULES): Register it. > * guix/git.scm (ls-remote-refs): New procedure. > > Co-authored-by: Sarah Morgensen Overall LGTM; comments below. > diff --git a/doc/guix.texi b/doc/guix.texi > index 36a0c7f5ec..26afb1607a 100644 > --- a/doc/guix.texi > +++ b/doc/guix.texi > @@ -11920,6 +11920,33 @@ the updater for @uref{https://launchpad.net, Lau= nchpad} packages. > @item generic-html > a generic updater that crawls the HTML page where the source tarball of > the package is hosted, when applicable. > +@item generic-git Please add a newline above. > +@lisp > +(package > + (name "foo") > + ;; ... > + (properties > + '((tag-prefix . "^release0-") > + (tag-suffix . "[a-z]?$") > + (tag-version-delimiter . ":")))) > +@end lisp=20=20=20=20=20=20 Very nice. Perhaps s/tag-/release-tag-/ for clarity? > +(define* (ls-remote-refs url #:key tags?) > + "Return the list of references advertised at Git repository URL. If T= AGS? > +is true, limit to only refs/tags." To remain consistent with existing naming conventions, I=E2=80=99d call it =E2=80=98remote-refs=E2=80=99. > + (with-libgit2 > + (call-with-temporary-directory > + (lambda (cache-directory) > + (let* ((repository (repository-init cache-directory)) > + ;; Create an in-memory remote so we don't touch disk. > + (remote (remote-create-anonymous repository url))) Too bad we need to create an empty repo; hopefully it costs next to nothing though. > + (remote-connect remote) > + (remote-disconnect remote) > + (repository-close! repository) > + > + (filter include? (map remote-head-name (remote-ls remote)))))))) Use =E2=80=98filter-map=E2=80=99. > +(define* (get-version-mapping tags #:key prefix suffix delim pre-release= s?) Please add a docstring and remove =E2=80=98get-=E2=80=99 from the name. :-) > + (define (guess-delim) > + (let ((total (length tags)) > + (dots (reduce + 0 (map (cut string-count <> #\.) tags))) > + (dashes (reduce + 0 (map (cut string-count <> #\-) tags))) > + (underscores (reduce + 0 (map (cut string-count <> #\_) tags))= )) > + (display (format #t "total: ~d, dots: ~d, dashes ~d, underscores ~= d~%" > + total dots dashes underscores)) Leftover? (Also display + format.) Please spell out =E2=80=98delimiter=E2=80=99 (info "(guix) Formatting Code"= ). > + (cond > + ((>=3D dots (* total 0.35)) ".") > + ((>=3D dashes (* total 0.8)) "-") > + ((>=3D underscores (* total 0.8)) "_") > + (else "")))) That=E2=80=99s a fancy heuristic. :-) > + (let ((mapping (fold alist-cons '() (map get-version tags) tags))) > + (stable-sort! (filter car mapping) entry +(define* (get-latest-tag url #:key prefix suffix delim pre-releases?) > + "Return the latest tag available from the Git repository at URL." Maybe =E2=80=9Cthe tag corresponding to the latest version=E2=80=9D. s/get-latest-tag/latest-tag/ > + (define (pre-release? tag) > + (any (lambda (rx) (regexp-exec (make-regexp rx regexp/icase) tag)) > + %pre-release-words)) Better call =E2=80=98make-regexp=E2=80=99 only once; so you could change =E2=80=98%pre-release-words=E2=80=99 to be a list of regexp objects instead= of a list of strings. > +(define (latest-git-tag-version package tag-prefix tag-suffix > + tag-version-delimiter refresh-pre-releas= es?) > + "Given a PACKAGE, the TAG-PREFIX, TAG-SUFFIX, TAG-VERSION-DELIMITER, a= nd > +REFRESH-PRE-RELEASES? properties of PACKAGE, returns the latest version= of > +PACKAGE." Maybe s/refresh-pre-releases?/accept-pre-preleases?/ Since this procedure takes a package, it probably doesn=E2=80=99t need the = other arguments: it can extract them from the package properties, rather than doing it at the call site. > +(define (latest-git-release package) > + "Return the latest release of PACKAGE." > + (let* ((name (package-name package)) > + (properties (package-properties package)) > + (tag-prefix (assq-ref properties 'tag-prefix)) > + (tag-suffix (assq-ref properties 'tag-suffix)) > + (tag-version-delimiter (assq-ref properties 'tag-version-delimi= ter)) > + (refresh-pre-releases? (assq-ref properties 'refresh-pre-releas= es?)) > + (old-version (package-version package)) > + (url (git-reference-url (origin-uri (package-source package)))) > + (new-version (latest-git-tag-version package > + tag-prefix > + tag-suffix > + tag-version-delimiter > + refresh-pre-releases?))) > + > + (if new-version > + (upstream-source > + (package name) > + (version new-version) > + (urls (list url))) > + ;; No new release or no tags available. > + #f))) Simply: (and new-version (upstream-source =E2=80=A6)). It would have been nice to have tests. I think testing =E2=80=98latest-git-release=E2=80=99 should be feasible without too much ha= ssle using the (guix tests git) infrastructure, as is done in tests/git.scm, with a package referring to a locally-created repo using a git-reference with a file:// URL. Could you send an updated patch? Thanks, Ludo=E2=80=99.