From mboxrd@z Thu Jan 1 00:00:00 1970 From: Efraim Flashner Subject: Re: `guix lint' warn of GitHub autogenerated source tarballs Date: Wed, 19 Dec 2018 21:29:26 +0200 Message-ID: <20181219192926.GB2581@macbook41> References: <87pntxwqx0.fsf@gnu.org> <08635A1A-EDA5-44B0-8C8A-532F16683154@flashner.co.il> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="z4+8/lEcDcG5Ke9S" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:48658) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gZhXC-0002n8-JX for guix-devel@gnu.org; Wed, 19 Dec 2018 14:29:35 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gZhX8-0007c4-02 for guix-devel@gnu.org; Wed, 19 Dec 2018 14:29:34 -0500 Received: from flashner.co.il ([178.62.234.194]:45450) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gZhX7-0007az-KU for guix-devel@gnu.org; Wed, 19 Dec 2018 14:29:29 -0500 Content-Disposition: inline In-Reply-To: List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Arun Isaac Cc: guix-devel@gnu.org --z4+8/lEcDcG5Ke9S Content-Type: multipart/mixed; boundary="7ZAtKRhVyVSsbBD2" Content-Disposition: inline --7ZAtKRhVyVSsbBD2 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Dec 19, 2018 at 11:13:56PM +0530, Arun Isaac wrote: >=20 > >>> Now that we are avoiding GitHub autogenerated source tarballs since > >>they > >>> are unstable and cause hash mismatch errors, can we have `guix lint' > >>> emit a warning if these autogenerated source tarballs are used? > >> > > I think I just posted a paste on IRC but haven't sent a patch > > yet. I'll grab it and submit it, it's almost done, just needs some > > cleaning up and tightening the test cases. >=20 > Great, thank you! >=20 Here's what I currently have. I don't think I've tried running the tests I've written yet, and Ludo said there was a better way to check if the download was a git-fetch or a url-fetch. As the logic is currently written it'll flag any package hosted on github owned by 'archive' or any package named 'archive' in addition to the ones we want. --=20 Efraim Flashner =D7=90=D7=A4=D7=A8=D7=99=D7=9D = =D7=A4=D7=9C=D7=A9=D7=A0=D7=A8 GPG key =3D A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted --7ZAtKRhVyVSsbBD2 Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="0001-lint-Add-checker-for-unstable-tarballs.patch" Content-Transfer-Encoding: quoted-printable =46rom 8a07c8aea1f23db48a9e69956ad15f79f0f70e35 Mon Sep 17 00:00:00 2001 =46rom: Efraim Flashner Date: Tue, 23 Oct 2018 12:01:53 +0300 Subject: [PATCH] lint: Add checker for unstable tarballs. * guix/scripts/lint.scm (check-source-unstable-tarball): New procedure. (%checkers): Add it. * tests/lint.scm ("source-unstable-tarball", source-unstable-tarball: source #f", "source-unstable-tarball: valid", source-unstable-tarball: not-github", source-unstable-tarball: git-fetch"): New tests. --- guix/scripts/lint.scm | 23 ++++++++++++++- tests/lint.scm | 68 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 90 insertions(+), 1 deletion(-) diff --git a/guix/scripts/lint.scm b/guix/scripts/lint.scm index e477bf0dd..cce7af66c 100644 --- a/guix/scripts/lint.scm +++ b/guix/scripts/lint.scm @@ -7,7 +7,7 @@ ;;; Copyright =C2=A9 2016 Hartmut Goebel ;;; Copyright =C2=A9 2017 Alex Kost ;;; Copyright =C2=A9 2017 Tobias Geerinckx-Rice -;;; Copyright =C2=A9 2017 Efraim Flashner +;;; Copyright =C2=A9 2017, 2018 Efraim Flashner ;;; ;;; This file is part of GNU Guix. ;;; @@ -747,6 +747,23 @@ descriptions maintained upstream." (G_ "the source file name should contain the package n= ame") 'source)))) =20 +(define (check-source-unstable-tarball package) + "Emit a warning if PACKAGE's source is an autogenerated tarball." + (define (github-tarball? origin) + (string-contains origin "github.com")) + (define (autogenerated-tarball? origin) + (string-contains origin "/archive/")) + (let ((origin (package-source package))) + (unless (not origin) ; check for '(source #f)' + (let ((uri (origin-uri origin)) + (dl-method (origin-method origin))) + (unless (not (pk dl-method "url-fetch")) + (when (and (github-tarball? uri) + (autogenerated-tarball? uri)) + (emit-warning package + (G_ "the source URI should not be an autogenerated= tarball") + 'source))))))) + (define (check-mirror-url package) "Check whether PACKAGE uses source URLs that should be 'mirror://'." (define (check-mirror-uri uri) ;XXX: could be optimized @@ -1051,6 +1068,10 @@ or a list thereof") (name 'source-file-name) (description "Validate file names of sources") (check check-source-file-name)) + (lint-checker + (name 'source-unstable-tarball) + (description "Check for autogenerated tarballs") + (check check-source-unstable-tarball)) (lint-checker (name 'derivation) (description "Report failure to compile a package to a derivation") diff --git a/tests/lint.scm b/tests/lint.scm index ab0e8b9a8..723a35107 100644 --- a/tests/lint.scm +++ b/tests/lint.scm @@ -571,6 +571,74 @@ (check-source-file-name pkg))) "file name should contain the package name")))) =20 +(test-assert "source-unstable-tarball" + (not + (->bool + (string-contains + (with-warnings + (let ((pkg (dummy-package "x" + (source + (origin + (method url-fetch) + (uri "https://github.com/example/example/archive/= v0.0.tar.gz") + (sha256 %null-sha256)))))) + (check-source-unstable-tarball pkg))) + "source URI should not be an autogenerated tarball")))) + +(test-assert "source-unstable-tarball: source #f" + (not + (->bool + (string-contains + (with-warnings + (let ((pkg (dummy-package "x" + (source #f)))) + (check-source-unstable-tarball pkg))) + "source URI should not be an autogenerated tarball")))) + +(test-assert "source-unstable-tarball: valid" + (not + (->bool + (string-contains + (with-warnings + (let ((pkg (dummy-package "x" + (source + (origin + (method url-fetch) + (uri "https://github.com/example/example/releases= /download/x-0.0/x-0.0.tar.gz") + (sha256 %null-sha256)))))) + (check-source-unstable-tarball pkg))) + "source URI should not be an autogenerated tarball")))) + +(test-assert "source-unstable-tarball: not-github" + (not + (->bool + (string-contains + (with-warnings + (let ((pkg (dummy-package "x" + (source + (origin + (method url-fetch) + (uri "https://bitbucket.org/archive/example/downl= oad/x-0.0.tar.gz") + (sha256 %null-sha256)))))) + (check-source-unstable-tarball pkg))) + "source URI should not be an autogenerated tarball")))) + +(test-assert "source-unstable-tarball: git-fetch" + (not + (->bool + (string-contains + (with-warnings + (let ((pkg (dummy-package "x" + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/archive/example.g= it") + (commit "0"))) + (sha256 %null-sha256)))))) + (check-source-unstable-tarball pkg))) + "source URI should not be an autogenerated tarball")))) + (test-skip (if (http-server-can-listen?) 0 1)) (test-equal "source: 200" "" --=20 2.19.1 --7ZAtKRhVyVSsbBD2-- --z4+8/lEcDcG5Ke9S Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAlwanBIACgkQQarn3Mo9 g1HwNQ/+NM/aKJlYAW3haQI0MZNODFheW1YkBW6S/mc9cPdsWm2pwZyYjZAbxxh7 pgox2wYhq/Z+n8TOgzwArB48X3hVMbcDOKf6Ma6n9XRMgF0UiAn1R8+NzHszowHg 62XBjuAY4Mc/VmwLJL/Q6dKQLXkl1QHHTELHGGgfdwlwbLCQWC1ZQFPytkKEY375 FUNS/obp9+zKRhkjBib6mETQz0kItkQ9hyqRGkPDFfN7Gzo20y+fbV+bLRrqbOyw g55UHJl0RLJmSIG0KBKwgS0/R1wxparfmC3T7SNqWIe1wO8vzKXjdhjfu841EJpm FVhD+24Kh4nY0VJMV5VJz1GQYWRiGO+Rm4tvuIbEFwrKZ137A4zbAh7v9UTSx4+k 2HrUORWhHfbdnjUDGFy1TjH07GsEhZwEBORw5gsgaJOpG8S1aT8mt0Pvnt5/3l2c ftVSndupjnYwDALi/UPJem9eZrscX492bdHUvJXEfiPaS3AO+P9Ox6dGOdCKJUZw takJfmw6nUDAtd7cVcw/uR2eP2SucRia6Pw5nlvKhgGHNdNizP5ZW40QPnDhGFcj k1LyYANJTM/IzkoHSITc4ZnQvcTMrA9qc4KsEvxDA+X+TbVGwc8+QzZuSUnOxX9y aSh5v3DulwyQNCiUHBN6DagySjRE3u/YDWEehgMBrBHf283q7RM= =cCI3 -----END PGP SIGNATURE----- --z4+8/lEcDcG5Ke9S--