From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id 0C9WNggVZl/OegAA0tVLHw (envelope-from ) for ; Sat, 19 Sep 2020 14:26:16 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id 2N4gMggVZl9rCwAAbx9fmQ (envelope-from ) for ; Sat, 19 Sep 2020 14:26:16 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id D36499402AA for ; Sat, 19 Sep 2020 14:26:15 +0000 (UTC) Received: from localhost ([::1]:33404 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kJdoc-0005dC-Ho for larch@yhetil.org; Sat, 19 Sep 2020 10:26:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33010) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kJdoQ-0005ch-2f for bug-guix@gnu.org; Sat, 19 Sep 2020 10:26:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:36163) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kJdoP-0002n8-Pu for bug-guix@gnu.org; Sat, 19 Sep 2020 10:26:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kJdoP-0005QS-N8 for bug-guix@gnu.org; Sat, 19 Sep 2020 10:26:01 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#43501: Whitelisting/blacklisting transitive package dependencies Resent-From: Danny Milosavljevic Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sat, 19 Sep 2020 14:26:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43501 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Mark H Weaver Received: via spool by 43501-submit@debbugs.gnu.org id=B43501.160052550320713 (code B ref 43501); Sat, 19 Sep 2020 14:26:01 +0000 Received: (at 43501) by debbugs.gnu.org; 19 Sep 2020 14:25:03 +0000 Received: from localhost ([127.0.0.1]:47673 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJdnS-0005Nx-VO for submit@debbugs.gnu.org; Sat, 19 Sep 2020 10:25:03 -0400 Received: from dd26836.kasserver.com ([85.13.145.193]:42452) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJdnP-0005NS-6d for 43501@debbugs.gnu.org; Sat, 19 Sep 2020 10:25:01 -0400 Received: from localhost (80-110-126-103.cgn.dynamic.surfer.at [80.110.126.103]) by dd26836.kasserver.com (Postfix) with ESMTPSA id 853E63360809; Sat, 19 Sep 2020 16:24:57 +0200 (CEST) Date: Sat, 19 Sep 2020 16:24:11 +0200 From: Danny Milosavljevic Message-ID: <20200919162411.1b6f621b@scratchpost.org> In-Reply-To: <87wo0q77t3.fsf@netris.org> References: <878sd6zgii.fsf@gmail.com> <87wo0q77t3.fsf@netris.org> X-Mailer: Claws Mail 3.17.5 (GTK+ 2.24.32; x86_64-unknown-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="Sig_/9iZZ=xi4/LE6qBy1R/=UhNR"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -1.7 (-) X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Maxim Cournoyer , 43501@debbugs.gnu.org Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Spam-Score: -1.11 X-TUID: vja2wTxEk+4a --Sig_/9iZZ=xi4/LE6qBy1R/=UhNR Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Hi, On Fri, 18 Sep 2020 23:40:13 -0400 Mark H Weaver wrote: > I think it's important that Guix core functionality should be usable > without installing a collection of patented media codecs. Those plugins > should be purely optional. In my opinion, we should find a way to > eliminate those dependencies. I agree that it would be good to prevent weird dependencies from creeping i= n--for your stated reasons, but also for a lot of other reasons, chief of which is= that the most secure source code is the source code that has been eliminated. Also, Guix sometimes pulls in transitive dependencies for the weirdest thin= gs. The "*-minimal" packages we have make it less bad. Still, it would nice to also have something that automatically checks wheth= er there are weird transitive inputs of a package, for each package (*especial= ly* in order to use that for "-minimal" packages). I sometimes add #:disallowed-references after tracking down problems of unintended transitive inputs (for example see f2fs-tools/static). But even that disallows just one specific reference (one package version). What I want is to disallow any package of that name entirely in the depende= ncy graph--or even disallow references to specific source files (or other group= s of packages) entirely. And I want it to keep disallowing it mechanically without me having to remember it. Guix lint already does something like I want, but for direct (non-transitiv= e) inputs. It should be possible to add a "guix lint" check that also checks transitive inputs of packages for suspicious packages. The maintenance of a transitive-whitelist/-blacklist per package would then probably be have to be done inside guix lint, though. It could be nicer if there were package fields for those for it eventually. But for now, I guess inside guix lint is good enough. That said, for practicality one has to find some kind of groups of packages, in order to keep the whitelist/blacklist from ballooning. For now, I assume that each group has an extra source file--which we know is not true in Guix right now. But we could make it true. I started to add something to guix lint (possible procedures to use: package-transitive-inputs, package-transitive-propagated-inputs, package-transitive-native-inputs)--see patch below. But note that it just complains about everything now--we would still have to specify what is "bad". I would suggest to have a whitelist (of file names) and a blacklist (of file names), and the following: If a package has a whitelist and a transitive dependency is not on the whitelist, complain. If a package does not have a whitelist but does have a blacklist and a transitive dependency is on the blacklist, complain. I still find it illuminating as it is now. Try: $ guix lint qemu [...] gnu/packages/virtualization.scm:260:5: qemu@5.0.0: 'gnu/packages/dbm.scm' s= hould probably not be referred to (but it is--because of packages (gdbm)) [...] gnu/packages/virtualization.scm:260:5: qemu@5.0.0: 'gnu/packages/spice.scm'= should probably not be referred to (but it is by packages (libcacard spice= usbredir virglrenderer spice-protocol)) gnu/packages/virtualization.scm:260:5: qemu@5.0.0: 'gnu/packages/gl.scm' sh= ould probably not be referred to (but it is by packages (libepoxy mesa)) [...] gnu/packages/virtualization.scm:260:5: qemu@5.0.0: 'gnu/packages/dbm.scm' s= hould probably not be referred to (but it is by packages (gdbm)) [...] gnu/packages/virtualization.scm:260:5: qemu@5.0.0: 'gnu/packages/pulseaudio= .scm' should probably not be referred to (but it is because of packages (pu= lseaudio)) WTF! Also, for the special case where no package in a source file A should refer= to any package in a source file B, it could be enough to establish a convention of commenting out the respective "#:use-module (...)" in source file A (and adding a "DO NOT USE" text to it), and never deleting that comment. That way, once somebody had found what module one should not ever import, he could document that fact. diff --git a/guix/lint.scm b/guix/lint.scm index ec43a4dcad..d65ac34441 100644 --- a/guix/lint.scm +++ b/guix/lint.scm @@ -72,6 +72,7 @@ #:export (check-description-style check-inputs-should-be-native check-inputs-should-not-be-an-input-at-all + check-transitive-input-sanity check-patch-file-names check-synopsis-style check-derivation @@ -287,6 +288,38 @@ of a package, and INPUT-NAMES, a list of package speci= fications such as input)))) packages outputs)))) =20 +(define (check-transitive-input-sanity package) + (let* ((examined-package-name (package-name package)) + (examined-package-location (package-location package)) + (examined-package-source-file-name (location-file examined-packag= e-location)) + (examined-package-dependency-source-file-names + (delete examined-package-source-file-name + (delete-duplicates + (map (match-lambda + ((key dependency . rest) (location-file (package-locatio= n dependency)))) + (package-transitive-target-inputs package)))))) + (map (lambda (source-file-name) + (let ((packages-in-source-file + (filter (match-lambda + ((key dependency . rest) (string=3D? source-fi= le-name + (location-f= ile + (package-l= ocation dependency))))) + (package-transitive-target-inputs package)))) + (make-warning package + (G_ "'~a' should probably not be referred to (but it is--bec= ause of packages ~a)") + (list source-file-name (map (match-lambda + ((key dependency . rest) + (package-name dependency))) + packages-in-source-file)) + #:field 'inputs))) + examined-package-dependency-source-file-names))) + (delete examined-package-source-file-name + (delete-duplicates + (map (match-lambda + ((key dependency . rest) (location-file (package-locatio= n dependency)))) + (package-transitive-target-inputs package)))))) + (map (lambda (source-file-name) + (let ((packages-in-source-file + (filter (match-lambda + ((key dependency . rest) (string=3D? source-fi= le-name + (location-f= ile + (package-l= ocation dependency))))) + (package-transitive-target-inputs package)))) + (make-warning package + (G_ "'~a' should probably not be referred to (but it is--bec= ause of packages ~a)") + (list source-file-name (map (match-lambda + ((key dependency . rest) + (package-name dependency))) + packages-in-source-file)) + #:field 'inputs))) + examined-package-dependency-source-file-names))) + + ;; if examined-package-name like '%qemu%': + ;; (package-name (map package-transitive-inputs)) no gstre= amer; source file not + ;; gstreamer.scm, gtk.scm. + ;; allowed references to location-files + ;; TODO: gstreamer should not be anywhere in any transitive inputs of an= y qemu + (define (check-inputs-should-be-native package) ;; Emit a warning if some inputs of PACKAGE are likely to belong to its ;; native inputs. @@ -1378,6 +1411,10 @@ them for PACKAGE." =20 (define %local-checkers (list + (lint-checker + (name 'transitive-inputs) + (description "Checks transitive inputs") + (check check-transitive-input-sanity)) (lint-checker (name 'description) (description "Validate package descriptions") --Sig_/9iZZ=xi4/LE6qBy1R/=UhNR Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAl9mFIsACgkQ5xo1VCww uqXtZAf/VHwgXpYbcEtzGuHSMX68vHOmDb4ZP6li5HQ7H5JwBADQwfC8jyQke47k UM5QVokU04Sg3oh2hvjwGngmVe3UrtvHLvg5EyGi+endKJAee891dikWCWQUrCLC RYbNU16qgFCs1jrtDQwUrP2b2ztveut9ElumyF3Y+lXmU4/laC4/76fP0NrEpaEe SS5ATpR5SHOXHU0FzE3gmc8dq359Ha1Wf8iAZUy7Yd8nv72jUURwvsbJKqEk6p2q nwy2ZeCuHmyF1W1AE512h/Wg1Gt5U7Y5v/R52JHyQ8uZ+WHmABNk8BRcf4oOO6Dx dmB0tY5/gqqhGUffew7RImvqPoJxvw== =VKZJ -----END PGP SIGNATURE----- --Sig_/9iZZ=xi4/LE6qBy1R/=UhNR--