From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id sT8cBww0A19lKgAA0tVLHw (envelope-from ) for ; Mon, 06 Jul 2020 14:24:12 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id BCLNAgw0A1/lTAAAB5/wlQ (envelope-from ) for ; Mon, 06 Jul 2020 14:24:12 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 4A48A9407C7 for ; Mon, 6 Jul 2020 14:24:11 +0000 (UTC) Received: from localhost ([::1]:42768 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jsS2S-0005yl-TT for larch@yhetil.org; Mon, 06 Jul 2020 10:24:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52066) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jsS2M-0005yA-MT for guix-patches@gnu.org; Mon, 06 Jul 2020 10:24:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:51299) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jsS2M-0002DI-3l for guix-patches@gnu.org; Mon, 06 Jul 2020 10:24:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jsS2M-0007tv-0J for guix-patches@gnu.org; Mon, 06 Jul 2020 10:24:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#42123] [PATCH] linux-libre: Enable module compression. Resent-From: Mathieu Othacehe Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 06 Jul 2020 14:24:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 42123 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 42123@debbugs.gnu.org Received: via spool by 42123-submit@debbugs.gnu.org id=B42123.159404541830342 (code B ref 42123); Mon, 06 Jul 2020 14:24:01 +0000 Received: (at 42123) by debbugs.gnu.org; 6 Jul 2020 14:23:38 +0000 Received: from localhost ([127.0.0.1]:34612 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jsS1x-0007tK-Ug for submit@debbugs.gnu.org; Mon, 06 Jul 2020 10:23:38 -0400 Received: from eggs.gnu.org ([209.51.188.92]:37560) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jsS1t-0007t5-Hw for 42123@debbugs.gnu.org; Mon, 06 Jul 2020 10:23:36 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:55625) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jsS1n-00027g-Tt; Mon, 06 Jul 2020 10:23:27 -0400 Received: from lfbn-ann-1-356-169.w86-200.abo.wanadoo.fr ([86.200.73.169]:40520 helo=meru) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jsS1m-00080S-H5; Mon, 06 Jul 2020 10:23:27 -0400 From: Mathieu Othacehe References: <20200629142434.21308-1-othacehe@gnu.org> <87366atdve.fsf@gnu.org> <87lfjx6nb9.fsf@gnu.org> <873664ltqt.fsf@gnu.org> Date: Mon, 06 Jul 2020 16:23:23 +0200 In-Reply-To: <873664ltqt.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Mon, 06 Jul 2020 14:20:58 +0200") Message-ID: <878sfw7mec.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -3.3 (---) X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Spam-Score: -1.01 X-TUID: FGYGY4/5ohix --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hey, > I don=E2=80=99t have other ideas, but both solutions sound good to me. U= sing > (guix zlib) is slightly more =E2=80=9Celegant=E2=80=9D IMO, but no big de= al. I don=E2=80=99t > expect any significant difference from the use of in-process > decompression, unless we really have to go and decompress many modules > in a row. Creating the initrd implies to create the module name database, and it ends-up decompressing every single module. Using the in-process method it takes 2 seconds, using the second method 30 seconds. So, I opted for the first solution as you suggested. Here's an attached patch that fixes the situation. Thanks, Mathieu --=-=-= Content-Type: text/x-diff; charset=utf-8 Content-Disposition: inline; filename=import.patch Content-Transfer-Encoding: quoted-printable >From 8bbf343510091fad4a08758e0115a70410c1c8d7 Mon Sep 17 00:00:00 2001 From: Mathieu Othacehe Date: Mon, 6 Jul 2020 16:04:21 +0200 Subject: [PATCH] self: Add with-imported-modules+config and use it. Introduce "with-imported-modules+config" and use it to replace every call to "with-imported-modules" that would trigger an import of (guix config) modul= e. * guix/self.scm (not-config?): New procedure, (with-imported-modules+config): new macro. * guix/profiles.scm (linux-module-database): Replace with-imported-modules = by with-imported-modules+config. * gnu/system/shadow.scm (account-shepherd-service): Ditto. * gnu/system/linux-initrd.scm (raw-initrd): Ditto. * gnu/services/base.scm (default-serial-port): Ditto, (file-system-shepherd-service): ditto, (udev-shepherd-service): ditto. * gnu/services.scm (activation-script): Ditto. * gnu/machine/ssh.scm (machine-check-initrd-modules): Ditto. --- gnu/machine/ssh.scm | 8 ++++---- gnu/services.scm | 6 +++--- gnu/services/base.scm | 11 +++++------ gnu/system/linux-initrd.scm | 12 ++++++------ gnu/system/shadow.scm | 6 +++--- guix/profiles.scm | 6 +++--- guix/self.scm | 24 ++++++++++++++++++++++++ 7 files changed, 48 insertions(+), 25 deletions(-) diff --git a/gnu/machine/ssh.scm b/gnu/machine/ssh.scm index 4148639292..7369eb2136 100644 --- a/gnu/machine/ssh.scm +++ b/gnu/machine/ssh.scm @@ -33,6 +33,7 @@ #:use-module (guix records) #:use-module (guix remote) #:use-module (guix scripts system reconfigure) + #:use-module (guix self) #:use-module (guix ssh) #:use-module (guix store) #:use-module (guix utils) @@ -246,10 +247,9 @@ not available in the initrd." (define (missing-modules fs) (define remote-exp (let ((device (file-system-device fs))) - (with-imported-modules (source-module-closure - '((gnu build file-systems) - (gnu build linux-modules) - (gnu system uuid))) + (with-imported-modules+config '((gnu build file-systems) + (gnu build linux-modules) + (gnu system uuid)) #~(begin (use-modules (gnu build file-systems) (gnu build linux-modules) diff --git a/gnu/services.scm b/gnu/services.scm index f6dc56d940..4d7371cd78 100644 --- a/gnu/services.scm +++ b/gnu/services.scm @@ -28,6 +28,7 @@ #:use-module (guix combinators) #:use-module (guix channels) #:use-module (guix describe) + #:use-module (guix self) #:use-module (guix sets) #:use-module (guix ui) #:use-module ((guix utils) #:select (source-properties->location)) @@ -542,9 +543,8 @@ ACTIVATION-SCRIPT-TYPE." (map (cut program-file "activate-service.scm" <>) gexps)) =20 (program-file "activate.scm" - (with-imported-modules (source-module-closure - '((gnu build activation) - (guix build utils))) + (with-imported-modules+config '((gnu build activation) + (guix build utils)) #~(begin (use-modules (gnu build activation) (guix build utils)) diff --git a/gnu/services/base.scm b/gnu/services/base.scm index 6ea7ef8e7e..94dfeb2315 100644 --- a/gnu/services/base.scm +++ b/gnu/services/base.scm @@ -30,6 +30,7 @@ ;;; along with GNU Guix. If not, see . =20 (define-module (gnu services base) + #:use-module (guix self) #:use-module (guix store) #:use-module (guix deprecation) #:use-module (gnu services) @@ -832,8 +833,8 @@ the message of the day, among other things." (define (default-serial-port) "Return a gexp that determines a reasonable default serial port to use as the tty. This is primarily useful for headless systems." - (with-imported-modules (source-module-closure - '((gnu build linux-boot))) ;for 'find-long-optio= ns' + (with-imported-modules+config + '((gnu build linux-boot)) ;for 'find-long-options' #~(begin ;; console=3Ddevice,options ;; device: can be tty0, ttyS0, lp0, ttyUSB0 (serial). @@ -886,8 +887,7 @@ to use as the tty. This is primarily useful for headle= ss systems." =20 (modules '((ice-9 match) (gnu build linux-boot))) (start - (with-imported-modules (source-module-closure - '((gnu build linux-boot))) + (with-imported-modules+config '((gnu build linux-boot)) #~(lambda args (let ((defaulted-tty #$(or tty (default-serial-port)))) (apply @@ -1935,8 +1935,7 @@ item of @var{packages}." =20 (documentation "Populate the /dev directory, dynamically.") (start - (with-imported-modules (source-module-closure - '((gnu build linux-boot))) + (with-imported-modules+config '((gnu build linux-boot)) #~(lambda () (define udevd ;; 'udevd' from eudev. diff --git a/gnu/system/linux-initrd.scm b/gnu/system/linux-initrd.scm index 99ec82246b..8779ef58d7 100644 --- a/gnu/system/linux-initrd.scm +++ b/gnu/system/linux-initrd.scm @@ -28,6 +28,7 @@ #:use-module ((guix derivations) #:select (derivation->output-path)) #:use-module (guix modules) + #:use-module (guix self) #:use-module (gnu packages compression) #:use-module (gnu packages disk) #:use-module (gnu packages linux) @@ -214,12 +215,11 @@ upon error." (flat-linux-module-directory linux linux-modules)) =20 (expression->initrd - (with-imported-modules (source-module-closure - '((gnu build linux-boot) - (guix build utils) - (guix build bournish) - (gnu system file-systems) - (gnu build file-systems))) + (with-imported-modules+config '((gnu build linux-boot) + (guix build utils) + (guix build bournish) + (gnu system file-systems) + (gnu build file-systems)) #~(begin (use-modules (gnu build linux-boot) (gnu system file-systems) diff --git a/gnu/system/shadow.scm b/gnu/system/shadow.scm index a69339bc07..e140f06913 100644 --- a/gnu/system/shadow.scm +++ b/gnu/system/shadow.scm @@ -22,6 +22,7 @@ (define-module (gnu system shadow) #:use-module (guix records) #:use-module (guix gexp) + #:use-module (guix self) #:use-module (guix store) #:use-module (guix modules) #:use-module (guix sets) @@ -321,9 +322,8 @@ accounts among ACCOUNTS+GROUPS." (one-shot? #t) (modules '((gnu build activation) (gnu system accounts))) - (start (with-imported-modules (source-module-closure - '((gnu build activation) - (gnu system accounts))) + (start (with-imported-modules+config '((gnu build activation) + (gnu system accounts)) #~(lambda () (activate-user-home (map sexp->user-account diff --git a/guix/profiles.scm b/guix/profiles.scm index f34f73e17e..f11e400dd3 100644 --- a/guix/profiles.scm +++ b/guix/profiles.scm @@ -40,6 +40,7 @@ #:use-module (guix gexp) #:use-module (guix modules) #:use-module (guix monads) + #:use-module (guix self) #:use-module (guix store) #:use-module (ice-9 vlist) #:use-module (ice-9 match) @@ -1205,9 +1206,8 @@ This is meant to be used as a profile hook." (define kmod ; lazy reference (module-ref (resolve-interface '(gnu packages linux)) 'kmod)) (define build - (with-imported-modules (source-module-closure - '((guix build utils) - (gnu build linux-modules))) + (with-imported-modules+config '((guix build utils) + (gnu build linux-modules)) #~(begin (use-modules (ice-9 ftw) (ice-9 match) diff --git a/guix/self.scm b/guix/self.scm index e1350a7403..82bb55f8e7 100644 --- a/guix/self.scm +++ b/guix/self.scm @@ -33,6 +33,8 @@ #:use-module (srfi srfi-35) #:use-module (ice-9 match) #:export (make-config.scm + not-config? + with-imported-modules+config whole-package ;for internal use in 'guix p= ull' compiled-guix guix-derivation)) @@ -1063,6 +1065,24 @@ Info manual." ;; made relative to a nonexistent anonymous module. #:splice? #t)) =20 +(define not-config? + ;; Select (guix =E2=80=A6) and (gnu =E2=80=A6) modules, except (guix con= fig). + (match-lambda + (('guix 'config) #f) + (('guix rest ...) #t) + (('gnu rest ...) #t) + (rest #f))) + +(define-syntax-rule (with-imported-modules+config modules exp ...) + "Import the closure of MODULES and evaluate EXP within this context. If= the +(guix config) module is part of the closure, it is not selected. This mod= ule +is always replaced by a mocked-one, created by MAKE-CONFIG.SCM pocedure." + (with-imported-modules `(,@(source-module-closure + modules + #:select? not-config?) + ((guix config) =3D> ,(make-config.scm))) + exp ...)) + ;;; ;;; Building. @@ -1213,3 +1233,7 @@ is not supported." (if guix (lower-object guix) (return #f))))) + +;; Local Variables: +;; eval: (put 'with-imported-modules+config 'scheme-indent-function 2) +;; End: --=20 2.24.0 --=-=-=--