From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id uBJ8Ow/fq2JhWQAAbAwnHQ (envelope-from ) for ; Fri, 17 Jun 2022 03:55:28 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id yEdQOw/fq2I0DAAAauVa8A (envelope-from ) for ; Fri, 17 Jun 2022 03:55:27 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 1A1AC3E6C4 for ; Fri, 17 Jun 2022 03:55:27 +0200 (CEST) Received: from localhost ([::1]:60694 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o21Cn-0003Lh-Qy for larch@yhetil.org; Thu, 16 Jun 2022 21:55:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35006) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o21CQ-0003LU-JS for guix-patches@gnu.org; Thu, 16 Jun 2022 21:55:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:49516) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1o21CQ-0001C6-A7 for guix-patches@gnu.org; Thu, 16 Jun 2022 21:55:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1o21CQ-00057k-3l for guix-patches@gnu.org; Thu, 16 Jun 2022 21:55:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#55231] [PATCH v2 1/2] Allows copying of out-of-tree modules to the Linux initrd. References: <87wnf3pv87.fsf@ditto.jhoto.spork.org> In-Reply-To: <87wnf3pv87.fsf@ditto.jhoto.spork.org> Resent-From: Brian Cully Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 17 Jun 2022 01:55:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 55231 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 55231@debbugs.gnu.org Cc: Brian Cully Received: via spool by 55231-submit@debbugs.gnu.org id=B55231.165543087219655 (code B ref 55231); Fri, 17 Jun 2022 01:55:02 +0000 Received: (at 55231) by debbugs.gnu.org; 17 Jun 2022 01:54:32 +0000 Received: from localhost ([127.0.0.1]:43413 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o21Bv-00056x-MQ for submit@debbugs.gnu.org; Thu, 16 Jun 2022 21:54:32 -0400 Received: from coleridge.kublai.com ([166.84.7.167]:61636 helo=mail.spork.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o21Bt-00056o-E6 for 55231@debbugs.gnu.org; Thu, 16 Jun 2022 21:54:30 -0400 Received: from psyduck.jhoto.kublai.com (ool-18b8e9e7.dyn.optonline.net [24.184.233.231]) by mail.spork.org (Postfix) with ESMTPSA id D30182ED5; Thu, 16 Jun 2022 21:54:28 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=spork.org; s=dkim; t=1655430868; bh=bAGDRFoQXaijSpIrQ2E9pmzYBZAM01wSer3hcq0o/dU=; h=From:To:Cc:Subject:Date; b=oP8YnXBFILby+6wwOxtQhnW3mc5I/ERWxkt430P10UoHLzJhrTe2iE7gGD7/0CI8J nzDLCfF9p4wjZaq/pl/6gZDLXy9WeRlU1mNAnEO0gloHLkkWzYQi9w22Omon6jDAJ3 rRIUovAqV73x1fKmtKd9VtS91AXDIfxHI8wOOaFA= Date: Thu, 16 Jun 2022 21:51:59 -0400 Message-Id: <88d91cf303fd82c3667149ee8a647527f44bf571.1655430718.git.bjc@spork.org> X-Mailer: git-send-email 2.36.1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" Reply-to: Brian Cully X-ACL-Warn: , Brian Cully via Guix-patches From: Brian Cully via Guix-patches via X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1655430927; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=mP++96zcZdC8gBU3f2hkojejMDB0FrQpqOsf7rdsBrE=; b=nWnH2aLiKtpH6noH1+q48KMs3k5mdy0MvvaPzobDGDkPHqFoNL1nqUGPrnxSHkcuf1yjaO SeSjf+7Mm5k+obZ6mymKnKSZAGu1zeEVWmZoaku1ieZboSn/TyZkz9EXDgKlThRCrmHsld aK653f79LwJDKZgv6gTqCPbO+jhNkLwY1XJJwcEsYW4qWsK0q9tR+C56lKplVi9gB0/2UW 1rJmCyBjNGyRo5BK+zxUj0jPR0NeJ6wcnIxIQbps7OWYpIGdkaApuL0mPrbJmhuqBsSUPJ /5+ifEYaRusJVlkbRVueHphCLCbinevq+gGwDWYvf3e6Fy9xXArWGJcCjKvGyQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1655430927; a=rsa-sha256; cv=none; b=BoqdVhfLLeEFPTRf2nwmY6e8Flo8Jx+UdkvEZbDgzxXeQL+uxzjCpMX9mwOuczsVT9LuRu sj67Cv/QzofCUYV7ECeLvnmHbnyNAB4kVK975sA0m4ab0+uWxYKaFz4MCKOewfNiB2I2wq BBigd/WVRmGdAK6O+Rmo/FdXwO/SbuFTFapAWCJKA0XaQYCjdKKMuN9qf7qQLfRf3AN7l1 cRJxt5TZdB02L6+MQd79NI1OLr+tYMbHahLrBsxMCITPZmEFz67gdEFGL+FOQ5LabOvmG9 F9J/BlmWEeIoI/Rgt4f7zSY1Wh1HhP3Ui1sa4rH4o6JTvJRXGpct5hmPuaHNGA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=spork.org header.s=dkim header.b=oP8YnXBF; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -2.49 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=spork.org header.s=dkim header.b=oP8YnXBF; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 1A1AC3E6C4 X-Spam-Score: -2.49 X-Migadu-Scanner: scn0.migadu.com X-TUID: 6qvHu6OlIy/u With this patch, modules for ‘initrd-modules’ will not only be searched for in the in-tree Linux modules, but also any additional modules specified in ‘kernel-loadable-modules’. * gnu/build/linux-modules.scm (find-module-file): change DIRECTORY argument to DIRECTORIES. Now takes a list of directories to search, rather than a single one. * gnu/system/linux-initrd.scm (flat-linux-module-directory): change LINUX argument to PACKAGES. Now contains a list of file-likes to search for modules. (raw-initrd): Add LINUX-EXTRA-MODULE-PATHS keyword argument. Pass it to (flat-linux-extra-module-paths) along with the selected LINUX package. (base-initrd): Add LINUX-EXTRA-MODULE-PATHS keyword argument. Pass it to (raw-initrd). * gnu/system.scm (operating-system-initrd-file): pass in operating system definition's kernel-loadable-modules into (make-initrd) as LINUX-EXTRA-MODULE-PATHS. --- I've removed the new operating-system slot in preference to re-using kernel-loadable-modules, as discussed. I did leave the old argument names for the lower level routines in place, as I feel that they more accurately represent how the data are being used. I've also pulled out the documentation. Once this patch set is beaten into acceptability, the documentation can be adjusted if that's deemed necessary. gnu/build/linux-modules.scm | 19 ++++++++------ gnu/system.scm | 2 ++ gnu/system/linux-initrd.scm | 52 +++++++++++++++++++++++++------------ 3 files changed, 48 insertions(+), 25 deletions(-) diff --git a/gnu/build/linux-modules.scm b/gnu/build/linux-modules.scm index 053720574b..97b7e429ea 100644 --- a/gnu/build/linux-modules.scm +++ b/gnu/build/linux-modules.scm @@ -225,8 +225,8 @@ (define (file-name->module-name file) '.ko[.gz|.xz]' and normalizing it." (normalize-module-name (strip-extension (basename file)))) -(define (find-module-file directory module) - "Lookup module NAME under DIRECTORY, and return its absolute file name. +(define (find-module-file directories module) + "Lookup module NAME under DIRECTORIES, and return its absolute file name. NAME can be a file name with or without '.ko', or it can be a module name. Raise an error if it could not be found. @@ -247,16 +247,19 @@ (define (find-module-file directory module) (else chr))) module)))) - (match (find-files directory - (lambda (file stat) - (member (strip-extension - (basename file)) names))) + (match (append-map (lambda (directory) + (find-files directory + (lambda (file _stat) + (member (strip-extension + (basename file)) + names)))) + directories) ((file) file) (() - (error "kernel module not found" module directory)) + (error "kernel module not found" module directories)) ((_ ...) - (error "several modules by that name" module directory)))) + (error "several modules by that name" module directories)))) (define* (recursive-module-dependencies files #:key (lookup-module dot-ko)) diff --git a/gnu/system.scm b/gnu/system.scm index ba1b7b5152..6f52377c8d 100644 --- a/gnu/system.scm +++ b/gnu/system.scm @@ -1313,6 +1313,8 @@ (define (operating-system-initrd-file os) #:linux (operating-system-kernel os) #:linux-modules (operating-system-initrd-modules os) + #:linux-extra-module-paths + (operating-system-kernel-loadable-modules os) #:mapped-devices mapped-devices #:keyboard-layout (operating-system-keyboard-layout os))) diff --git a/gnu/system/linux-initrd.scm b/gnu/system/linux-initrd.scm index 4c4c78e444..50a182d7d5 100644 --- a/gnu/system/linux-initrd.scm +++ b/gnu/system/linux-initrd.scm @@ -119,13 +119,23 @@ (define* (expression->initrd exp `(#:references-graphs (("closure" ,init)))) "/initrd.cpio.gz")) -(define (flat-linux-module-directory linux modules) +(define (flat-linux-module-directory packages modules) "Return a flat directory containing the Linux kernel modules listed in -MODULES and taken from LINUX." +MODULES and taken from PACKAGES." (define imported-modules (source-module-closure '((gnu build linux-modules) (guix build utils)))) + (define (package+out->input package out) + (gexp-input package out)) + + (define package-inputs + (map (lambda (p) + (match p + ((p o) (package+out->input p o)) + (p (package+out->input p "out")))) + packages)) + (define build-exp (with-imported-modules imported-modules (with-extensions (list guile-zlib) @@ -135,11 +145,12 @@ (define (flat-linux-module-directory linux modules) (srfi srfi-1) (srfi srfi-26)) - (define module-dir - (string-append #$linux "/lib/modules")) + (define module-dirs + (map (cut string-append <> "/lib/modules") + '#$package-inputs)) (define modules - (let* ((lookup (cut find-module-file module-dir <>)) + (let* ((lookup (cut find-module-file module-dirs <>)) (modules (map lookup '#$modules))) (append modules (recursive-module-dependencies @@ -172,20 +183,23 @@ (define* (raw-initrd file-systems #:key (linux linux-libre) (linux-modules '()) + (linux-extra-module-paths '()) (mapped-devices '()) (keyboard-layout #f) (helper-packages '()) qemu-networking? volatile-root? (on-error 'debug)) - "Return as a file-like object a raw initrd, with kernel -modules taken from LINUX. FILE-SYSTEMS is a list of file-systems to be -mounted by the initrd, possibly in addition to the root file system specified -on the kernel command line via 'root'. LINUX-MODULES is a list of kernel -modules to be loaded at boot time. MAPPED-DEVICES is a list of device -mappings to realize before FILE-SYSTEMS are mounted. -HELPER-PACKAGES is a list of packages to be copied in the initrd. It may include -e2fsck/static or other packages needed by the initrd to check root partition. + "Return as a file-like object a raw initrd, with kernel modules taken from +LINUX. FILE-SYSTEMS is a list of file-systems to be mounted by the initrd, +possibly in addition to the root file system specified on the kernel command +line via 'root'. LINUX-MODULES is a list of kernel modules to be loaded at +boot time. LINUX-EXTRA-MODULE-PATHS is a list of file-like objects which will +be searched for modules in addition to the linux kernel. MAPPED-DEVICES is a +list of device mappings to realize before FILE-SYSTEMS are mounted. +HELPER-PACKAGES is a list of packages to be copied in the initrd. It may +include e2fsck/static or other packages needed by the initrd to check root +partition. When true, KEYBOARD-LAYOUT is a record denoting the desired console keyboard layout. This is done before MAPPED-DEVICES are set up and @@ -221,7 +235,8 @@ (define* (raw-initrd file-systems #~()))) (define kodir - (flat-linux-module-directory linux linux-modules)) + (flat-linux-module-directory (cons linux linux-extra-module-paths) + linux-modules)) (expression->initrd (with-imported-modules (source-module-closure @@ -366,6 +381,7 @@ (define* (base-initrd file-systems #:key (linux linux-libre) (linux-modules '()) + (linux-extra-module-paths '()) (mapped-devices '()) (keyboard-layout #f) qemu-networking? @@ -386,9 +402,10 @@ (define* (base-initrd file-systems QEMU-NETWORKING? and VOLATILE-ROOT? behaves as in raw-initrd. The initrd is automatically populated with all the kernel modules necessary -for FILE-SYSTEMS and for the given options. Additional kernel -modules can be listed in LINUX-MODULES. They will be added to the initrd, and -loaded at boot time in the order in which they appear." +for FILE-SYSTEMS and for the given options. Additional kernel modules can be +listed in LINUX-MODULES. Additional search paths for modules can be listed in +LINUX-EXTRA-MODULE-PATHS. They will be added to the initrd, and loaded at +boot time in the order in which they appear." (define linux-modules* ;; Modules added to the initrd and loaded from the initrd. `(,@linux-modules @@ -408,6 +425,7 @@ (define* (base-initrd file-systems (raw-initrd file-systems #:linux linux #:linux-modules linux-modules* + #:linux-extra-module-paths linux-extra-module-paths #:mapped-devices mapped-devices #:helper-packages helper-packages #:keyboard-layout keyboard-layout -- 2.36.1