From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id OCYjEAM6iWIq4gAAbAwnHQ (envelope-from ) for ; Sat, 21 May 2022 21:14:11 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id wG8gDwM6iWKbXQAAG6o9tA (envelope-from ) for ; Sat, 21 May 2022 21:14:11 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 328A43A942 for ; Sat, 21 May 2022 21:14:10 +0200 (CEST) Received: from localhost ([::1]:50984 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nsUYC-0003yu-TS for larch@yhetil.org; Sat, 21 May 2022 15:14:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48228) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nsUY8-0003ym-3U for guix-patches@gnu.org; Sat, 21 May 2022 15:14:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:49079) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nsUY6-0003vA-Ez for guix-patches@gnu.org; Sat, 21 May 2022 15:14:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nsUY6-0002lJ-AP for guix-patches@gnu.org; Sat, 21 May 2022 15:14:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#55231] [PATCH v2] initrd: Allow extra search paths with =?UTF-8?Q?=E2=80=98initrd-extra-module-paths=E2=80=99?= References: <87wnf3pv87.fsf@ditto.jhoto.spork.org> In-Reply-To: <87wnf3pv87.fsf@ditto.jhoto.spork.org> Resent-From: Brian Cully Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 21 May 2022 19:14:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 55231 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 55231@debbugs.gnu.org Cc: Brian Cully Received: via spool by 55231-submit@debbugs.gnu.org id=B55231.165316039710553 (code B ref 55231); Sat, 21 May 2022 19:14:02 +0000 Received: (at 55231) by debbugs.gnu.org; 21 May 2022 19:13:17 +0000 Received: from localhost ([127.0.0.1]:42976 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nsUXM-0002k8-DT for submit@debbugs.gnu.org; Sat, 21 May 2022 15:13:17 -0400 Received: from coleridge.kublai.com ([166.84.7.167]:57942 helo=mail.spork.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nsUXK-0002k0-7W for 55231@debbugs.gnu.org; Sat, 21 May 2022 15:13:15 -0400 Received: from guix.jhoto.kublai.com (unknown [IPv6:2001:470:1f07:1b9:c714:ed8b:7cf4:a700]) by mail.spork.org (Postfix) with ESMTPSA id 0CBD470CA; Sat, 21 May 2022 15:13:14 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=spork.org; s=dkim; t=1653160394; bh=lye+eu/C0xitMixOX5/Vcr/gOROF3AtimFdB3XpxwBQ=; h=From:To:Cc:Subject:Date; b=EQaIDMHaHNHCBgRat23MBgGBIUWTE3pOmwJOaaGN34YKQSUaIPVao54MejZrw9M2h +AwtIcKg2zg6Ka4PwXcMR0ZHrhlG3VzIGPMknElfI4CXq6anA8uqvQsnR73PIhSJGS y22pe8MOaSjEvywbBmnMva96qVnwAJLD+uKA7etM= Date: Sat, 21 May 2022 15:12:45 -0400 Message-Id: <2053c4ab42dfe2719cfc377934ac2fb9bcb500a9.1653160364.git.bjc@spork.org> X-Mailer: git-send-email 2.36.1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" Reply-to: Brian Cully X-ACL-Warn: , Brian Cully via Guix-patches From: Brian Cully via Guix-patches via X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1653160450; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=jpr4U2cJFYYp52H4MrJwwRTMS5TlDsrpXBO/7u5OD5s=; b=gPiXI1aTDQ64wIk7o1x3hQyg29iiaTQoNB1mCNfYHDRJZGzzShE2wKQ343mhUHLotIwgKB w2AM3Sdbsd9gd3fNU7PtR3xtQAF6zF2uhA7z5J2Leu1+0jqEILTCTh7fDWjiSJfjnDqJPy K/wXRBL2j66+UeBUfcHyvvOeeB0l5QBviFgiwTgjRMhcAUs+Q9r6BozP7tq3jvdkhmwwIa NH4QGFPLKpd0Cy60XaSFO7iacJkUzxpeZk8wqd5YSgD9S2oznnQEWGmLrNSYNRTpDgui9n y4Rt3GRYK2Lo1+mPvY14Lm3xBdHZaOfn3YAuzSzGx7eMOx9XWVBi992G70KUTg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1653160450; a=rsa-sha256; cv=none; b=SWTaXEOjO0tIOr3A8suiqU/v+ZDUJ/C/Qxm5mJYvkcYkCOSus03EsML76XP5AuOeJNS2ce K1Ta+YFZ1cBRG03sl7vZvdJoQwv//4TkWjXMKLDEnby2kvRjcmUSt+rK3UXaTOBKHq5Z/1 sO9RBbkA9BH0vIYPvs3QTmCT2gYBQtXELRA++CA4ppq2hbzOyBcJFY6lg5aT60m7+rYQvt r8g/alsv47AbaLUBH+4N2qs7gs1szmZ2k+ml8ImJ0d6LmzjXhm1eQiR38JmoeuPVL2Hs7W ifYKrfOr/OceloGbZxBg5yU+zcWA0xeS1+GTQvQt616XJbzF9WAu8MzOwr5FXg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=spork.org header.s=dkim header.b=EQaIDMHa; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -3.55 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=spork.org header.s=dkim header.b=EQaIDMHa; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 328A43A942 X-Spam-Score: -3.55 X-Migadu-Scanner: scn1.migadu.com X-TUID: i4i0Yn14L7nQ From: Brian Cully This patch allows copying of out-of-tree kernel modules to the Linux initrd. For out-of-tree modules to found, an extra slot has been added to ‘operating-system’: ‘initrd-extra-module-paths’, which contains a list containing items of either FILE-LIKE or (FILE-LIKE OUTPUT) which will be searched for modules in addition to the standard Linux module path. The required modules can then be added to ‘initrd-modules’ as normal and all paths will be searched for it, including for any modules depended on. * doc: Update documentation for ‘initrd-extra-module-paths’. * gnu/build/linux-modules.scm (find-module-file): change DIRECTORY argument to DIRECTORIES. Now takes a list of directories to search, rather than a single one. * gnu/system.scm : Add INITRD-EXTRA-MODULE-PATHS field and accessor. Takes a list of file-like objects. * gnu/system/linux-initrd.scm (flat-linux-module-directory): change LINUX argument to PACKAGES. Now contains a list of file-likes to search for modules. (raw-initrd): Add LINUX-EXTRA-MODULE-PATHS keyword argument. Pass it to (flat-linux-extra-module-paths) along with the selected LINUX package. (base-initrd): Add LINUX-EXTRA-MODULE-PATHS keyword argument. Pass it to (raw-initrd). --- This version includes both the previous patch and the documentation. doc/guix.texi | 42 ++++++++++++++++++++++++------ gnu/build/linux-modules.scm | 19 ++++++++------ gnu/system.scm | 5 ++++ gnu/system/linux-initrd.scm | 52 +++++++++++++++++++++++++------------ 4 files changed, 85 insertions(+), 33 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index faa35060ef..9fd45ea209 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -15490,6 +15490,16 @@ operating-system Reference The list of Linux kernel modules that need to be available in the initial RAM disk. @xref{Initial RAM Disk}. +@item @code{initrd-extra-module-paths} (default: @code{'()}) +@cindex initrd +@cindex initial RAM disk +A list of paths outside of the Linux kernel tree to search for Linux +kernel modules. + +The items in this may be either file-like objects (usually packages), or +a list where the first element is a package and the second is an +output--e.g. @code{(list (list zfs "module"))}. + @item @code{initrd} (default: @code{base-initrd}) A procedure that returns an initial RAM disk for the Linux kernel. This field is provided to support low-level customization and @@ -35816,6 +35826,19 @@ Initial RAM Disk (initrd-modules (cons "megaraid_sas" %base-initrd-modules))) @end lisp +If a module listed in @code{initrd-modules} is not included in the +Linux-libre kernel, then the location to it must be added to the +@code{initrd-extra-module-paths} list. For example, if your root file +system exists on a ZFS pool, then your configuration might look like the +following: + +@lisp +(operating-system + ;; @dots{} + (initrd-modules (cons "zfs" %base-initrd-modules)) + (initrd-extra-module-paths (list (list zfs "module")))) +@end lisp + @defvr {Scheme Variable} %base-initrd-modules This is the list of kernel modules included in the initrd by default. @end defvr @@ -35929,13 +35952,15 @@ Initial RAM Disk @cindex initrd @cindex initial RAM disk @deffn {Scheme Procedure} raw-initrd @var{file-systems} @ - [#:linux-modules '()] [#:mapped-devices '()] @ - [#:keyboard-layout #f] @ + [#:linux-modules '()] [#:linux-extra-module-paths '()] @ + [#:mapped-devices '()] [#:keyboard-layout #f] @ [#:helper-packages '()] [#:qemu-networking? #f] [#:volatile-root? #f] Return a derivation that builds a raw initrd. @var{file-systems} is a list of file systems to be mounted by the initrd, possibly in addition to the root file system specified on the kernel command line via @option{root}. @var{linux-modules} is a list of kernel modules to be loaded at boot time. +@var{linux-extra-module-paths} is a list of file-like objects to be searched +for kernel modules. @var{mapped-devices} is a list of device mappings to realize before @var{file-systems} are mounted (@pxref{Mapped Devices}). @var{helper-packages} is a list of packages to be copied in the initrd. @@ -35960,12 +35985,13 @@ Initial RAM Disk @deffn {Scheme Procedure} base-initrd @var{file-systems} @ [#:mapped-devices '()] [#:keyboard-layout #f] @ [#:qemu-networking? #f] [#:volatile-root? #f] @ - [#:linux-modules '()] -Return as a file-like object a generic initrd, with kernel -modules taken from @var{linux}. @var{file-systems} is a list of file-systems to be -mounted by the initrd, possibly in addition to the root file system specified -on the kernel command line via @option{root}. @var{mapped-devices} is a list of device -mappings to realize before @var{file-systems} are mounted. + [#:linux-modules '()] [#:linux-extra-module-paths '()] +Return as a file-like object a generic initrd, with kernel modules taken +from @var{linux} and @var{linux-extra-module-paths}. @var{file-systems} +is a list of file-systems to be mounted by the initrd, possibly in +addition to the root file system specified on the kernel command line +via @option{root}. @var{mapped-devices} is a list of device mappings to +realize before @var{file-systems} are mounted. When true, @var{keyboard-layout} is a @code{} record denoting the desired console keyboard layout. This is done before @var{mapped-devices} diff --git a/gnu/build/linux-modules.scm b/gnu/build/linux-modules.scm index 053720574b..97b7e429ea 100644 --- a/gnu/build/linux-modules.scm +++ b/gnu/build/linux-modules.scm @@ -225,8 +225,8 @@ (define (file-name->module-name file) '.ko[.gz|.xz]' and normalizing it." (normalize-module-name (strip-extension (basename file)))) -(define (find-module-file directory module) - "Lookup module NAME under DIRECTORY, and return its absolute file name. +(define (find-module-file directories module) + "Lookup module NAME under DIRECTORIES, and return its absolute file name. NAME can be a file name with or without '.ko', or it can be a module name. Raise an error if it could not be found. @@ -247,16 +247,19 @@ (define (find-module-file directory module) (else chr))) module)))) - (match (find-files directory - (lambda (file stat) - (member (strip-extension - (basename file)) names))) + (match (append-map (lambda (directory) + (find-files directory + (lambda (file _stat) + (member (strip-extension + (basename file)) + names)))) + directories) ((file) file) (() - (error "kernel module not found" module directory)) + (error "kernel module not found" module directories)) ((_ ...) - (error "several modules by that name" module directory)))) + (error "several modules by that name" module directories)))) (define* (recursive-module-dependencies files #:key (lookup-module dot-ko)) diff --git a/gnu/system.scm b/gnu/system.scm index ba3a1865d7..6c712e5cea 100644 --- a/gnu/system.scm +++ b/gnu/system.scm @@ -107,6 +107,7 @@ (define-module (gnu system) operating-system-label operating-system-default-label operating-system-initrd-modules + operating-system-initrd-extra-module-paths operating-system-initrd operating-system-users operating-system-groups @@ -236,6 +237,8 @@ (define-record-type* operating-system (initrd-modules operating-system-initrd-modules ; list of strings (thunked) ; it's system-dependent (default %base-initrd-modules)) + (initrd-extra-module-paths operating-system-initrd-extra-module-paths ; list of file-likes + (default '())) (firmware operating-system-firmware ; list of packages (default %base-firmware)) @@ -1312,6 +1315,8 @@ (define (operating-system-initrd-file os) #:linux (operating-system-kernel os) #:linux-modules (operating-system-initrd-modules os) + #:linux-extra-module-paths + (operating-system-initrd-extra-module-paths os) #:mapped-devices mapped-devices #:keyboard-layout (operating-system-keyboard-layout os))) diff --git a/gnu/system/linux-initrd.scm b/gnu/system/linux-initrd.scm index 4c4c78e444..50a182d7d5 100644 --- a/gnu/system/linux-initrd.scm +++ b/gnu/system/linux-initrd.scm @@ -119,13 +119,23 @@ (define* (expression->initrd exp `(#:references-graphs (("closure" ,init)))) "/initrd.cpio.gz")) -(define (flat-linux-module-directory linux modules) +(define (flat-linux-module-directory packages modules) "Return a flat directory containing the Linux kernel modules listed in -MODULES and taken from LINUX." +MODULES and taken from PACKAGES." (define imported-modules (source-module-closure '((gnu build linux-modules) (guix build utils)))) + (define (package+out->input package out) + (gexp-input package out)) + + (define package-inputs + (map (lambda (p) + (match p + ((p o) (package+out->input p o)) + (p (package+out->input p "out")))) + packages)) + (define build-exp (with-imported-modules imported-modules (with-extensions (list guile-zlib) @@ -135,11 +145,12 @@ (define (flat-linux-module-directory linux modules) (srfi srfi-1) (srfi srfi-26)) - (define module-dir - (string-append #$linux "/lib/modules")) + (define module-dirs + (map (cut string-append <> "/lib/modules") + '#$package-inputs)) (define modules - (let* ((lookup (cut find-module-file module-dir <>)) + (let* ((lookup (cut find-module-file module-dirs <>)) (modules (map lookup '#$modules))) (append modules (recursive-module-dependencies @@ -172,20 +183,23 @@ (define* (raw-initrd file-systems #:key (linux linux-libre) (linux-modules '()) + (linux-extra-module-paths '()) (mapped-devices '()) (keyboard-layout #f) (helper-packages '()) qemu-networking? volatile-root? (on-error 'debug)) - "Return as a file-like object a raw initrd, with kernel -modules taken from LINUX. FILE-SYSTEMS is a list of file-systems to be -mounted by the initrd, possibly in addition to the root file system specified -on the kernel command line via 'root'. LINUX-MODULES is a list of kernel -modules to be loaded at boot time. MAPPED-DEVICES is a list of device -mappings to realize before FILE-SYSTEMS are mounted. -HELPER-PACKAGES is a list of packages to be copied in the initrd. It may include -e2fsck/static or other packages needed by the initrd to check root partition. + "Return as a file-like object a raw initrd, with kernel modules taken from +LINUX. FILE-SYSTEMS is a list of file-systems to be mounted by the initrd, +possibly in addition to the root file system specified on the kernel command +line via 'root'. LINUX-MODULES is a list of kernel modules to be loaded at +boot time. LINUX-EXTRA-MODULE-PATHS is a list of file-like objects which will +be searched for modules in addition to the linux kernel. MAPPED-DEVICES is a +list of device mappings to realize before FILE-SYSTEMS are mounted. +HELPER-PACKAGES is a list of packages to be copied in the initrd. It may +include e2fsck/static or other packages needed by the initrd to check root +partition. When true, KEYBOARD-LAYOUT is a record denoting the desired console keyboard layout. This is done before MAPPED-DEVICES are set up and @@ -221,7 +235,8 @@ (define* (raw-initrd file-systems #~()))) (define kodir - (flat-linux-module-directory linux linux-modules)) + (flat-linux-module-directory (cons linux linux-extra-module-paths) + linux-modules)) (expression->initrd (with-imported-modules (source-module-closure @@ -366,6 +381,7 @@ (define* (base-initrd file-systems #:key (linux linux-libre) (linux-modules '()) + (linux-extra-module-paths '()) (mapped-devices '()) (keyboard-layout #f) qemu-networking? @@ -386,9 +402,10 @@ (define* (base-initrd file-systems QEMU-NETWORKING? and VOLATILE-ROOT? behaves as in raw-initrd. The initrd is automatically populated with all the kernel modules necessary -for FILE-SYSTEMS and for the given options. Additional kernel -modules can be listed in LINUX-MODULES. They will be added to the initrd, and -loaded at boot time in the order in which they appear." +for FILE-SYSTEMS and for the given options. Additional kernel modules can be +listed in LINUX-MODULES. Additional search paths for modules can be listed in +LINUX-EXTRA-MODULE-PATHS. They will be added to the initrd, and loaded at +boot time in the order in which they appear." (define linux-modules* ;; Modules added to the initrd and loaded from the initrd. `(,@linux-modules @@ -408,6 +425,7 @@ (define* (base-initrd file-systems (raw-initrd file-systems #:linux linux #:linux-modules linux-modules* + #:linux-extra-module-paths linux-extra-module-paths #:mapped-devices mapped-devices #:helper-packages helper-packages #:keyboard-layout keyboard-layout -- 2.36.1