unofficial mirror of guix-patches@gnu.org 
 help / color / mirror / code / Atom feed
From: Brian Cully via Guix-patches via <guix-patches@gnu.org>
To: 55231@debbugs.gnu.org
Cc: Brian Cully <bjc@kublai.com>
Subject: [bug#55231] [PATCH v2] initrd: Allow extra search paths with ‘initrd-extra-module-paths’
Date: Sat, 21 May 2022 15:12:45 -0400	[thread overview]
Message-ID: <2053c4ab42dfe2719cfc377934ac2fb9bcb500a9.1653160364.git.bjc@spork.org> (raw)
In-Reply-To: <87wnf3pv87.fsf@ditto.jhoto.spork.org>

From: Brian Cully <bjc@kublai.com>

This patch allows copying of out-of-tree kernel modules to the Linux
initrd.

For out-of-tree modules to found, an extra slot has been added to
‘operating-system’: ‘initrd-extra-module-paths’, which contains a list
containing items of either FILE-LIKE or (FILE-LIKE OUTPUT) which will be
searched for modules in addition to the standard Linux module path. The
required modules can then be added to ‘initrd-modules’ as normal and all paths
will be searched for it, including for any modules depended on.

* doc: Update documentation for ‘initrd-extra-module-paths’.
* gnu/build/linux-modules.scm (find-module-file): change DIRECTORY argument to
DIRECTORIES. Now takes a list of directories to search, rather than a single
one.
* gnu/system.scm <operating-system>: Add INITRD-EXTRA-MODULE-PATHS
field and accessor. Takes a list of file-like objects.
* gnu/system/linux-initrd.scm (flat-linux-module-directory): change LINUX
argument to PACKAGES. Now contains a list of file-likes to search for modules.
(raw-initrd): Add LINUX-EXTRA-MODULE-PATHS keyword argument. Pass it
to (flat-linux-extra-module-paths) along with the selected LINUX package.
(base-initrd): Add LINUX-EXTRA-MODULE-PATHS keyword argument. Pass it
to (raw-initrd).
---
This version includes both the previous patch and the documentation.

 doc/guix.texi               | 42 ++++++++++++++++++++++++------
 gnu/build/linux-modules.scm | 19 ++++++++------
 gnu/system.scm              |  5 ++++
 gnu/system/linux-initrd.scm | 52 +++++++++++++++++++++++++------------
 4 files changed, 85 insertions(+), 33 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index faa35060ef..9fd45ea209 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -15490,6 +15490,16 @@ operating-system Reference
 The list of Linux kernel modules that need to be available in the
 initial RAM disk.  @xref{Initial RAM Disk}.
 
+@item @code{initrd-extra-module-paths} (default: @code{'()})
+@cindex initrd
+@cindex initial RAM disk
+A list of paths outside of the Linux kernel tree to search for Linux
+kernel modules.
+
+The items in this may be either file-like objects (usually packages), or
+a list where the first element is a package and the second is an
+output--e.g. @code{(list (list zfs "module"))}.
+
 @item @code{initrd} (default: @code{base-initrd})
 A procedure that returns an initial RAM disk for the Linux
 kernel.  This field is provided to support low-level customization and
@@ -35816,6 +35826,19 @@ Initial RAM Disk
   (initrd-modules (cons "megaraid_sas" %base-initrd-modules)))
 @end lisp
 
+If a module listed in @code{initrd-modules} is not included in the
+Linux-libre kernel, then the location to it must be added to the
+@code{initrd-extra-module-paths} list.  For example, if your root file
+system exists on a ZFS pool, then your configuration might look like the
+following:
+
+@lisp
+(operating-system
+  ;; @dots{}
+  (initrd-modules (cons "zfs" %base-initrd-modules))
+  (initrd-extra-module-paths (list (list zfs "module"))))
+@end lisp
+
 @defvr {Scheme Variable} %base-initrd-modules
 This is the list of kernel modules included in the initrd by default.
 @end defvr
@@ -35929,13 +35952,15 @@ Initial RAM Disk
 @cindex initrd
 @cindex initial RAM disk
 @deffn {Scheme Procedure} raw-initrd @var{file-systems} @
-       [#:linux-modules '()] [#:mapped-devices '()] @
-       [#:keyboard-layout #f] @
+       [#:linux-modules '()] [#:linux-extra-module-paths '()] @
+       [#:mapped-devices '()] [#:keyboard-layout #f] @
        [#:helper-packages '()] [#:qemu-networking? #f] [#:volatile-root? #f]
 Return a derivation that builds a raw initrd.  @var{file-systems} is
 a list of file systems to be mounted by the initrd, possibly in addition to
 the root file system specified on the kernel command line via @option{root}.
 @var{linux-modules} is a list of kernel modules to be loaded at boot time.
+@var{linux-extra-module-paths} is a list of file-like objects to be searched
+for kernel modules.
 @var{mapped-devices} is a list of device mappings to realize before
 @var{file-systems} are mounted (@pxref{Mapped Devices}).
 @var{helper-packages} is a list of packages to be copied in the initrd.
@@ -35960,12 +35985,13 @@ Initial RAM Disk
 @deffn {Scheme Procedure} base-initrd @var{file-systems} @
        [#:mapped-devices '()] [#:keyboard-layout #f] @
        [#:qemu-networking? #f] [#:volatile-root? #f] @
-       [#:linux-modules '()]
-Return as a file-like object a generic initrd, with kernel
-modules taken from @var{linux}.  @var{file-systems} is a list of file-systems to be
-mounted by the initrd, possibly in addition to the root file system specified
-on the kernel command line via @option{root}.  @var{mapped-devices} is a list of device
-mappings to realize before @var{file-systems} are mounted.
+       [#:linux-modules '()] [#:linux-extra-module-paths '()]
+Return as a file-like object a generic initrd, with kernel modules taken
+from @var{linux} and @var{linux-extra-module-paths}.  @var{file-systems}
+is a list of file-systems to be mounted by the initrd, possibly in
+addition to the root file system specified on the kernel command line
+via @option{root}.  @var{mapped-devices} is a list of device mappings to
+realize before @var{file-systems} are mounted.
 
 When true, @var{keyboard-layout} is a @code{<keyboard-layout>} record denoting
 the desired console keyboard layout.  This is done before @var{mapped-devices}
diff --git a/gnu/build/linux-modules.scm b/gnu/build/linux-modules.scm
index 053720574b..97b7e429ea 100644
--- a/gnu/build/linux-modules.scm
+++ b/gnu/build/linux-modules.scm
@@ -225,8 +225,8 @@ (define (file-name->module-name file)
 '.ko[.gz|.xz]' and normalizing it."
   (normalize-module-name (strip-extension (basename file))))
 
-(define (find-module-file directory module)
-  "Lookup module NAME under DIRECTORY, and return its absolute file name.
+(define (find-module-file directories module)
+  "Lookup module NAME under DIRECTORIES, and return its absolute file name.
 NAME can be a file name with or without '.ko', or it can be a module name.
 Raise an error if it could not be found.
 
@@ -247,16 +247,19 @@ (define (find-module-file directory module)
                            (else chr)))
                        module))))
 
-  (match (find-files directory
-                     (lambda (file stat)
-                       (member (strip-extension
-                                (basename file)) names)))
+  (match (append-map (lambda (directory)
+                       (find-files directory
+                                   (lambda (file _stat)
+                                     (member (strip-extension
+                                              (basename file))
+                                             names))))
+                       directories)
     ((file)
      file)
     (()
-     (error "kernel module not found" module directory))
+     (error "kernel module not found" module directories))
     ((_ ...)
-     (error "several modules by that name" module directory))))
+     (error "several modules by that name" module directories))))
 
 (define* (recursive-module-dependencies files
                                         #:key (lookup-module dot-ko))
diff --git a/gnu/system.scm b/gnu/system.scm
index ba3a1865d7..6c712e5cea 100644
--- a/gnu/system.scm
+++ b/gnu/system.scm
@@ -107,6 +107,7 @@ (define-module (gnu system)
             operating-system-label
             operating-system-default-label
             operating-system-initrd-modules
+            operating-system-initrd-extra-module-paths
             operating-system-initrd
             operating-system-users
             operating-system-groups
@@ -236,6 +237,8 @@ (define-record-type* <operating-system> operating-system
   (initrd-modules operating-system-initrd-modules ; list of strings
                   (thunked)                       ; it's system-dependent
                   (default %base-initrd-modules))
+  (initrd-extra-module-paths operating-system-initrd-extra-module-paths ; list of file-likes
+                             (default '()))
 
   (firmware operating-system-firmware             ; list of packages
             (default %base-firmware))
@@ -1312,6 +1315,8 @@ (define (operating-system-initrd-file os)
                #:linux (operating-system-kernel os)
                #:linux-modules
                (operating-system-initrd-modules os)
+               #:linux-extra-module-paths
+               (operating-system-initrd-extra-module-paths os)
                #:mapped-devices mapped-devices
                #:keyboard-layout (operating-system-keyboard-layout os)))
 
diff --git a/gnu/system/linux-initrd.scm b/gnu/system/linux-initrd.scm
index 4c4c78e444..50a182d7d5 100644
--- a/gnu/system/linux-initrd.scm
+++ b/gnu/system/linux-initrd.scm
@@ -119,13 +119,23 @@ (define* (expression->initrd exp
                               `(#:references-graphs (("closure" ,init))))
                "/initrd.cpio.gz"))
 
-(define (flat-linux-module-directory linux modules)
+(define (flat-linux-module-directory packages modules)
   "Return a flat directory containing the Linux kernel modules listed in
-MODULES and taken from LINUX."
+MODULES and taken from PACKAGES."
   (define imported-modules
     (source-module-closure '((gnu build linux-modules)
                              (guix build utils))))
 
+  (define (package+out->input package out)
+    (gexp-input package out))
+
+  (define package-inputs
+    (map (lambda (p)
+           (match p
+             ((p o) (package+out->input p o))
+             (p     (package+out->input p "out"))))
+         packages))
+
   (define build-exp
     (with-imported-modules imported-modules
       (with-extensions (list guile-zlib)
@@ -135,11 +145,12 @@ (define (flat-linux-module-directory linux modules)
                          (srfi srfi-1)
                          (srfi srfi-26))
 
-            (define module-dir
-              (string-append #$linux "/lib/modules"))
+            (define module-dirs
+              (map (cut string-append <> "/lib/modules")
+                   '#$package-inputs))
 
             (define modules
-              (let* ((lookup  (cut find-module-file module-dir <>))
+              (let* ((lookup  (cut find-module-file module-dirs <>))
                      (modules (map lookup '#$modules)))
                 (append modules
                         (recursive-module-dependencies
@@ -172,20 +183,23 @@ (define* (raw-initrd file-systems
                       #:key
                       (linux linux-libre)
                       (linux-modules '())
+                      (linux-extra-module-paths '())
                       (mapped-devices '())
                       (keyboard-layout #f)
                       (helper-packages '())
                       qemu-networking?
                       volatile-root?
                       (on-error 'debug))
-  "Return as a file-like object a raw initrd, with kernel
-modules taken from LINUX.  FILE-SYSTEMS is a list of file-systems to be
-mounted by the initrd, possibly in addition to the root file system specified
-on the kernel command line via 'root'.  LINUX-MODULES is a list of kernel
-modules to be loaded at boot time. MAPPED-DEVICES is a list of device
-mappings to realize before FILE-SYSTEMS are mounted.
-HELPER-PACKAGES is a list of packages to be copied in the initrd. It may include
-e2fsck/static or other packages needed by the initrd to check root partition.
+  "Return as a file-like object a raw initrd, with kernel modules taken from
+LINUX.  FILE-SYSTEMS is a list of file-systems to be mounted by the initrd,
+possibly in addition to the root file system specified on the kernel command
+line via 'root'.  LINUX-MODULES is a list of kernel modules to be loaded at
+boot time. LINUX-EXTRA-MODULE-PATHS is a list of file-like objects which will
+be searched for modules in addition to the linux kernel. MAPPED-DEVICES is a
+list of device mappings to realize before FILE-SYSTEMS are mounted.
+HELPER-PACKAGES is a list of packages to be copied in the initrd. It may
+include e2fsck/static or other packages needed by the initrd to check root
+partition.
 
 When true, KEYBOARD-LAYOUT is a <keyboard-layout> record denoting the desired
 console keyboard layout.  This is done before MAPPED-DEVICES are set up and
@@ -221,7 +235,8 @@ (define* (raw-initrd file-systems
           #~())))
 
   (define kodir
-    (flat-linux-module-directory linux linux-modules))
+    (flat-linux-module-directory (cons linux linux-extra-module-paths)
+                                 linux-modules))
 
   (expression->initrd
    (with-imported-modules (source-module-closure
@@ -366,6 +381,7 @@ (define* (base-initrd file-systems
                       #:key
                       (linux linux-libre)
                       (linux-modules '())
+                      (linux-extra-module-paths '())
                       (mapped-devices '())
                       (keyboard-layout #f)
                       qemu-networking?
@@ -386,9 +402,10 @@ (define* (base-initrd file-systems
 QEMU-NETWORKING? and VOLATILE-ROOT? behaves as in raw-initrd.
 
 The initrd is automatically populated with all the kernel modules necessary
-for FILE-SYSTEMS and for the given options.  Additional kernel
-modules can be listed in LINUX-MODULES.  They will be added to the initrd, and
-loaded at boot time in the order in which they appear."
+for FILE-SYSTEMS and for the given options.  Additional kernel modules can be
+listed in LINUX-MODULES.  Additional search paths for modules can be listed in
+LINUX-EXTRA-MODULE-PATHS.  They will be added to the initrd, and loaded at
+boot time in the order in which they appear."
   (define linux-modules*
     ;; Modules added to the initrd and loaded from the initrd.
     `(,@linux-modules
@@ -408,6 +425,7 @@ (define* (base-initrd file-systems
   (raw-initrd file-systems
               #:linux linux
               #:linux-modules linux-modules*
+              #:linux-extra-module-paths linux-extra-module-paths
               #:mapped-devices mapped-devices
               #:helper-packages helper-packages
               #:keyboard-layout keyboard-layout
-- 
2.36.1





  parent reply	other threads:[~2022-05-21 19:14 UTC|newest]

Thread overview: 23+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-05-02 19:11 [bug#55231] [PATCH v1] initrd: Allow extra search paths with ‘initrd-extra-module-paths’ Brian Cully via Guix-patches via
2022-05-02 22:03 ` Maxime Devos
2022-05-02 22:53   ` Brian Cully via Guix-patches via
2022-05-13 19:46 ` Kaelyn via Guix-patches via
2022-05-13 20:25 ` Maxime Devos
2022-05-21 19:12 ` Brian Cully via Guix-patches via [this message]
2022-06-01 15:54   ` Ludovic Courtès
2022-06-02 20:40     ` Brian Cully via Guix-patches via
2022-06-03  7:27       ` Ludovic Courtès
2022-05-30 20:14 ` Kaelyn via Guix-patches via
2022-06-02 21:23 ` Kaelyn via Guix-patches via
2022-06-17  1:51 ` [bug#55231] [PATCH v2 1/2] Allows copying of out-of-tree modules to the Linux initrd Brian Cully via Guix-patches via
2022-06-17 20:34   ` [bug#55231] [PATCH v1] initrd: Allow extra search paths with ‘initrd-extra-module-paths’ Ludovic Courtès
2022-06-18 19:11 ` [bug#55231] [PATCH v3 1/2] Allows copying of out-of-tree modules to the Linux initrd Brian Cully via Guix-patches via
2022-06-18 19:11   ` [bug#55231] [PATCH v3 2/2] doc: ‘initrd-modules’ will search ‘kernel-loadable-modules’ Brian Cully via Guix-patches via
2022-06-18 20:34     ` Maxime Devos
2022-06-18 20:43       ` Brian Cully via Guix-patches via
2022-06-18 22:34         ` Maxime Devos
2022-06-18 23:11           ` Brian Cully via Guix-patches via
2022-06-19 12:05             ` Maxime Devos
2022-06-21 12:34 ` [bug#55231] [PATCH v1] initrd: Allow extra search paths with ‘initrd-extra-module-paths’ Kaelyn via Guix-patches via
2022-06-24 13:28 ` [bug#55231] [PATCH v4 1/2] Allows copying of out-of-tree modules to the Linux initrd Brian Cully via Guix-patches via
2022-06-24 13:28   ` [bug#55231] [PATCH v4 2/2] doc: ‘initrd-modules’ will search ‘kernel-loadable-modules’ Brian Cully via Guix-patches via

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://guix.gnu.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=2053c4ab42dfe2719cfc377934ac2fb9bcb500a9.1653160364.git.bjc@spork.org \
    --to=guix-patches@gnu.org \
    --cc=55231@debbugs.gnu.org \
    --cc=bjc@kublai.com \
    --cc=bjc@spork.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).