Loading modules built using linux-module-build-system

Loading modules built using linux-module-build-system

[Jelle Licht]

Hello Guix,

Not too long ago, the linux-module-build-system was introduced. I ran
into some code in the wild written by Alex Griffin that defines a
shepherd service that does the following for a given kernel-module
package:

- set the LINUX_MODULE_DIRECTORY environment variable to
  <out>/lib/modules
- call modprobe on the .ko file (without .ko)

I have verified this way of loading modules to work, but was wondering
whether we should rather provide a `out-of-tree-kernel-module' service
of sorts to do this.

To resolve out-of-tree kernel module dependencies, I guess we would need
to construct a union of all outputs so we can pass along one value for
LINUX_MODULE_DIRECTORY that contains all out-of-tree modules that might
be needed for one invocation of modprobe.

Another issue is working with custom kernels; Alex' approach allows one
to override the module package by providing an expression that resolves
to a package object, which can use guile's `(inherit my-module)'
approach in combination with `substitute-keyword-arguments' to do
override the `#:linux' argument. This works and has the benefit of being
pretty explicit in defining what we want to happen. The drawback is that
one could possibly try to load a module that was built against a
different kernel version than the one in your operating system
expression.

Is there a way by which a service can refer to the
e.g. `operating-system-kernel' of the operating-system it is embedded
in?

- Jelle

Re: Loading modules built using linux-module-build-system

[Jelle Licht]

ping :-)

Jelle Licht <jlicht@fsfe.org> writes:

> Hello Guix,
>
> Not too long ago, the linux-module-build-system was introduced. I ran
> into some code in the wild written by Alex Griffin that defines a
> shepherd service that does the following for a given kernel-module
> package:
>
> - set the LINUX_MODULE_DIRECTORY environment variable to
>   <out>/lib/modules
> - call modprobe on the .ko file (without .ko)
>
> I have verified this way of loading modules to work, but was wondering
> whether we should rather provide a `out-of-tree-kernel-module' service
> of sorts to do this.

I saw a contribution for a kernel module built using
linux-module-build-system today, and I have to wonder, how are people
making use of these modules on their Guix System installation? I
currently use a hacked-together shepherd one-shot service that simply
calls modprobe, but was wondering whether folks have a nicer solution
for this.

> [snip]
> Is there a way by which a service can refer to the
> e.g. `operating-system-kernel' of the operating-system it is embedded
> in?

I actually figured this one out: As long as your service snippet
is directly embedded in the operating-system declaration, one can use:
`(operating-system-kernel this-operating-system)'.

- Jelle

Re: Loading modules built using linux-module-build-system

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 1585 bytes --]

Hi,

On Tue, 09 Jul 2019 00:50:10 +0200
Jelle Licht <jlicht@fsfe.org> wrote:

> I have verified this way of loading modules to work, but was wondering
> whether we should rather provide a `out-of-tree-kernel-module' service
> of sorts to do this.
> 
> To resolve out-of-tree kernel module dependencies, I guess we would need
> to construct a union of all outputs so we can pass along one value for
> LINUX_MODULE_DIRECTORY that contains all out-of-tree modules that might
> be needed for one invocation of modprobe.

That sounds like a strange special-casing of out-of-tree modules.

What prevents us from presenting the union
as /run/current-system/kernel/lib/modules in the first place?
That would allow everyone, including all programs on the system and the initrd,
to use all modules.

Them being in-tree or out-of-tree is a technical detail that should be of no
technical consequence to anyone.

I suggest to add a field to the operating-system declaration that
lists all the packages that contain loadable kernel modules.  We could
also just make operating-system-kernel be a list--which would be the
most direct way I guess.  After all, "the" kernel consists of all those
modules and the zImage--also in RAM in kernel space while using the stuff.

That said, extensibility would be nice--if services could request what
kernel modules to add to the operating-system.

P.S.

How's that supposed to work with the Hurd?
I'd guess they are even more modular than this.
If we add kernel modularity, might as well also take into account the Hurd.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Loading modules built using linux-module-build-system

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 1306 bytes --]

I just thought of a better way.

Represent the Linux kernel (including out-of-tree-modules) as a profile,
most probably just the system profile (which already exists--extend it).

A user package (for example if referred to by a system service) would
propagate-input the kernel module package it requires.

That would also make sure that conflicts are handled (otherwise they'd be
"handled" by the kernel doing strange stuff at runtime depending on which
package's "foo.ko" got modprobed first if there are multiple files with
the same name "foo.ko" in different packages).

The union to be presented as /run/current-system/kernel/lib/modules would
be all the packages of the system profile which have a
/lib/modules/<kernel-version> directory in their derivations.

It would all automagically work, provided that we modify Guix system so that
whoever populates /run/current-system/kernel/lib/modules does the union.

The limitation of it not be possible for a regular user to add kernel
modules is OK I think.  That's where I draw the line--a normal profile
has no business adding Linux kernel modules to kernel space in the first
place.

Not sure how the interactions with Guix on a foreign distribution would be,
though.  Do those even use the same Guile modules touched here?

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Loading modules built using linux-module-build-system

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 5735 bytes --]

A patch to guix master which

* Puts the kernel modules (including any other packages that have "lib/modules"
inside their derivation) into /run/booted-system/profile/lib/modules
* Ensures that depmod is invoked on that
* Makes the modprobe wrapper use it

is provided below.

The case when there's only one package in the system profile (the package is the
kernel) could still be optimized:
Then, depmod doesn't need to run (it already ran when building the kernel in the
first place).

Probably, the profile hook will fail in some cases because modules.dep already
exists in the source.  The patch still needs to be adapted for that.

Also, tests need to be added (system test in vm).  Help wanted :)

(Right now I didn't test it because "guix system reconfigure" fails when
building qemu-4.1.0 three times in a row (for now), wasting many hours)

Also, long term, it might make sense to use something else instead of
profile-service-type (something like a new linux-module-service-type).

diff --git a/gnu/services.scm b/gnu/services.scm
index 6ee05d4580..d2977c2012 100644
--- a/gnu/services.scm
+++ b/gnu/services.scm
@@ -491,7 +491,11 @@ ACTIVATION-SCRIPT-TYPE."
     (program-file "modprobe"
                   #~(begin
                       (setenv "LINUX_MODULE_DIRECTORY"
-                              "/run/booted-system/kernel/lib/modules")
+                              (if (file-exists? "/run/booted-system/profile/lib/modules")
+                                  "/run/booted-system/profile/lib/modules"
+                                  ;; Provides compatibility with previous
+                                  ;; Guix generations.
+                                  "/run/booted-system/kernel/lib/modules"))
                       (apply execl #$modprobe
                              (cons #$modprobe (cdr (command-line))))))))
 
diff --git a/gnu/system.scm b/gnu/system.scm
index a353b1a5c8..2dfbb9e73f 100644
--- a/gnu/system.scm
+++ b/gnu/system.scm
@@ -887,9 +887,11 @@ we're running in the final root."
 (define* (operating-system-profile os)
   "Return a derivation that builds the system profile of OS."
   (mlet* %store-monad
-      ((services -> (operating-system-services os))
-       (profile (fold-services services
-                               #:target-type profile-service-type)))
+      ((kernel -> (operating-system-kernel os))
+       (services -> (operating-system-services os))
+       (profile (cons kernel (fold-services services
+                                            #:target-type
+                                            profile-service-type))))
     (match profile
       (("profile" profile)
        (return profile)))))
diff --git a/guix/profiles.scm b/guix/profiles.scm
index cd3b21e390..13b5bf1a85 100644
--- a/guix/profiles.scm
+++ b/guix/profiles.scm
@@ -9,6 +9,7 @@
 ;;; Copyright © 2017 Huang Ying <huang.ying.caritas@gmail.com>
 ;;; Copyright © 2017 Maxim Cournoyer <maxim.cournoyer@gmail.com>
 ;;; Copyright © 2019 Kyle Meyer <kyle@kyleam.com>
+;;; Copyright © 2019 Danny Milosavljevic <dannym@scratchpost.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -1125,6 +1126,43 @@ for both major versions of GTK+."
                               (hook . gtk-im-modules)))
           (return #f)))))
 
+(define (linux-module-database manifest)
+  (mlet %store-monad
+    ((kmod (manifest-lookup-package manifest "kmod")))
+    (define build
+      (with-imported-modules '((guix build utils)
+                               (guix build union))
+       #~(begin
+          (use-modules (srfi srfi-26)
+                       (guix build utils)
+                       (guix build union)
+                       (ice-9 match))
+          (let ((destdir (string-append #$output "/lib/modules"))
+                (dirs (filter file-exists?
+                              (map (cut string-append <>
+                                        "/lib/modules")
+                                   '#$(manifest-inputs manifest))))
+                (System.maps (filter file-exists?
+                              (map (cut string-append <> "/System.map")
+                                   '#$(manifest-inputs manifest))))
+                (Module.symvers (filter file-exists?
+                              (map (cut string-append <> "/Module.symvers")
+                                   '#$(manifest-inputs manifest)))))
+              (mkdir-p (string-append #$output "/lib"))
+              (union-build destdir dirs #:create-all-directories? #t)
+              (exit (zero? (system* (string-append #$kmod "/bin/depmod")
+                    "-e" ; Report symbols that aren't supplied
+                    "-w" ; Warn on duplicates
+                    "-b" destdir
+                    "-F" (match System.maps ((x) x))
+                    "-E" (match Module.symvers ((x) x)))))))))
+    (gexp->derivation "linux-module-database" build
+                      #:local-build? #t
+                      #:substitutable? #f
+                      #:properties
+                      `((type . profile-hook)
+                        (hook . linux-module-database)))))
+
 (define (xdg-desktop-database manifest)
   "Return a derivation that builds the @file{mimeinfo.cache} database from
 desktop files.  It's used to query what applications can handle a given
@@ -1425,7 +1463,8 @@ MANIFEST."
         gtk-im-modules
         texlive-configuration
         xdg-desktop-database
-        xdg-mime-database))
+        xdg-mime-database
+        linux-module-database))
 
 (define* (profile-derivation manifest
                              #:key

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Loading modules built using linux-module-build-system

[Ludovic Courtès]

Hi Danny,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> A patch to guix master which
>
> * Puts the kernel modules (including any other packages that have "lib/modules"
> inside their derivation) into /run/booted-system/profile/lib/modules
> * Ensures that depmod is invoked on that
> * Makes the modprobe wrapper use it
>
> is provided below.

Nice!

I’m wondering if we could avoid clobbering the global profile with the
kernel and module packages, though (as is currently the case.)

> --- a/gnu/system.scm
> +++ b/gnu/system.scm
> @@ -887,9 +887,11 @@ we're running in the final root."
>  (define* (operating-system-profile os)
>    "Return a derivation that builds the system profile of OS."
>    (mlet* %store-monad
> -      ((services -> (operating-system-services os))
> -       (profile (fold-services services
> -                               #:target-type profile-service-type)))
> +      ((kernel -> (operating-system-kernel os))
> +       (services -> (operating-system-services os))
> +       (profile (cons kernel (fold-services services
> +                                            #:target-type
> +                                            profile-service-type))))
>      (match profile
>        (("profile" profile)
>         (return profile)))))

The value of ‘profile’ above can never match this pattern, or am I
missing something?

> +(define (linux-module-database manifest)
> +  (mlet %store-monad
> +    ((kmod (manifest-lookup-package manifest "kmod")))
> +    (define build
> +      (with-imported-modules '((guix build utils)
> +                               (guix build union))
> +       #~(begin
> +          (use-modules (srfi srfi-26)
> +                       (guix build utils)
> +                       (guix build union)
> +                       (ice-9 match))
> +          (let ((destdir (string-append #$output "/lib/modules"))
> +                (dirs (filter file-exists?
> +                              (map (cut string-append <>
> +                                        "/lib/modules")
> +                                   '#$(manifest-inputs manifest))))
> +                (System.maps (filter file-exists?
> +                              (map (cut string-append <> "/System.map")
> +                                   '#$(manifest-inputs manifest))))
> +                (Module.symvers (filter file-exists?
> +                              (map (cut string-append <> "/Module.symvers")
> +                                   '#$(manifest-inputs manifest)))))
> +              (mkdir-p (string-append #$output "/lib"))
> +              (union-build destdir dirs #:create-all-directories? #t)
> +              (exit (zero? (system* (string-append #$kmod "/bin/depmod")
> +                    "-e" ; Report symbols that aren't supplied
> +                    "-w" ; Warn on duplicates
> +                    "-b" destdir
> +                    "-F" (match System.maps ((x) x))
> +                    "-E" (match Module.symvers ((x) x)))))))))
> +    (gexp->derivation "linux-module-database" build
> +                      #:local-build? #t
> +                      #:substitutable? #f
> +                      #:properties
> +                      `((type . profile-hook)
> +                        (hook . linux-module-database)))))

Note that this fails if ‘kmod’ is no in the profile.

Besides, I wonder how much is missing from (gnu build linux-modules) to
do this without resorting to kmod.  :-)

>  (define (xdg-desktop-database manifest)
>    "Return a derivation that builds the @file{mimeinfo.cache} database from
>  desktop files.  It's used to query what applications can handle a given
> @@ -1425,7 +1463,8 @@ MANIFEST."
>          gtk-im-modules
>          texlive-configuration
>          xdg-desktop-database
> -        xdg-mime-database))
> +        xdg-mime-database
> +        linux-module-database))

Maybe we should not add this hook by default, to avoid overhead for a
very unusual use case.  The caller of ‘profile-derivation’ could add it
when needed.

Thinking more about it, what about handling the union/profile thing in
‘operating-system-directory-base-entries’?  We could still use
‘profile-derivation’ with the hook you wrote, but we’d be able to keep
that here instead of adding it to (guix profiles).

Also, if we take that route, we would probably need a
‘linux-module-packages’ field in <operating-system>.

WDYT?

Thanks,
Ludo’.

Re: Loading modules built using linux-module-build-system

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 2178 bytes --]

Hi Ludo,

On Tue, 22 Oct 2019 14:24:51 +0200
Ludovic Courtès <ludo@gnu.org> wrote:

> I’m wondering if we could avoid clobbering the global profile with the
> kernel and module packages, though (as is currently the case.)

Sounds good.

> >  (define* (operating-system-profile os)
> >    "Return a derivation that builds the system profile of OS."
> >    (mlet* %store-monad
> > -      ((services -> (operating-system-services os))
> > -       (profile (fold-services services
> > -                               #:target-type profile-service-type)))
> > +      ((kernel -> (operating-system-kernel os))
> > +       (services -> (operating-system-services os))
> > +       (profile (cons kernel (fold-services services
> > +                                            #:target-type
> > +                                            profile-service-type))))
> >      (match profile
> >        (("profile" profile)
> >         (return profile)))))  
> 
> The value of ‘profile’ above can never match this pattern, or am I
> missing something?

Ahhhhh!  Now I see it.

> Besides, I wonder how much is missing from (gnu build linux-modules) to
> do this without resorting to kmod.  :-)

True :)

> Maybe we should not add this hook by default, to avoid overhead for a
> very unusual use case.  The caller of ‘profile-derivation’ could add it
> when needed.

I think the eventual overhead of it is minimal because it can just check
whether "lib/modules" is anywhere in the source and if not, skip everything.

> Thinking more about it, what about handling the union/profile thing in
> ‘operating-system-directory-base-entries’?  We could still use
> ‘profile-derivation’ with the hook you wrote, but we’d be able to keep
> that here instead of adding it to (guix profiles).

Should we just change out the "kernel" entry to be the union?  After all,
that's what actually happens at runtime as far as the Linux kernel is
concerned.

> Also, if we take that route, we would probably need a
> ‘linux-module-packages’ field in <operating-system>.

Or make the `kernel` field a list, including linux-libre ;-)

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Loading modules built using linux-module-build-system

[Ludovic Courtès]

Hello,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

>> Thinking more about it, what about handling the union/profile thing in
>> ‘operating-system-directory-base-entries’?  We could still use
>> ‘profile-derivation’ with the hook you wrote, but we’d be able to keep
>> that here instead of adding it to (guix profiles).
>
> Should we just change out the "kernel" entry to be the union?  After all,
> that's what actually happens at runtime as far as the Linux kernel is
> concerned.

Yes, that sounds good.

>> Also, if we take that route, we would probably need a
>> ‘linux-module-packages’ field in <operating-system>.
>
> Or make the `kernel` field a list, including linux-libre ;-)

Rather than a list, we could have a ‘make-linux-libre-union’ procedure
returning a <package>, so that we preserve consistent typing.

That is, people could write:

  (kernel linux-libre)

or:

  (kernel (make-linux-libre-union linux-libre some-package))

WDYT?

Thanks,
Ludo’.

Re: Loading modules built using linux-module-build-system

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 707 bytes --]

Hi Ludo,

On Sun, 17 Nov 2019 21:35:32 +0100
Ludovic Courtès <ludo@gnu.org> wrote:
> Rather than a list, we could have a ‘make-linux-libre-union’ procedure
> returning a <package>, so that we preserve consistent typing.
> 
> That is, people could write:
> 
>   (kernel linux-libre)
> 
> or:
> 
>   (kernel (make-linux-libre-union linux-libre some-package))
> 
> WDYT?

Hmm, isn't it more like a profile?  I mean it would work the way above but
there's (presumably) some reason why SOME-PACKAGE was an extra package.

We don't have to use the /run/current-system/profile for that, it could be
a new one.

What are the downside of using a profile vs. using a package in that way?

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Loading modules built using linux-module-build-system

[Ludovic Courtès]

Hello,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> On Sun, 17 Nov 2019 21:35:32 +0100
> Ludovic Courtès <ludo@gnu.org> wrote:
>> Rather than a list, we could have a ‘make-linux-libre-union’ procedure
>> returning a <package>, so that we preserve consistent typing.
>> 
>> That is, people could write:
>> 
>>   (kernel linux-libre)
>> 
>> or:
>> 
>>   (kernel (make-linux-libre-union linux-libre some-package))
>> 
>> WDYT?
>
> Hmm, isn't it more like a profile?  I mean it would work the way above but
> there's (presumably) some reason why SOME-PACKAGE was an extra package.

You’re right, the union thing above is like a profile.

> We don't have to use the /run/current-system/profile for that, it could be
> a new one.
>
> What are the downside of using a profile vs. using a package in that way?

No downside to using a profile, as long as it’s not
/run/current-system/profile.  The only remaining question is the
programming interface.

Possible options include ‘make-linux-libre-union’ above or a new
‘linux-module-packages’ field in <operating-system> as discussed at
<https://lists.gnu.org/archive/html/guix-devel/2019-10/msg00514.html>.

HTH,
Ludo’.

Re: Loading modules built using linux-module-build-system

[Giovanni Biscuolo]

Hi Danny,

Danny Milosavljevic <dannym@scratchpost.org> writes:

[...]

> A user package (for example if referred to by a system service) would
> propagate-input the kernel module package it requires.

Yes! \O/

[...]

> The limitation of it not be possible for a regular user to add kernel
> modules is OK I think.

When I wear the regular user hat I would simply like to forget about
kernel modules, letting the service use (reference a package) whatever
kernel module it needs to do the job

> That's where I draw the line--a normal profile has no business adding
> Linux kernel modules to kernel space in the first place.

I agree, that is a system level profile business

[...]

AFAIU sounds like a good plan, sorry I just canno help here :-S

Thanks! Gio'

-- 
Giovanni Biscuolo

Xelera IT Infrastructures