unofficial mirror of guix-patches@gnu.org 
 help / color / mirror / code / Atom feed
From: raid5atemyhomework via Guix-patches via <guix-patches@gnu.org>
To: Danny Milosavljevic <dannym@scratchpost.org>
Cc: 45692@debbugs.gnu.org
Subject: [bug#45692] kernel-module-configuration-service for configuring kernel parameters
Date: Thu, 07 Jan 2021 05:38:05 +0000	[thread overview]
Message-ID: <7kahfDckMJIQd8o2gH_zQ-G9ORP_P0w-VD44R2OJHNP6rJHss8przoTyObTYxx_UEI5CERTD3OlGORnTCBmZA2VMASWpF2Ea0Hx1ZAbUh-I=@protonmail.com> (raw)
In-Reply-To: <pilwAVCb0NEG83BVTSEqXkLCNEvBBrnb-QyKV_9PoF4xuCjSXKuF-F3_L5lSOke5BRjagVrUoSzJCST1fkigF162f9j76K12jBraNMAj7E4=@protonmail.com>

Hi Danny,

> > See also https://issues.guix.info/issue/42193 for an earlier attempt (which
> > is already very far--but it has a bug somewhere). There's also already a
> > kernel profile thing like you wrote in that patchset.
> > (Note that I would prefer there not to be a "LOAD?" in there because it
> > confuses loading the module (which is usually NOT started by user space
> > but by the kernel on its own) and confguring the module (which has to be
> > done by user space because it's specifying policy, not mechanism))
>
> Looks like that patchset was merged in, so basically I can just depend on that? So the first patch in this patchset would be dropped?

No, sorry, my mistake, only one patch completely unrelated to the actual new service type was merged in.

In any case --- is this objection something that would block this patchset from being added to Guix? I appreciate that we should "do it right" --- but there's also an argument for "keep it simple", and the first patch in this patchset gets us a good part of the way to what is needed.

It seems that my patch is equivalent to the existing WIP 2/6 `kernel-profile-service-type`, so maybe I can just steal that patch for now, write tests specific for it, and get on with running ZFS in production?

Can we at least get patch 3/4 of *this* patchset merged since it's a trivial bugfix?  How about patch 2/4, which is not so trivial, but does give some flexibility in case other filesystems want to be as ambitious as ZFS is?

I imagine a later, much more comprehensive `kernel-module-configuration-service-type` can be built on top of `kernel-profile-service-type`, `kernel-module-loader-service-type`, and a later `kernel-module-options-service-type`.  Then `zfs-service-type` can be modified to use that `kernel-module-configuration-service-type` instead of rolling its own bits and pieces.

----

On the other hand, if we want to think of `/` on ZFS, then we need a notion of kernel modules that are added to the `initrd` file, rather than to the `kernel` directory.

Diving into `gnu/system/linux-initrd.scm`, I note that we can provide a kernel package and a list of modules to copy from the kernel package to the `initrd` by a `(flat-linux-module-directory linux modules)` function.

I imagine that it would be possible to modify this as follows:

* Have `flat-linux-module-directory` accept a list of packages from which to find module names, not just a single package.
* Remove more code from `(operating-system-directory-base-entries os)`, and put the creation of the `"initrd"` file into a service, in much the same way that my patch does (unless there's an existing more-guixy way of putting files into `initrd`?).
  * Create a service type that gathers packages whose modules are to be added to the `initrd`, and if that list is non-empty, pass it to a new key `#:extra-linux-module-packages` of the `initrd` function.
    * `raw-initrd` would then pass that field together with the `linux` argument to the modified `flat-linux-module-directory` procedure.
  * Create another service type that gathers module names to be loaded at `boot` and appends them to the `#:linux-modules` to the `initrd` function.
  * Modify the `kernel-module-options-service-type` to pass in options via kernel command line always, maybe.

Then ZFS module can get loaded early, at boot, before the switch from `initrd` root to the "real" root.

* Create a service type that gathers additional pre-mount actions, which `raw-initrd` will accept as a new key `#:additional-pre-mount` and append to the `pre-mount` it passes to `boot-system`.
  * The `zfs-service-type` would then extend this service type to pass in an action to perform a `zpool import -a -N`, which makes ZFS scan for devices containing ZFS pools.
* Somehow figure out a static build for the ZFS package, so we can use a statically-linked `zpool` in the above.
  * The `zfs-service-type` already inherits a ZFS package from the given `base-package`, I imagine it would be possible to make further inheritance which modifies the build to be static.  It does require an additional build, though.  Maybe an additional `root-on-zfs?` field in `zfs-configuration` can gate this, so that we don't add the static build and extend the boot script if `(not root-on-zfs?)` (i.e. use `kernel-module-configuration-service-type` if `(not root-on-zfs?)`, else use the new put-it-in-the-initrd service types).
* Modify the various checks elsewhere so that ZFS poolnames can be passed as the `device` field of `file-system` records.  See https://issues.guix.gnu.org/45643#3
  * Then the installer can be modified so that if ZFS is included with the installer's profile, it can look at ZFS pools and offer to install to a ZFS pool instead of a `/dev` partition directly, then add `(file-system (device "rootpool") (mountpoint "/") (type "zfs"))` and the appropriate `zfs-service-type` and etc. and now we get `/` on ZFS.

But that can come later, much much later, I just need ZFS, *any* ZFS, before RAID5 eats more of my homework.

Thanks
raid5atemyhomework


> > But I guess the ZFS Linux kernel module can't be built-in into the kernel
> > anyway.
> > But that's a special case--in general, it's very much possible to make modules
> > built-in.
>
> ZFScan be built-in to the kernel, Ubuntu does it. You can't distribute it like that (Ubuntu distributes it like that but presumably they have enough lawyers to muddy the waters so that they can get away with it), but as the documentation in this patch notes: the user has every right to do whatever they want on the machine they own, including build a Linux kernel that has ZFS built-in and run it, they just can't make that version available to somebody else.
>
> So to go whole-hog, we would have a service that replaces the kernel package and inserts kernel module sources in-tree somehow, then compiles Linux-libre on the user's machine. That would probably be a lot more painful to install ZFS with (the user has to recompile the whole kernel at each update of either kernel or ZFS, whereas with a kernel module the user has to recompile just the kernel module), so maybe kernel module is still better overall.
>
> > > -                         ;; You'd think we could've used kernel-module-loader-service-type,
> > >
> > >
> >
> > Definitely.
> >
> > > -                         ;; but the kernel-module-loader shepherd service is dependent on
> > >
> > >
> > > -                         ;; file-systems,
> > >
> > >
> >
> > Yes--but why is that dependent on 'file-systems ? Is it because it needs /proc ?
> > Or is it an oversight ? I would prefer to get rid of this dependency and then
> > use kernel-module-loader-service-type.
>
> Dunno --- one VM I tested, I removed the`zfs-scan-automount` shepherd service from the `file-systems` target, and the VM still wouldn't boot, claiming a stack overflow (the same error which I got when I was still trying to use kernel-module-loader-service-type here). Or maybe I just got confused with which VM was which, testing VMs wasn't a stress-free vacation. I just want ZFS, because MD RAID5 ate my homework, this is getting tiresome...
>
> One thing I notice about `kernel-module-loader-service-type` is that it's not instantiated in essential services, or indeed anywhere in Guix. A few services do extend it. But my very rough understanding is that if you're going to extend a service, it had better be instantiated once in the list of services.
>
> In particular I note that the documentation for `kernel-module-loader-service-type` shows an example where it uses `service` to program the `kernel-module-loader-service-type`, not `simple-service`. This suggests to me that `kernel-module-loader-service-type` is broken because it's not in the list of essential services but is extensible. Maybe. It's designed as an extensible service, but isn't instantiated at default. Maybe that's what really bit me and not the shepherd circular dependency loop? shrug
>
> > Also, this manual loading of kernel modules is not supposed to be the way to
> > do things in Linux. That a kernel module was compiled as a module is
> > an implementation detail--so Linux should (and usually does) automatically
> > load kernel modules the first time a device for them is accessed (after all,
> > how would user space know whether something is compiled as a module or
> > built-in--that would be too much to ask).
>
> So how do I get ZFS loaded? Note that the devices it targets are block devices and it needs to scan for block devices that are formatted for ZFS. Do other filesystems have some autoload rule?
>
> Thanks
> raid5atemyhomework






  reply	other threads:[~2021-01-07  5:41 UTC|newest]

Thread overview: 81+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-01-06 15:52 [bug#45692] [PATCH 0/4] Even Better ZFS Support on Guix raid5atemyhomework via Guix-patches via
2021-01-06 15:54 ` [bug#45692] [PATCH 1/4] gnu: Allow services to install kernel-loadable modules raid5atemyhomework via Guix-patches via
2021-01-08 16:16   ` raid5atemyhomework via Guix-patches via
2021-02-10 14:13   ` [bug#45692] [PATCH 0/4] Even Better ZFS Support on Guix Ludovic Courtès
2021-02-10 15:44   ` Ludovic Courtès
2021-02-10 16:49     ` raid5atemyhomework via Guix-patches via
2021-01-06 15:55 ` [bug#45692] [PATCH 2/4] gnu: Make file-systems target extensible by services raid5atemyhomework via Guix-patches via
2021-01-23 13:05   ` 宋文武
2021-01-25  0:18     ` guix-patches--- via
2021-02-10 14:17       ` [bug#45692] [PATCH 0/4] Even Better ZFS Support on Guix Ludovic Courtès
2021-02-10 14:46         ` raid5atemyhomework via Guix-patches via
2021-01-06 15:56 ` [bug#45692] [PATCH 3/4] gnu: Fix ZFS package raid5atemyhomework via Guix-patches via
2021-01-07  8:23   ` Danny Milosavljevic
2021-01-06 15:57 ` [bug#45692] [PATCH 4/4] gnu: Add ZFS service raid5atemyhomework via Guix-patches via
2021-01-06 19:41   ` [bug#45703] kernel-module-configuration-service for configuring kernel parameters Danny Milosavljevic
2021-01-07  0:04     ` raid5atemyhomework via Guix-patches via
2021-01-07  5:38       ` raid5atemyhomework via Guix-patches via [this message]
2021-01-07  9:16         ` [bug#42193] " raid5atemyhomework via Guix-patches via
2021-01-08 15:02   ` [bug#45692] [PATCH 4/4] gnu: Add ZFS service raid5atemyhomework via Guix-patches via
2021-01-09  8:31     ` raid5atemyhomework via Guix-patches via
2021-02-08  3:31       ` Danny Milosavljevic
2021-02-08  6:25         ` raid5atemyhomework via Guix-patches via
2021-02-10 14:27     ` [bug#45692] [PATCH 0/4] Even Better ZFS Support on Guix Ludovic Courtès
2021-02-10 14:32       ` raid5atemyhomework via Guix-patches via
2021-02-13  1:49       ` raid5atemyhomework via Guix-patches via
2021-03-22 14:33 ` [bug#45692] [PATCH v3 0/3] New patch series for " raid5atemyhomework via Guix-patches via
2021-03-28 12:55   ` Léo Le Bouter via Guix-patches via
2021-03-29  4:39     ` raid5atemyhomework via Guix-patches via
2021-07-23 15:11       ` raid5atemyhomework via Guix-patches via
2021-03-22 14:33 ` [bug#45692] [PATCH v3 1/3] gnu: Allow services to install kernel-loadable modules raid5atemyhomework via Guix-patches via
2021-05-11 14:17   ` Danny Milosavljevic
2021-03-22 14:34 ` [bug#45692] [PATCH v3 2/3] gnu: Add zfs-auto-snapshot raid5atemyhomework via Guix-patches via
2021-05-11 14:05   ` Danny Milosavljevic
2021-05-13  1:21     ` raid5atemyhomework via Guix-patches via
2021-05-13 13:08       ` Danny Milosavljevic
2021-03-22 14:35 ` [bug#45692] [PATCH v3 3/3] gnu: Add ZFS service type raid5atemyhomework via Guix-patches via
2021-07-25 14:03   ` raid5atemyhomework via Guix-patches via
2021-07-25 14:31 ` [bug#45692] [PATCH v4 " raid5atemyhomework via Guix-patches via
2021-08-01  9:41   ` raid5atemyhomework via Guix-patches via
2021-08-10 11:43     ` raid5atemyhomework via Guix-patches via
2021-08-31  0:48       ` raid5atemyhomework via Guix-patches via
2021-09-02 20:57   ` Maxime Devos
2021-09-02 22:22     ` Maxime Devos
2021-09-03 12:41       ` raid5atemyhomework via Guix-patches via
2021-09-04 18:58     ` raid5atemyhomework via Guix-patches via
2021-09-06  8:08     ` zimoun
2021-09-06 10:40       ` Maxime Devos
2021-09-06 11:08         ` raid5atemyhomework via Guix-patches via
2021-09-06 17:17         ` zimoun
2021-09-07  9:54           ` Maxime Devos
2021-09-08  1:23             ` raid5atemyhomework via Guix-patches via
2021-09-15 14:04               ` raid5atemyhomework via Guix-patches via
2021-09-21  9:42                 ` zimoun
2021-09-04 21:19   ` Xinglu Chen
2021-09-06 10:52     ` raid5atemyhomework via Guix-patches via
2021-09-06 14:22       ` Xinglu Chen
2021-09-02 21:24 ` [bug#45692] Gaslighting Mason Loring Bliss
2021-09-03 12:22   ` Maxime Devos
2021-09-06  7:59   ` [bug#45692] zimoun
2021-09-30 14:56 ` [bug#45692] [PATCH v5 3/3] gnu: Add ZFS service type raid5atemyhomework via Guix-patches via
2021-10-19 13:18   ` raid5atemyhomework via Guix-patches via
2021-10-27  7:30     ` raid5atemyhomework via Guix-patches via
2021-10-27 16:38       ` pelzflorian (Florian Pelz)
2021-11-30 15:26         ` raid5atemyhomework via Guix-patches via
2021-12-12 13:32           ` raid5atemyhomework via Guix-patches via
2021-12-21 21:15             ` [bug#45643] [PATCH 0/3] Better Support for ZFS on Guix Brice Waegeneire
2022-01-01 11:59               ` [bug#45692] bug#45643: " raid5atemyhomework via Guix-patches via
2022-01-19 14:24                 ` raid5atemyhomework via Guix-patches via
2022-01-07  4:21 ` [bug#45692] " raid5atemyhomework via Guix-patches via
2022-02-14 14:10   ` raid5atemyhomework via Guix-patches via
2022-02-18  7:13     ` raid5atemyhomework via Guix-patches via
2022-03-16 23:44       ` raid5atemyhomework via Guix-patches via
2022-03-17  8:24       ` Liliana Marie Prikler
2022-03-17 17:22         ` Maxime Devos
2022-03-17 18:38           ` zimoun
2022-03-17 19:10             ` Maxime Devos
2022-03-19 14:24           ` raid5atemyhomework via Guix-patches via
2022-03-20  4:42             ` Maxim Cournoyer
2022-03-19 14:09         ` raid5atemyhomework via Guix-patches via
2022-03-19 16:22           ` Leo Famulari
2022-03-19 14:25 ` [bug#45692] (No Subject) raid5atemyhomework via Guix-patches via

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://guix.gnu.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='7kahfDckMJIQd8o2gH_zQ-G9ORP_P0w-VD44R2OJHNP6rJHss8przoTyObTYxx_UEI5CERTD3OlGORnTCBmZA2VMASWpF2Ea0Hx1ZAbUh-I=@protonmail.com' \
    --to=guix-patches@gnu.org \
    --cc=45692@debbugs.gnu.org \
    --cc=dannym@scratchpost.org \
    --cc=raid5atemyhomework@protonmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).