From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id cIYYLSef9l/2FgAA0tVLHw (envelope-from ) for ; Thu, 07 Jan 2021 05:41:59 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id cKYBKSef9l9VXwAAB5/wlQ (envelope-from ) for ; Thu, 07 Jan 2021 05:41:59 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 278D39404D0 for ; Thu, 7 Jan 2021 05:41:59 +0000 (UTC) Received: from localhost ([::1]:59262 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kxO3a-0008CN-16 for larch@yhetil.org; Thu, 07 Jan 2021 00:41:58 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39214) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kxO0k-00053e-2u for guix-patches@gnu.org; Thu, 07 Jan 2021 00:39:02 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:34556) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kxO0j-00086e-Qt for guix-patches@gnu.org; Thu, 07 Jan 2021 00:39:01 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kxO0j-0007GF-O1 for guix-patches@gnu.org; Thu, 07 Jan 2021 00:39:01 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#45692] kernel-module-configuration-service for configuring kernel parameters Resent-From: raid5atemyhomework Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 07 Jan 2021 05:39:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 45692 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Danny Milosavljevic X-Debbugs-Original-Cc: "ludo@gnu.org" , raid5atemyhomework via Guix-patches via , "45692@debbugs.gnu.org" <45692@debbugs.gnu.org> Received: via spool by submit@debbugs.gnu.org id=B.160999791827880 (code B ref -1); Thu, 07 Jan 2021 05:39:01 +0000 Received: (at submit) by debbugs.gnu.org; 7 Jan 2021 05:38:38 +0000 Received: from localhost ([127.0.0.1]:46102 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kxO0L-0007Fa-CU for submit@debbugs.gnu.org; Thu, 07 Jan 2021 00:38:38 -0500 Received: from lists.gnu.org ([209.51.188.17]:43880) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kxO04-0007FA-10 for submit@debbugs.gnu.org; Thu, 07 Jan 2021 00:38:36 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39102) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kxO02-0004Bt-AU for guix-patches@gnu.org; Thu, 07 Jan 2021 00:38:18 -0500 Received: from mail-40135.protonmail.ch ([185.70.40.135]:28105) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kxNzy-0007jK-7x for guix-patches@gnu.org; Thu, 07 Jan 2021 00:38:17 -0500 Date: Thu, 07 Jan 2021 05:38:05 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail; t=1609997890; bh=wV4po8bvpOhi9FVXRrjBOgBSUnNRGk1sLkxlx5jP/TQ=; h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From; b=BEhekRIV6PzW1qiyGRdO2+tNdlhndCBXjmutt2iiOJ3+43drwyc5kuJqwnM+2Oy9M IAUuuN/aAew5QamRVaBZFKaVJ70l7EcOsDX5mQ5a6iZb7X9d07a4bYSHPwdNtXB9wt Uyoh9BL4OTHLw6sK7WU3+qC3Wa6RGuCho3GtCe4M= Message-ID: <7kahfDckMJIQd8o2gH_zQ-G9ORP_P0w-VD44R2OJHNP6rJHss8przoTyObTYxx_UEI5CERTD3OlGORnTCBmZA2VMASWpF2Ea0Hx1ZAbUh-I=@protonmail.com> In-Reply-To: References: <6wemXB-PfHUqbuVr5-XRf0-tY4cKGGtKiUqrZPrIZYXoBw17L3xRuZrGOJQfTo5PKfFNCM8KyRTllidoc7asPE2x98BTiJSPVR7OSjxCuw8=@protonmail.com> <20210106204134.38e83db4@scratchpost.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=185.70.40.135; envelope-from=raid5atemyhomework@protonmail.com; helo=mail-40135.protonmail.ch X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 45692@debbugs.gnu.org Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" Reply-to: raid5atemyhomework , raid5atemyhomework via Guix-patches From: raid5atemyhomework via Guix-patches via X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -2.85 Authentication-Results: aspmx1.migadu.com; dkim=fail (headers rsa verify failed) header.d=protonmail.com header.s=protonmail header.b=BEhekRIV; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: 278D39404D0 X-Spam-Score: -2.85 X-Migadu-Scanner: scn0.migadu.com X-TUID: J/Q9RrSCp6IE Hi Danny, > > See also https://issues.guix.info/issue/42193 for an earlier attempt (w= hich > > is already very far--but it has a bug somewhere). There's also already = a > > kernel profile thing like you wrote in that patchset. > > (Note that I would prefer there not to be a "LOAD?" in there because it > > confuses loading the module (which is usually NOT started by user space > > but by the kernel on its own) and confguring the module (which has to b= e > > done by user space because it's specifying policy, not mechanism)) > > Looks like that patchset was merged in, so basically I can just depend on= that? So the first patch in this patchset would be dropped? No, sorry, my mistake, only one patch completely unrelated to the actual ne= w service type was merged in. In any case --- is this objection something that would block this patchset = from being added to Guix? I appreciate that we should "do it right" --- but= there's also an argument for "keep it simple", and the first patch in this= patchset gets us a good part of the way to what is needed. It seems that my patch is equivalent to the existing WIP 2/6 `kernel-profil= e-service-type`, so maybe I can just steal that patch for now, write tests = specific for it, and get on with running ZFS in production? Can we at least get patch 3/4 of *this* patchset merged since it's a trivia= l bugfix? How about patch 2/4, which is not so trivial, but does give some= flexibility in case other filesystems want to be as ambitious as ZFS is? I imagine a later, much more comprehensive `kernel-module-configuration-ser= vice-type` can be built on top of `kernel-profile-service-type`, `kernel-mo= dule-loader-service-type`, and a later `kernel-module-options-service-type`= . Then `zfs-service-type` can be modified to use that `kernel-module-confi= guration-service-type` instead of rolling its own bits and pieces. ---- On the other hand, if we want to think of `/` on ZFS, then we need a notion= of kernel modules that are added to the `initrd` file, rather than to the = `kernel` directory. Diving into `gnu/system/linux-initrd.scm`, I note that we can provide a ker= nel package and a list of modules to copy from the kernel package to the `i= nitrd` by a `(flat-linux-module-directory linux modules)` function. I imagine that it would be possible to modify this as follows: * Have `flat-linux-module-directory` accept a list of packages from which t= o find module names, not just a single package. * Remove more code from `(operating-system-directory-base-entries os)`, and= put the creation of the `"initrd"` file into a service, in much the same w= ay that my patch does (unless there's an existing more-guixy way of putting= files into `initrd`?). * Create a service type that gathers packages whose modules are to be add= ed to the `initrd`, and if that list is non-empty, pass it to a new key `#:= extra-linux-module-packages` of the `initrd` function. * `raw-initrd` would then pass that field together with the `linux` arg= ument to the modified `flat-linux-module-directory` procedure. * Create another service type that gathers module names to be loaded at `= boot` and appends them to the `#:linux-modules` to the `initrd` function. * Modify the `kernel-module-options-service-type` to pass in options via = kernel command line always, maybe. Then ZFS module can get loaded early, at boot, before the switch from `init= rd` root to the "real" root. * Create a service type that gathers additional pre-mount actions, which `r= aw-initrd` will accept as a new key `#:additional-pre-mount` and append to = the `pre-mount` it passes to `boot-system`. * The `zfs-service-type` would then extend this service type to pass in a= n action to perform a `zpool import -a -N`, which makes ZFS scan for device= s containing ZFS pools. * Somehow figure out a static build for the ZFS package, so we can use a st= atically-linked `zpool` in the above. * The `zfs-service-type` already inherits a ZFS package from the given `b= ase-package`, I imagine it would be possible to make further inheritance wh= ich modifies the build to be static. It does require an additional build, = though. Maybe an additional `root-on-zfs?` field in `zfs-configuration` ca= n gate this, so that we don't add the static build and extend the boot scri= pt if `(not root-on-zfs?)` (i.e. use `kernel-module-configuration-service-t= ype` if `(not root-on-zfs?)`, else use the new put-it-in-the-initrd service= types). * Modify the various checks elsewhere so that ZFS poolnames can be passed a= s the `device` field of `file-system` records. See https://issues.guix.gnu= .org/45643#3 * Then the installer can be modified so that if ZFS is included with the = installer's profile, it can look at ZFS pools and offer to install to a ZFS= pool instead of a `/dev` partition directly, then add `(file-system (devic= e "rootpool") (mountpoint "/") (type "zfs"))` and the appropriate `zfs-serv= ice-type` and etc. and now we get `/` on ZFS. But that can come later, much much later, I just need ZFS, *any* ZFS, befor= e RAID5 eats more of my homework. Thanks raid5atemyhomework > > But I guess the ZFS Linux kernel module can't be built-in into the kern= el > > anyway. > > But that's a special case--in general, it's very much possible to make = modules > > built-in. > > ZFScan be built-in to the kernel, Ubuntu does it. You can't distribute it= like that (Ubuntu distributes it like that but presumably they have enough= lawyers to muddy the waters so that they can get away with it), but as the= documentation in this patch notes: the user has every right to do whatever= they want on the machine they own, including build a Linux kernel that has= ZFS built-in and run it, they just can't make that version available to so= mebody else. > > So to go whole-hog, we would have a service that replaces the kernel pack= age and inserts kernel module sources in-tree somehow, then compiles Linux-= libre on the user's machine. That would probably be a lot more painful to i= nstall ZFS with (the user has to recompile the whole kernel at each update = of either kernel or ZFS, whereas with a kernel module the user has to recom= pile just the kernel module), so maybe kernel module is still better overal= l. > > > > - ;; You'd think we could've used kernel-modu= le-loader-service-type, > > > > > > > > > > Definitely. > > > > > - ;; but the kernel-module-loader shepherd se= rvice is dependent on > > > > > > > > > - ;; file-systems, > > > > > > > > > > Yes--but why is that dependent on 'file-systems ? Is it because it need= s /proc ? > > Or is it an oversight ? I would prefer to get rid of this dependency an= d then > > use kernel-module-loader-service-type. > > Dunno --- one VM I tested, I removed the`zfs-scan-automount` shepherd ser= vice from the `file-systems` target, and the VM still wouldn't boot, claimi= ng a stack overflow (the same error which I got when I was still trying to = use kernel-module-loader-service-type here). Or maybe I just got confused w= ith which VM was which, testing VMs wasn't a stress-free vacation. I just w= ant ZFS, because MD RAID5 ate my homework, this is getting tiresome... > > One thing I notice about `kernel-module-loader-service-type` is that it's= not instantiated in essential services, or indeed anywhere in Guix. A few = services do extend it. But my very rough understanding is that if you're go= ing to extend a service, it had better be instantiated once in the list of = services. > > In particular I note that the documentation for `kernel-module-loader-ser= vice-type` shows an example where it uses `service` to program the `kernel-= module-loader-service-type`, not `simple-service`. This suggests to me that= `kernel-module-loader-service-type` is broken because it's not in the list= of essential services but is extensible. Maybe. It's designed as an extens= ible service, but isn't instantiated at default. Maybe that's what really b= it me and not the shepherd circular dependency loop? shrug > > > Also, this manual loading of kernel modules is not supposed to be the w= ay to > > do things in Linux. That a kernel module was compiled as a module is > > an implementation detail--so Linux should (and usually does) automatica= lly > > load kernel modules the first time a device for them is accessed (after= all, > > how would user space know whether something is compiled as a module or > > built-in--that would be too much to ask). > > So how do I get ZFS loaded? Note that the devices it targets are block de= vices and it needs to scan for block devices that are formatted for ZFS. Do= other filesystems have some autoload rule? > > Thanks > raid5atemyhomework