From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id 8tSpOKy+yV5HQQAA0tVLHw (envelope-from ) for ; Sun, 24 May 2020 00:24:12 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id iCj9M6y+yV5pXQAAB5/wlQ (envelope-from ) for ; Sun, 24 May 2020 00:24:12 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 486BA94030E for ; Sun, 24 May 2020 00:24:12 +0000 (UTC) Received: from localhost ([::1]:37810 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jceQz-0007iy-FE for larch@yhetil.org; Sat, 23 May 2020 20:24:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49128) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jceQs-0007ih-Cl for guix-patches@gnu.org; Sat, 23 May 2020 20:24:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:52762) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jceQs-00028s-3s for guix-patches@gnu.org; Sat, 23 May 2020 20:24:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jceQr-0001jU-WE for guix-patches@gnu.org; Sat, 23 May 2020 20:24:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#41011] [PATCH] gnu: grub: Support for network boot via tftp/nfs. Resent-From: Stefan Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 24 May 2020 00:24:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 41011 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Mathieu Othacehe Cc: 41011@debbugs.gnu.org Received: via spool by 41011-submit@debbugs.gnu.org id=B41011.15902797826574 (code B ref 41011); Sun, 24 May 2020 00:24:01 +0000 Received: (at 41011) by debbugs.gnu.org; 24 May 2020 00:23:02 +0000 Received: from localhost ([127.0.0.1]:36075 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jcePt-0001hi-7h for submit@debbugs.gnu.org; Sat, 23 May 2020 20:23:02 -0400 Received: from vsmx012.vodafonemail.xion.oxcs.net ([153.92.174.90]:19807) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jcePr-0001hR-50 for 41011@debbugs.gnu.org; Sat, 23 May 2020 20:23:00 -0400 Received: from vsmx004.vodafonemail.xion.oxcs.net (unknown [192.168.75.198]) by mta-8-out.mta.xion.oxcs.net (Postfix) with ESMTP id 4FF84F34E11; Sun, 24 May 2020 00:22:53 +0000 (UTC) Received: from macbook-pro.kuh-wiese.my-router.de (unknown [2.206.141.243]) by mta-8-out.mta.xion.oxcs.net (Postfix) with ESMTPA id BA50B19AE6B; Sun, 24 May 2020 00:22:48 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) From: Stefan In-Reply-To: <87d06vcbrr.fsf@gnu.org> Date: Sun, 24 May 2020 02:22:47 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <8A871A19-81A8-46D3-86C1-DA9FA209ED70@vodafonemail.de> References: <9AAFEFF4-8ACE-4C95-975F-67C3F4FDAF81@vodafonemail.de> <87a72gi4kz.fsf@gmail.com> <1179D890-7D6C-43D8-A286-DA7A0F61D585@vodafonemail.de> <0CE089DD-54B6-43AD-8CE7-8EC7004768EB@vodafonemail.de> <87d06vcbrr.fsf@gnu.org> X-Mailer: Apple Mail (2.3124) X-VADE-STATUS: LEGIT X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -1.7 (-) X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Spam-Score: 0.99 X-TUID: JwnKjgvhUb7l Hi Mathieu! > Am 23.05.2020 um 10:10 schrieb Mathieu Othacehe : >=20 >> + #~(lambda (bootloader target mount-point) >> + "Install GRUB as e.g. \"bootx64.efi\" or \"bootarm64.efi\" = \"into >> +EFI-SUBDIR, which is usually \"efi/boot\" or \"efi/Guix\" below the = directory >=20 > Is that TARGET or EFI-SUBDIR? I really mean with EFI-SUBDIR the argument from the enclosing = install-grub-efi-net function, and with TARGET the argument of the = lambda function. The prototype of the lambda function is fixed by some = other guix machinery, it is not possible to pass an EFI-SUBDIR argument = directly. Take for example this path, the default: "/mnt/boot/efi/boot". Then = MOUNT-POINT is "/mnt" (like in 'guix system init config.scm /mnt'), = TARGET is "/boot" (which you also have to specify inside = (operating-system (bootloader (target "/boot") =E2=80=A6) =E2=80=A6), = and EFI-SUBDIR is the remaining "efi/boot". In my personal case I need MOUNT-POINT as "/" (well, 'guix system init = =E2=80=A6' was only necessary once), TARGET as "boot-nfs" (my special = setup), and EFI-SUBDIR as "efi/boot" (because the U-Boot is using this = path by default). For a TFFP server this TARGET with /boot (or /boot-nfs in my case) is = the directory whose content has to be served. It can=E2=80=99t be = /boot/efi (like for the normal grub-efi bootloader), as the TFTP server = needs to serve several firmware files and the U-Boot.=C2=B9 When the = U-Boot takes control, by default it tries to load /efi/boot/bootaa64.efi = via TFTP. It may be possible to configure the U-Boot to look for files below a = /efi/Guix directory or any other path, I=E2=80=99m not really sure about = this yet. At least /efi/boot is the documented last resort for UEFI = systems to look for some =E2=80=9Cremovable madia=E2=80=9D to boot over = TFTP. Next GRUB takes control and looks for its grub.cfg and all its other = files via TFTP at the path specified during 'grub-mknetdir = --subdir=3D=E2=80=A6'. As the TFTP root is /boot (where U-Boot and other = stuff may live), this now has to be efi/boot. So you get all GRUB files = like /boot/efi/boot/grub.cfg served via TFTP from the directory = /boot/efi/boot. For the usual grub-efi-bootloader this is a bit different. Here the = normal path is /mnt/boot/efi/Guix and for the 'grub-install' command = these parameters are required: '=E2=80=94-boot-directory /mnt/boot = --bootloader-id Guix --efi-directory /boot/efi'. Here the "boot" part to = '--boot-directory' is hard coded, as well as "Guix". So the only part = you are free to chose is "efi" in (operating-system (bootloader (target = "boot/efi") =E2=80=A6) =E2=80=A6). Additionally the usual = grub-efi-bootloader looks for /boot/grub.cfg which thus is not residing = inside the efi partition like all its other files. >> +TARGET for the system whose root is mounted at MOUNT-POINT." >> + (let* ((mount-point-list (delete "" (string-split = mount-point #\/))) >> + (target-list (delete "" (string-split target #\/))) >> + (net-dir >> + (string-append "/" (string-join (append >> + mount-point-list >> + target-list) >> + "/"))) >=20 > I think you can use something like "(in-vicinity mount-point target)" > to do the same job. MOUNT-POINT may be just "/", TARGET is "/boot". I=E2=80=99m avoiding = double slashes, like "//boot", also for all the other paths. In an = earlier version I had the =E2=80=9Cproblem=E2=80=9D that GRUB searched = files with the prefix "//efi/boot" via TFTP. This confused me on one = side, but on the other side I feared that another TFTP server might get = problems with this. Avoiding double slashes seems not to be something that in-vicinity is = taking care about. And TARGET with its "/boot" looks like an absolute = path, making it an improper argument to in-vicinity: =E2=80=9Cin-vicinity = should allow filename to override vicinity when filename is an absolute = pathname and vicinity is equal to the value of (user-vicinity). The = behavior of in-vicinity when filename is absolute and vicinity is not = equal to the value of (user-vicinity) is unspecified.=E2=80=9D A bit further down I need to count the directory levels up to the "/gnu" = store to construct a proper link, so I need the result of a string-split = anyway.=20 So all in all =E2=80=93 although it looks a bit complicated =E2=80=93 = I=E2=80=99d like to keep this. Hm, but I wasn=E2=80=99t that strict with the EFI-SUBDIR argument. And = potentially MOUNT-POINT may be a relative path, may it? Do you know? In = that case there is a bug in prepending the "/" to net-dir, turning a = relative into an absolute path. >=20 >> + ;; Tell 'grub-install' that there might be a = LUKS-encrypted /boot or >> + ;; root partition. >> + (setenv "GRUB_ENABLE_CRYPTODISK" "y") >> + (invoke/quiet (string-append bootloader = "/bin/grub-mknetdir") >> + (string-append "--net-directory=3D" net-dir) >> + (string-append "--subdir=3D" subdir)) >> + (false-if-exception >> + (delete-file efi-bootloader-link)) >> + (symlink #$efi-bootloader >> + efi-bootloader-link) >> + (false-if-exception >> + (delete-file store-link)) >> + (symlink store >> + store-link))))) >=20 > What's the purpose of those two symlinks, isn't grub-mknetdir taking > care of all this? No, it doesn=E2=80=99t. The final .efi file after calling = 'grub-mknetdir' is /boot/efi/boot/arm64-efi/core.efi. I guess when using = PXE there is a way to point to this efi file, so GRUB doesn=E2=80=99t = care. But without this the U-Boot looks for the standardised = /efi/boot/bootaa64.efi file via TFTP. So the first symlink fixes this = gap without removing other possibilities. The second symlink is for GRUB itself. Inside /boot/efi/boot/grub.cfg = there are commands to specify the root device, usually via a file system = label or a UUID. But in this case the root device specification is just = =E2=80=9Dset root=3D(tftp)=E2=80=9D. Following lines look up files by = e.g. /gnu/store/=E2=80=A6-grub-2.04/share/grub/unicode.pf2, which GRUB = will then try to access via TFTP. But the root for the TFTP server is = /boot from the TARGET argument, and not /. So GRUB trying to access = /gnu/store/=E2=80=A6 results in an access to /boot/gnu/store/=E2=80=A6. = Therefore the symlink from /boot/gnu to ../gnu is needed. And as TARGET = can freely be chosen the number of up-levels needs to be calculated. Where the usual grub-efi-bootloader is able to find and access the real = root device directly, the grub-efi-net-bootloader can only use TFTP. = Therefore we have to make the real =E2=80=9Croot device=E2=80=9D = accessible via TFPT. I=E2=80=99m not sure if this is wise from a = security point of view. But otherwise all files listed in the grub.cfg = would need to be copied. This would be an overhead and leads to more = complications, when deleting system generations; then obsolete copies = need determined and removed again.=20 > Creating a system test for this may be a bit difficult, but if you = could > add a section in the documentation describing how to setup a > 'grub-efi-net-bootloader, that would be great! I have some difficulties with this =E2=80=93 despite not having = knowledge with texi yet. So help is very welcome. You need to make use of an NFS server, serving your / directory.=20 You need a TFTP server set up to serve the files of our /boot directory = with access to your /gnu store via the /boot/gnu symlink as well.=20 You need to setup your DNS server to send some boot-options.=20 I use a =E2=80=9CDiskStation=E2=80=9D with its UI for all this but = needed to fiddle around with dnsmasq for the DNS. So this is nothing for = a guix documentation. However, this might be even out of scope. But = ideally there would be examples of how to achieve all this with guix = itself. I have no experience using guix for anything of this. Then I use all this to boot a Raspberry Pi 3b in aarch64 mode. This = requires a none free firmware, additional configuration files for the = firmware, the (modified) U-Boot. And there is even more. I compiled the = kernel linux (currently from the mainline) with a Raspberry specific = defconfig-package as an input for the linux package and an additional = function to modify the defconfig on the fly. There are more patches to follow for at least parts of all this. But what I actually want to say with this: Even to reproduce my setup, = still a lot other stuff is missing. And I have no other computer for = tinkering. I can=E2=80=99t try out this patch on a usual x86=E2=80=A664 = machine. I can't really tell, if this will work right out of the box for = other people on different computers. Before documenting this for the public, I would wish that at least = someone else with access to an x86_64 UEFI system gives it a try. I'm = pretty sure it will work. Besides all the server stuff with NFS, TFTP and DNS, there are only few = specialities to know: (operating-system (file-systems (cons (file-system (mount-point "/") (type "nfs") (device ":/your/servers/path/to/guix-root") (options "addr=3D10.11.12.2,vers=3D4.1")) %base-file-systems))=20 (bootloader=20 (target "/boot") (bootloader-configuration (bootloader grub-efi-net-bootloader) =E2=80=A6)) =E2=80=A6) The IP address of your server needs to be specified via an =E2=80=9Caddr=3D= =E2=80=9D option. To make use of the defaults of the new = grub-efi-net-bootloader, the target field has to be "/boot". Bye Stefan =C2=B9 Well, actually =E2=80=93 as for grub-efi-net-bootloader you are = now free to use any path for target =E2=80=93 you can use "/boot/efi". = But then you need to make this your TFTP server=E2=80=99s root, and you = need to use the make-grub-efi-net-bootloader macro with "/boot/efi" as = well, and you will end up in having double efi folders as in = /boot/efi/efi/boot/grub.cfg etc. So this doesn=E2=80=99t make sense.=