* bug#47744: nfs-root-fs test is failing
@ 2021-04-13 8:10 Mathieu Othacehe
2021-04-17 17:19 ` Stefan
2021-09-26 12:57 ` bug#47744: " Stefan
0 siblings, 2 replies; 10+ messages in thread
From: Mathieu Othacehe @ 2021-04-13 8:10 UTC (permalink / raw)
To: 47744; +Cc: Stefan
Hello,
I'm working on fixing all the system tests before the release. The
"nfs-root-fs" test is failing this way:
--8<---------------cut here---------------start------------->8---
mathieu@meije ~/guix [env]$ cat /gnu/store/s1byimibj7ax9b4nqg4qmpvhyf8wlcp8-nfs-root-fs-test/start-nfs-boot-test.log
%%%% Starting test start-nfs-boot-test
Group begin: start-nfs-boot-test
Test begin:
test-name: "nfs services are running"
source-file: "/gnu/store/m2lp9cq1nhhy5bydbzcy4svpb2wgsmx3-nfs-root-fs-test-builder"
source-line: 1
source-form: (test-assert "nfs services are running" (wait-for-file "/var/run/rpc.statd.pid" server-marionette))
Test end:
result-kind: pass
actual-value: 185
Test begin:
test-name: "NFS port is ready"
source-file: "/gnu/store/m2lp9cq1nhhy5bydbzcy4svpb2wgsmx3-nfs-root-fs-test-builder"
source-line: 1
source-form: (test-assert "NFS port is ready" (wait-for-tcp-port 2049 server-marionette))
Test end:
result-kind: pass
actual-value: #t
Test begin:
test-name: "NFS statd port is ready"
source-file: "/gnu/store/m2lp9cq1nhhy5bydbzcy4svpb2wgsmx3-nfs-root-fs-test-builder"
source-line: 1
source-form: (test-assert "NFS statd port is ready" (wait-for-tcp-port 20002 server-marionette))
Test end:
result-kind: fail
actual-value: #f
actual-error: (misc-error #f "nobody's listening on port ~S" (20002) #f)
Test begin:
test-name: "NFS mountd port is ready"
source-file: "/gnu/store/m2lp9cq1nhhy5bydbzcy4svpb2wgsmx3-nfs-root-fs-test-builder"
source-line: 1
source-form: (test-assert "NFS mountd port is ready" (wait-for-tcp-port 20001 server-marionette))
Test end:
result-kind: fail
actual-value: #f
actual-error: (misc-error #f "nobody's listening on port ~S" (20001) #f)
Test begin:
source-file: "/gnu/store/m2lp9cq1nhhy5bydbzcy4svpb2wgsmx3-nfs-root-fs-test-builder"
source-line: 1
source-form: (test-assert "nfs-root-client booted")
Test end:
result-kind: pass
actual-value: "nfs-root-client booted"
Test begin:
test-name: "nfs client deposited file"
source-file: "/gnu/store/m2lp9cq1nhhy5bydbzcy4svpb2wgsmx3-nfs-root-fs-test-builder"
source-line: 1
source-form: (test-assert "nfs client deposited file" (wait-for-file "/export/mounts" server-marionette))
Test end:
result-kind: fail
actual-value: #f
actual-error: (misc-error #f "file didn't show up ~S" ("/export/mounts") #f)
--8<---------------cut here---------------end--------------->8---
Danny, Stefan, do you think you could have a look?
Thanks,
Mathieu
^ permalink raw reply [flat|nested] 10+ messages in thread* bug#47744: nfs-root-fs test is failing 2021-04-13 8:10 bug#47744: nfs-root-fs test is failing Mathieu Othacehe @ 2021-04-17 17:19 ` Stefan 2021-04-18 18:29 ` Stefan 2021-09-26 12:57 ` bug#47744: " Stefan 1 sibling, 1 reply; 10+ messages in thread From: Stefan @ 2021-04-17 17:19 UTC (permalink / raw) To: Mathieu Othacehe, Danny Milosavljevic; +Cc: 47744 Hi Mathieu! > I'm working on fixing all the system tests before the release. The > "nfs-root-fs" test is failing this way: > Danny, Stefan, do you think you could have a look? I remember that I did the sketch for this test, mainly copying from the other nfs tests. But in the end my Raspberry was too weak for testing or even compiling everything necessary. A bit later Danny improved the test and I got the impression that it was running. I’ll try to take a look tomorrow, let’s see. Bye Stefan ^ permalink raw reply [flat|nested] 10+ messages in thread
* bug#47744: nfs-root-fs test is failing 2021-04-17 17:19 ` Stefan @ 2021-04-18 18:29 ` Stefan 2021-04-22 9:30 ` Mathieu Othacehe 0 siblings, 1 reply; 10+ messages in thread From: Stefan @ 2021-04-18 18:29 UTC (permalink / raw) To: Mathieu Othacehe, Danny Milosavljevic; +Cc: 47744 [-- Attachment #1: Type: text/plain, Size: 3625 bytes --] Hi Mathieu and Danny! I’m now sure that this test has never been working. To get it working, first of all we need a change in (gnu system vm) to not discard a root file-system of type “nfs”. I did this change. I also cleaned up the test itself. See the attached patch. The test needs two virtual machines communicating over TCP. The NFS client machine needs the linux command line argument “ip=dhcp“ (or a static IP setting) or some other code inside the initrd to bring the network interface up and running before mounting its root file-system over NFS. And finally some QEMU networking needs to be setup to have the NFS server and client talk to each other with proper IP addresses. Unfortunately I have no clue how to achieve these two prerequisites. The kernel argument “ip=dhcp” does not help, as the driver for the network interface is not part of the linux-libre kernel. And also adding the initrd-module e1000 comes too late. On my Raspberry Pi I’m using a modified kernel with a build-in network driver and the “ip=dhcp” argument. It seems to me that the base-initrd is missing an argument to add some arbitrary expression, or an option to configure the network interface, or the shepherd service which is provisioning 'networking. For network booting the initrd in Guix currently seems to be useless – however, getting rid of it is not trivial either. With the attached changes, the NFS server side tests are passing, but the NFS client is missing an own IP address when trying to mount its root file-system, which results in this error when executing “make check-system TESTS=nfs-root”: Welcome, this is GNU's early boot Guile. Use '--repl' for an initrd REPL. loading kernel modules... [ 27.250034] e1000: Intel(R) PRO/1000 Network Driver [ 27.250975] e1000: Copyright (c) 1999-2006 Intel Corporation. [ 28.026651] PCI Interrupt Link [LNKC] enabled at IRQ 11 [ 28.377771] e1000 0000:00:03.0 eth0: (PCI:33MHz:32-bit) 52:54:00:12:34:56 [ 28.388012] e1000 0000:00:03.0 eth0: Intel(R) PRO/1000 Network Connection [ 28.706203] usbcore: registered new interface driver usb-storage [ 28.839354] usbcore: registered new interface driver uas [ 28.922481] hid: raw HID events driver (C) Jiri Kosina [ 28.942031] usbcore: registered new interface driver usbhid [ 28.943247] usbhid: USB HID core driver [ 29.425330] isci: Intel(R) C600 SAS Controller Driver - version 1.2.0 [ 30.100062] PCI Interrupt Link [LNKD] enabled at IRQ 10 [ 30.724003] PCI Interrupt Link [LNKA] enabled at IRQ 10 [ 31.335165] PCI Interrupt Link [LNKB] enabled at IRQ 11 [ 32.129714] virtio_blk virtio3: [vda] 143360 512-byte logical blocks (73.4 MB/70.0 MiB) [ 32.167257] vda: vda1 vda2 [ 32.526742] random: crng init done [ 32.624351] FS-Cache: Loaded [ 32.675399] 9pnet: Installing 9P2000 support [ 32.692551] 9p: Installing v9fs 9p2000 file system support [ 32.695268] FS-Cache: Netfs '9p' registered for caching configuring QEMU networking... [ 32.842717] e1000: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX [ 32.859236] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready In gnu/build/linux-boot.scm: 599:12 3 (_) 423:8 2 (mount-root-file-system ":/export" "nfs" # _ #:flags _ # ?) In unknown file: 1 (mount ":/export" "/real-root" "nfs" 1 "addr=127.0.0.1") In ice-9/boot-9.scm: 1669:16 0 (raise-exception _ #:continuable? _) In procedure mount: No such device [ 33.616971] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000000 Bye Stefan [-- Attachment #2: 0001-NFS-root-for-virtual-machines.patch --] [-- Type: application/octet-stream, Size: 13268 bytes --] From 76ef5ee8d5505bf82c3226398844224e22fa78e1 Mon Sep 17 00:00:00 2001 From: Stefan <stefan-guix@vodafonemail.de> Date: Sun, 18 Apr 2021 19:32:55 +0200 Subject: [PATCH] NFS root for virtual machines. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit gnu: system: Allow a root file-system over NFS for virtual machines. gnu: tests: Improve the test for a root file-system over NFS. However, the test is still failing. * gnu/system/vm.scm (virtualized-operating-system): Allow root file-systems over NFS. (system-qemu-image/shared-store-script): Respect the configured root file-system-device. * gnu/tests/nfs.scm (run-nfs-root-fs-test): Renamed to … (run-nfs-root-test): … this. Cleanup and improvements. (%test-nfs-root-fs): Renamed to … (%test-nfs-root): … this. Renamed the test from "nfs-root-fs" to "nfs-root". --- gnu/system/vm.scm | 30 ++++++--- gnu/tests/nfs.scm | 160 ++++++++++++++++++++-------------------------- 2 files changed, 92 insertions(+), 98 deletions(-) diff --git a/gnu/system/vm.scm b/gnu/system/vm.scm index 97adfa12fa..be0a4695f2 100644 --- a/gnu/system/vm.scm +++ b/gnu/system/vm.scm @@ -608,7 +608,8 @@ environment with the store shared with the host. MAPPINGS is a list of (let ((target (file-system-mount-point fs)) (source (file-system-device fs))) (or (string=? target (%store-prefix)) - (string=? target "/") + (and (string=? target "/") + (not (string=? (file-system-type fs) "nfs"))) (and (string? source) (string-prefix? "/dev/" source)) @@ -618,14 +619,20 @@ environment with the store shared with the host. MAPPINGS is a list of (uuid? source)))))) (operating-system-file-systems os))) - (define virtual-file-systems - (cons (file-system - (mount-point "/") - (device "/dev/vda1") - (type "ext4")) + (define (add-missing-root-fs user-file-systems) + (if (null? (filter (lambda (fs) + (string=? (file-system-mount-point fs) "/")) + user-file-systems)) + (cons (file-system + (mount-point "/") + (device "/dev/vda1") + (type "ext4")) + user-file-systems) + user-file-systems)) - (append (map mapping->file-system mappings) - user-file-systems))) + (define virtual-file-systems + (append (map mapping->file-system mappings) + (add-missing-root-fs user-file-systems))) (operating-system (inherit os) @@ -754,7 +761,12 @@ it is mostly useful when FULL-BOOT? is true." #:disk-image-size disk-image-size))) (define kernel-arguments #~(list #$@(if graphic? #~() #~("console=ttyS0")) - #+@(operating-system-kernel-arguments os "/dev/vda1"))) + #+@(operating-system-kernel-arguments + os + (file-system-device + (first (filter (lambda (fs) + (string=? (file-system-mount-point fs) "/")) + (operating-system-file-systems os))))))) (define qemu-exec #~(list #+(file-append qemu "/bin/" diff --git a/gnu/tests/nfs.scm b/gnu/tests/nfs.scm index 9b2b785176..33c019da89 100644 --- a/gnu/tests/nfs.scm +++ b/gnu/tests/nfs.scm @@ -27,6 +27,7 @@ #:use-module (gnu bootloader grub) #:use-module (gnu system) #:use-module (gnu system file-systems) + #:use-module (gnu system linux-initrd) #:use-module (gnu system shadow) #:use-module (gnu system vm) #:use-module (gnu services) @@ -40,7 +41,7 @@ #:use-module (guix monads) #:export (%test-nfs %test-nfs-server - %test-nfs-root-fs)) + %test-nfs-root)) (define %base-os (operating-system @@ -265,7 +266,7 @@ directories can be mounted.") (value (run-nfs-server-test)))) -(define (run-nfs-root-fs-test) +(define (run-nfs-root-test) "Run a test of an OS mounting its root file system via NFS." (define nfs-root-server-os (marionette-operating-system @@ -275,15 +276,8 @@ directories can be mounted.") (modify-services (operating-system-user-services %nfs-os) (nfs-service-type config => (nfs-configuration - (debug '(nfs nfsd mountd)) - ;;; Note: Adding the following line causes Guix to hang. - ;(rpcmountd-port 20001) - ;;; Note: Adding the following line causes Guix to hang. - ;(rpcstatd-port 20002) ; FIXME: Set broadcast port AND listening port. - (nfsd-port 2049) - (nfs-versions '("4.2")) (exports '(("/export" - "*(rw,insecure,no_subtree_check,crossmnt,fsid=root,no_root_squash,insecure,async)")))))))) + "*(rw,insecure,no_subtree_check,crossmnt,fsid=root,no_root_squash,async)")))))))) #:requirements '(nscd) #:imported-modules '((gnu services herd) (guix combinators)))) @@ -292,13 +286,23 @@ directories can be mounted.") (marionette-operating-system (operating-system (inherit (simple-operating-system (service dhcp-client-service-type))) + (host-name "nfs-client") (kernel-arguments '("ip=dhcp")) + (initrd-modules + (cons "e1000" %base-initrd-modules)) + (initrd (lambda (file-systems . rest) + ;; Create a standard initrd but set up networking + ;; with the parameters QEMU expects by default. + (apply base-initrd + file-systems + #:qemu-networking? #t + rest))) (file-systems (cons (file-system (type "nfs") (mount-point "/") (device ":/export") - (options "addr=127.0.0.1,vers=4.2")) + (options "addr=127.0.0.1")) %base-file-systems))) #:requirements '(nscd) #:imported-modules '((gnu services herd) @@ -313,102 +317,80 @@ directories can be mounted.") (mkdir #$output) (chdir #$output) - (test-begin "start-nfs-boot-test") - - ;;; Start up NFS server host. + (test-begin "start nfs-root server") (mkdir "/tmp/server") (define server-marionette (make-marionette (list #$(virtual-machine - nfs-root-server-os - ;(operating-system nfs-root-server-os) - ;(port-forwardings '( ; (111 . 111) - ; (2049 . 2049) - ; (20001 . 20001) - ; (20002 . 20002))) -)) + nfs-root-server-os)) #:socket-directory "/tmp/server")) - (marionette-eval - '(begin - (use-modules (gnu services herd)) - (current-output-port - (open-file "/dev/console" "w0")) - ;; FIXME: Instead statfs "/" and "/export" and wait until they - ;; are different file systems. But Guile doesn't seem to have - ;; statfs. - (sleep 5) - (chmod "/export" #o777) - (symlink "/gnu" "/export/gnu") - (start-service 'nscd) - (start-service 'networking) - (start-service 'nfs)) - server-marionette) - - ;;; Wait for the NFS services to be up and running. + (test-assert "nfs-root server boots" + (marionette-eval + '(begin + (use-modules (gnu services herd)) + (current-output-port + (open-file "/dev/console" "w0")) + (chmod "/export" #o777) + (symlink "/gnu" "/export/gnu") + (start-service 'nscd) + (start-service 'networking) + (start-service 'nfs) + #t) + server-marionette)) (test-assert "nfs services are running" - (wait-for-file "/var/run/rpc.statd.pid" server-marionette)) - - (test-assert "NFS port is ready" - (wait-for-tcp-port 2049 server-marionette)) - - (test-assert "NFS statd port is ready" - (wait-for-tcp-port 20002 server-marionette)) + (and (wait-for-file "/var/run/rpc.statd.pid" server-marionette) + (marionette-eval + '(zero? (system* (string-append #$nfs-utils "/sbin/showmount") + "-e" "nfs-server")) + server-marionette))) - (test-assert "NFS mountd port is ready" - (wait-for-tcp-port 20001 server-marionette)) - - ;;; FIXME: (test-assert "NFS portmapper port is ready" - ;;; FIXME: (wait-for-tcp-port 111 server-marionette)) + (test-end) - ;;; Start up NFS client host. + (test-begin "start nfs-root client") + (mkdir "/tmp/client") (define client-marionette (make-marionette (list #$(virtual-machine - nfs-root-client-os - ;(port-forwardings '((111 . 111) - ; (2049 . 2049) - ; (20001 . 20001) - ; (20002 . 20002))) - )))) - - (marionette-eval - '(begin - (use-modules (gnu services herd)) - (use-modules (rnrs io ports)) - - (current-output-port - (open-file "/dev/console" "w0")) - (let ((content (call-with-input-file "/proc/mounts" get-string-all))) - (call-with-output-file "/mounts.new" - (lambda (port) - (display content port)))) - (chmod "/mounts.new" #o777) - (rename-file "/mounts.new" "/mounts")) - client-marionette) - - (test-assert "nfs-root-client booted") + nfs-root-client-os)) + #:socket-directory "/tmp/client")) - ;;; Check whether NFS client host communicated with NFS server host. - - (test-assert "nfs client deposited file" - (wait-for-file "/export/mounts" server-marionette)) - (marionette-eval - '(begin - (current-output-port - (open-file "/dev/console" "w0")) - (call-with-input-file "/export/mounts" display)) - server-marionette) + (test-assert "nfs-root client boots" + (marionette-eval + '(begin + (use-modules (gnu services herd)) + (use-modules (rnrs io ports)) + (current-output-port + (open-file "/dev/console" "w0")) + (let ((content (call-with-input-file "/proc/mounts" get-string-all))) + (display content) + (newline) + (call-with-output-file "/mounts" + (lambda (port) + (display content port))) + content)) + client-marionette)) + + (test-assert "wait nfs-root client booted and deposited mounts file" + (and (wait-for-file "/export/mounts" server-marionette))) + (test-assert "nfs-root client booted and deposited mounts file" + (marionette-eval + '(begin + (current-output-port + (open-file "/dev/console" "w0")) + (call-with-input-file "/export/mounts" + get-string-all)) + server-marionette)) (test-end) (exit (= (test-runner-fail-count (test-runner-current)) 0))))) - (gexp->derivation "nfs-root-fs-test" test)) + (gexp->derivation "nfs-root-test" test)) -(define %test-nfs-root-fs +(define %test-nfs-root (system-test - (name "nfs-root-fs") + (name "nfs-root") (description "Test that an NFS server can be started and the exported -directory can be used as root file system.") - (value (run-nfs-root-fs-test)))) +directory can be used as root file system of an NFS client.") + (value (run-nfs-root-test)))) -- 2.31.1 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* bug#47744: nfs-root-fs test is failing 2021-04-18 18:29 ` Stefan @ 2021-04-22 9:30 ` Mathieu Othacehe 2021-04-22 19:28 ` bug#47956: " Stefan 0 siblings, 1 reply; 10+ messages in thread From: Mathieu Othacehe @ 2021-04-22 9:30 UTC (permalink / raw) To: Stefan; +Cc: 47744 Hello Stefan, Thanks for having a look! > I’m now sure that this test has never been working. I also share the impression that this test never worked. Plus the actual version has a lot of FIXME and commented lines that scare me off. > With the attached changes, the NFS server side tests are passing, but > the NFS client is missing an own IP address when trying to mount its > root file-system, which results in this error when executing “make > check-system TESTS=nfs-root”: I think I'll remove the "nfs-root-fs" test for now. To fulfil their role its important that the system test are functional. Otherwise its hard to distinguish a test failing because of a regression from a test that has always been broken. Any objections? Thanks, Mathieu ^ permalink raw reply [flat|nested] 10+ messages in thread
* bug#47956: nfs-root-fs test is failing 2021-04-22 9:30 ` Mathieu Othacehe @ 2021-04-22 19:28 ` Stefan 2021-04-25 17:30 ` Mathieu Othacehe 0 siblings, 1 reply; 10+ messages in thread From: Stefan @ 2021-04-22 19:28 UTC (permalink / raw) To: Mathieu Othacehe; +Cc: 47956 Hi Mathieu! > I think I'll remove the "nfs-root-fs" test for now. > Any objections? I’d like to ask you to apply my patch. It paves the way for using nfs-shares inside virtual machines, which is not possible today. This is certainly a good impovement. The patch also fixes the nfs-root test. It would certainly work, if networking in qemu during the bootup works and if the two virtual machines are able to communicate via IP to each other. It woud be sufficient to comment out the nfs-root test or even only the client part of it. But please don’t throw it away! If either the kernel or the initrd will be changed to support networking in qemu, then this test needs possibly only small adjustments to start working. For the networking part at bootup it is enough to add CONFIG_E1000 when building linux-libre. Is that an acceptable change? If so then I could give it another try. Bye Stefan ^ permalink raw reply [flat|nested] 10+ messages in thread
* bug#47956: nfs-root-fs test is failing 2021-04-22 19:28 ` bug#47956: " Stefan @ 2021-04-25 17:30 ` Mathieu Othacehe 2021-05-31 20:03 ` Stefan 0 siblings, 1 reply; 10+ messages in thread From: Mathieu Othacehe @ 2021-04-25 17:30 UTC (permalink / raw) To: Stefan; +Cc: 47956 Hello Stefan, > I’d like to ask you to apply my patch. It paves the way for using > nfs-shares inside virtual machines, which is not possible today. This > is certainly a good impovement. OK, I'll first need to dive into this NFS subject that I'm not familiar with. In the meantime, let's not block the release with that test failure which doesn't look like a regression. Unblocking the 1.3.0 release ticket. Thanks, Mathieu ^ permalink raw reply [flat|nested] 10+ messages in thread
* bug#47956: nfs-root-fs test is failing 2021-04-25 17:30 ` Mathieu Othacehe @ 2021-05-31 20:03 ` Stefan 0 siblings, 0 replies; 10+ messages in thread From: Stefan @ 2021-05-31 20:03 UTC (permalink / raw) To: Mathieu Othacehe, Danny Milosavljevic; +Cc: 47956 [-- Attachment #1: Type: text/plain, Size: 326 bytes --] Hi Mathieu and Danny! I got the nfs-root test working now with the attached patch. It makes use of the (modify-linux) function from my other patch series¹, as the current kernel for x86 is not capable to use an NFS root file system. Bye Stefan ¹ <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=48314> [-- Attachment #2: nfs-root-for-virtual-machines.patch --] [-- Type: application/octet-stream, Size: 14053 bytes --] NFS root for virtual machines. From: Stefan <stefan-guix@vodafonemail.de> gnu: system: Allow a root file-system over NFS for virtual machines. gnu: tests: Improve the test for a root file-system over NFS. However, the test is still failing. * gnu/system/vm.scm (virtualized-operating-system): Allow root file-systems over NFS. (system-qemu-image/shared-store-script): Respect the configured root file-system-device. * gnu/tests/nfs.scm (run-nfs-root-fs-test): Renamed to … (run-nfs-root-test): … this. Cleanup and improvements. (%test-nfs-root-fs): Renamed to … (%test-nfs-root): … this. Renamed the test from "nfs-root-fs" to "nfs-root". --- gnu/system/vm.scm | 30 ++++++-- gnu/tests/nfs.scm | 188 ++++++++++++++++++++++------------------------------- 2 files changed, 100 insertions(+), 118 deletions(-) diff --git a/gnu/system/vm.scm b/gnu/system/vm.scm index 97adfa12fa..be0a4695f2 100644 --- a/gnu/system/vm.scm +++ b/gnu/system/vm.scm @@ -608,7 +608,8 @@ environment with the store shared with the host. MAPPINGS is a list of (let ((target (file-system-mount-point fs)) (source (file-system-device fs))) (or (string=? target (%store-prefix)) - (string=? target "/") + (and (string=? target "/") + (not (string=? (file-system-type fs) "nfs"))) (and (string? source) (string-prefix? "/dev/" source)) @@ -618,14 +619,20 @@ environment with the store shared with the host. MAPPINGS is a list of (uuid? source)))))) (operating-system-file-systems os))) - (define virtual-file-systems - (cons (file-system - (mount-point "/") - (device "/dev/vda1") - (type "ext4")) + (define (add-missing-root-fs user-file-systems) + (if (null? (filter (lambda (fs) + (string=? (file-system-mount-point fs) "/")) + user-file-systems)) + (cons (file-system + (mount-point "/") + (device "/dev/vda1") + (type "ext4")) + user-file-systems) + user-file-systems)) - (append (map mapping->file-system mappings) - user-file-systems))) + (define virtual-file-systems + (append (map mapping->file-system mappings) + (add-missing-root-fs user-file-systems))) (operating-system (inherit os) @@ -754,7 +761,12 @@ it is mostly useful when FULL-BOOT? is true." #:disk-image-size disk-image-size))) (define kernel-arguments #~(list #$@(if graphic? #~() #~("console=ttyS0")) - #+@(operating-system-kernel-arguments os "/dev/vda1"))) + #+@(operating-system-kernel-arguments + os + (file-system-device + (first (filter (lambda (fs) + (string=? (file-system-mount-point fs) "/")) + (operating-system-file-systems os))))))) (define qemu-exec #~(list #+(file-append qemu "/bin/" diff --git a/gnu/tests/nfs.scm b/gnu/tests/nfs.scm index 9b2b785176..2190e3a715 100644 --- a/gnu/tests/nfs.scm +++ b/gnu/tests/nfs.scm @@ -27,12 +27,14 @@ #:use-module (gnu bootloader grub) #:use-module (gnu system) #:use-module (gnu system file-systems) + #:use-module (gnu system linux-initrd) #:use-module (gnu system shadow) #:use-module (gnu system vm) #:use-module (gnu services) #:use-module (gnu services base) #:use-module (gnu services nfs) #:use-module (gnu services networking) + #:use-module (gnu packages linux) #:use-module (gnu packages onc-rpc) #:use-module (gnu packages nfs) #:use-module (guix gexp) @@ -40,7 +42,7 @@ #:use-module (guix monads) #:export (%test-nfs %test-nfs-server - %test-nfs-root-fs)) + %test-nfs-root)) (define %base-os (operating-system @@ -265,44 +267,53 @@ directories can be mounted.") (value (run-nfs-server-test)))) \f -(define (run-nfs-root-fs-test) +(define (run-nfs-root-test) "Run a test of an OS mounting its root file system via NFS." (define nfs-root-server-os (marionette-operating-system (operating-system (inherit %nfs-os) (services - (modify-services (operating-system-user-services %nfs-os) - (nfs-service-type config => - (nfs-configuration - (debug '(nfs nfsd mountd)) - ;;; Note: Adding the following line causes Guix to hang. - ;(rpcmountd-port 20001) - ;;; Note: Adding the following line causes Guix to hang. - ;(rpcstatd-port 20002) ; FIXME: Set broadcast port AND listening port. - (nfsd-port 2049) - (nfs-versions '("4.2")) - (exports '(("/export" - "*(rw,insecure,no_subtree_check,crossmnt,fsid=root,no_root_squash,insecure,async)")))))))) + (cons* + (static-networking-service "ens3" "10.0.0.15" + #:netmask "255.255.255.0") + (modify-services (operating-system-user-services %nfs-os) + (delete dhcp-client-service-type) + (nfs-service-type config => + (nfs-configuration + (exports '(("/export" + "*(rw,fsid=0,async,no_wdelay,no_root_squash,insecure,no_subtree_check,crossmnt)"))))))))) #:requirements '(nscd) #:imported-modules '((gnu services herd) (guix combinators)))) (define nfs-root-client-os - (marionette-operating-system - (operating-system - (inherit (simple-operating-system (service dhcp-client-service-type))) - (kernel-arguments '("ip=dhcp")) - (file-systems (cons - (file-system - (type "nfs") - (mount-point "/") - (device ":/export") - (options "addr=127.0.0.1,vers=4.2")) - %base-file-systems))) - #:requirements '(nscd) - #:imported-modules '((gnu services herd) - (guix combinators)))) + (let ((base-os (simple-operating-system (service dhcp-client-service-type)))) + (marionette-operating-system + (operating-system + (inherit base-os) + (host-name "nfs-client") + (kernel (modify-linux #:linux (operating-system-kernel base-os) + #:configs '("CONFIG_E1000=y" + "CONFIG_ROOT_NFS=y" + "CONFIG_NFS_FS=y" + "CONFIG_NFS_V4=y"))) + (kernel-arguments '("ip=10.0.0.16:::255.255.255.0:::off")) + (file-systems (cons + (file-system + (type "nfs") + (mount-point "/") + (device ":/") + (options "addr=10.0.0.15,vers=4")) + %base-file-systems)) + (services (cons* + (static-networking-service "eth0" "10.0.0.16" + #:netmask "255.255.255.0" + #:requirement '()) + %base-services))) + #:requirements '(nscd) + #:imported-modules '((gnu services herd) + (guix combinators))))) (define test (with-imported-modules '((gnu build marionette)) @@ -313,102 +324,61 @@ directories can be mounted.") (mkdir #$output) (chdir #$output) - (test-begin "start-nfs-boot-test") - - ;;; Start up NFS server host. + (test-begin "start nfs-root server") (mkdir "/tmp/server") (define server-marionette - (make-marionette (list #$(virtual-machine - nfs-root-server-os - ;(operating-system nfs-root-server-os) - ;(port-forwardings '( ; (111 . 111) - ; (2049 . 2049) - ; (20001 . 20001) - ; (20002 . 20002))) -)) - #:socket-directory "/tmp/server")) + (make-marionette + (cons* #$(virtual-machine nfs-root-server-os) + '("-nic" "socket,listen=127.0.0.1:37915,model=e1000,mac=52:54:98:76:54:32")) + #:socket-directory "/tmp/server")) - (marionette-eval - '(begin - (use-modules (gnu services herd)) - (current-output-port - (open-file "/dev/console" "w0")) - ;; FIXME: Instead statfs "/" and "/export" and wait until they - ;; are different file systems. But Guile doesn't seem to have - ;; statfs. - (sleep 5) - (chmod "/export" #o777) - (symlink "/gnu" "/export/gnu") - (start-service 'nscd) - (start-service 'networking) - (start-service 'nfs)) - server-marionette) - - ;;; Wait for the NFS services to be up and running. + (test-assert "nfs-root server boots" + (marionette-eval + '(begin + (use-modules (gnu services herd)) + (current-output-port + (open-file "/dev/console" "w0")) + (chmod "/export" #o777) + (mkdir-p "/export/var/run") + (mkdir-p "/export/run") + (call-with-output-file "/export/run/hello" + (lambda (port) + (display "Hello, World!" port))) + (and (start-service 'networking) + (start-service 'nscd) + (start-service 'nfs))) + server-marionette)) (test-assert "nfs services are running" - (wait-for-file "/var/run/rpc.statd.pid" server-marionette)) - - (test-assert "NFS port is ready" - (wait-for-tcp-port 2049 server-marionette)) + (and (wait-for-file "/var/run/rpc.statd.pid" server-marionette) + (marionette-eval + '(zero? (system* (string-append #$nfs-utils "/sbin/showmount") + "-e" "nfs-server")) + server-marionette))) - (test-assert "NFS statd port is ready" - (wait-for-tcp-port 20002 server-marionette)) - - (test-assert "NFS mountd port is ready" - (wait-for-tcp-port 20001 server-marionette)) - - ;;; FIXME: (test-assert "NFS portmapper port is ready" - ;;; FIXME: (wait-for-tcp-port 111 server-marionette)) + (test-end) - ;;; Start up NFS client host. + (test-begin "start nfs-root client") + (mkdir "/tmp/client") (define client-marionette - (make-marionette (list #$(virtual-machine - nfs-root-client-os - ;(port-forwardings '((111 . 111) - ; (2049 . 2049) - ; (20001 . 20001) - ; (20002 . 20002))) - )))) - - (marionette-eval - '(begin - (use-modules (gnu services herd)) - (use-modules (rnrs io ports)) - - (current-output-port - (open-file "/dev/console" "w0")) - (let ((content (call-with-input-file "/proc/mounts" get-string-all))) - (call-with-output-file "/mounts.new" - (lambda (port) - (display content port)))) - (chmod "/mounts.new" #o777) - (rename-file "/mounts.new" "/mounts")) - client-marionette) - - (test-assert "nfs-root-client booted") - - ;;; Check whether NFS client host communicated with NFS server host. + (make-marionette + (cons* #$(virtual-machine nfs-root-client-os) + '("-nic" "socket,connect=127.0.0.1:37915,model=e1000,mac=52:54:98:76:54:33")) + #:socket-directory "/tmp/client")) - (test-assert "nfs client deposited file" - (wait-for-file "/export/mounts" server-marionette)) - (marionette-eval - '(begin - (current-output-port - (open-file "/dev/console" "w0")) - (call-with-input-file "/export/mounts" display)) - server-marionette) + (test-assert "wait nfs-root client booted and sees hello file" + (wait-for-file "/run/hello" client-marionette)) (test-end) (exit (= (test-runner-fail-count (test-runner-current)) 0))))) - (gexp->derivation "nfs-root-fs-test" test)) + (gexp->derivation "nfs-root-test" test)) -(define %test-nfs-root-fs +(define %test-nfs-root (system-test - (name "nfs-root-fs") + (name "nfs-root") (description "Test that an NFS server can be started and the exported -directory can be used as root file system.") - (value (run-nfs-root-fs-test)))) +directory can be used as root file system of an NFS client.") + (value (run-nfs-root-test)))) ^ permalink raw reply related [flat|nested] 10+ messages in thread
* bug#47744: nfs-root-fs test is failing 2021-04-13 8:10 bug#47744: nfs-root-fs test is failing Mathieu Othacehe 2021-04-17 17:19 ` Stefan @ 2021-09-26 12:57 ` Stefan 2021-09-27 22:34 ` Stefan 1 sibling, 1 reply; 10+ messages in thread From: Stefan @ 2021-09-26 12:57 UTC (permalink / raw) To: 47744 Hi! A solution for the failing nfs-root test is part of <http://issues.guix.gnu.org/48314>. I think this ticket can be closed. Bye Stefan ^ permalink raw reply [flat|nested] 10+ messages in thread
* bug#47744: nfs-root-fs test is failing 2021-09-26 12:57 ` bug#47744: " Stefan @ 2021-09-27 22:34 ` Stefan 2021-12-23 10:01 ` bug#47744: [PATCH] nfs-root " Mathieu Othacehe 0 siblings, 1 reply; 10+ messages in thread From: Stefan @ 2021-09-27 22:34 UTC (permalink / raw) To: 47744 [-- Attachment #1: Type: text/plain, Size: 227 bytes --] Hi! Sorry, that was not quite correct. I sent a patch months back, which did not make it into this ticket, which solved this ticket but depends on <http://issues.guix.gnu.org/48314>. Here it is again. Bye Stefan [-- Attachment #2: nfs-root-for-virtual-machines.patch --] [-- Type: application/octet-stream, Size: 14053 bytes --] NFS root for virtual machines. From: Stefan <stefan-guix@vodafonemail.de> gnu: system: Allow a root file-system over NFS for virtual machines. gnu: tests: Improve the test for a root file-system over NFS. However, the test is still failing. * gnu/system/vm.scm (virtualized-operating-system): Allow root file-systems over NFS. (system-qemu-image/shared-store-script): Respect the configured root file-system-device. * gnu/tests/nfs.scm (run-nfs-root-fs-test): Renamed to … (run-nfs-root-test): … this. Cleanup and improvements. (%test-nfs-root-fs): Renamed to … (%test-nfs-root): … this. Renamed the test from "nfs-root-fs" to "nfs-root". --- gnu/system/vm.scm | 30 ++++++-- gnu/tests/nfs.scm | 188 ++++++++++++++++++++++------------------------------- 2 files changed, 100 insertions(+), 118 deletions(-) diff --git a/gnu/system/vm.scm b/gnu/system/vm.scm index 97adfa12fa..be0a4695f2 100644 --- a/gnu/system/vm.scm +++ b/gnu/system/vm.scm @@ -608,7 +608,8 @@ environment with the store shared with the host. MAPPINGS is a list of (let ((target (file-system-mount-point fs)) (source (file-system-device fs))) (or (string=? target (%store-prefix)) - (string=? target "/") + (and (string=? target "/") + (not (string=? (file-system-type fs) "nfs"))) (and (string? source) (string-prefix? "/dev/" source)) @@ -618,14 +619,20 @@ environment with the store shared with the host. MAPPINGS is a list of (uuid? source)))))) (operating-system-file-systems os))) - (define virtual-file-systems - (cons (file-system - (mount-point "/") - (device "/dev/vda1") - (type "ext4")) + (define (add-missing-root-fs user-file-systems) + (if (null? (filter (lambda (fs) + (string=? (file-system-mount-point fs) "/")) + user-file-systems)) + (cons (file-system + (mount-point "/") + (device "/dev/vda1") + (type "ext4")) + user-file-systems) + user-file-systems)) - (append (map mapping->file-system mappings) - user-file-systems))) + (define virtual-file-systems + (append (map mapping->file-system mappings) + (add-missing-root-fs user-file-systems))) (operating-system (inherit os) @@ -754,7 +761,12 @@ it is mostly useful when FULL-BOOT? is true." #:disk-image-size disk-image-size))) (define kernel-arguments #~(list #$@(if graphic? #~() #~("console=ttyS0")) - #+@(operating-system-kernel-arguments os "/dev/vda1"))) + #+@(operating-system-kernel-arguments + os + (file-system-device + (first (filter (lambda (fs) + (string=? (file-system-mount-point fs) "/")) + (operating-system-file-systems os))))))) (define qemu-exec #~(list #+(file-append qemu "/bin/" diff --git a/gnu/tests/nfs.scm b/gnu/tests/nfs.scm index 9b2b785176..2190e3a715 100644 --- a/gnu/tests/nfs.scm +++ b/gnu/tests/nfs.scm @@ -27,12 +27,14 @@ #:use-module (gnu bootloader grub) #:use-module (gnu system) #:use-module (gnu system file-systems) + #:use-module (gnu system linux-initrd) #:use-module (gnu system shadow) #:use-module (gnu system vm) #:use-module (gnu services) #:use-module (gnu services base) #:use-module (gnu services nfs) #:use-module (gnu services networking) + #:use-module (gnu packages linux) #:use-module (gnu packages onc-rpc) #:use-module (gnu packages nfs) #:use-module (guix gexp) @@ -40,7 +42,7 @@ #:use-module (guix monads) #:export (%test-nfs %test-nfs-server - %test-nfs-root-fs)) + %test-nfs-root)) (define %base-os (operating-system @@ -265,44 +267,53 @@ directories can be mounted.") (value (run-nfs-server-test)))) \f -(define (run-nfs-root-fs-test) +(define (run-nfs-root-test) "Run a test of an OS mounting its root file system via NFS." (define nfs-root-server-os (marionette-operating-system (operating-system (inherit %nfs-os) (services - (modify-services (operating-system-user-services %nfs-os) - (nfs-service-type config => - (nfs-configuration - (debug '(nfs nfsd mountd)) - ;;; Note: Adding the following line causes Guix to hang. - ;(rpcmountd-port 20001) - ;;; Note: Adding the following line causes Guix to hang. - ;(rpcstatd-port 20002) ; FIXME: Set broadcast port AND listening port. - (nfsd-port 2049) - (nfs-versions '("4.2")) - (exports '(("/export" - "*(rw,insecure,no_subtree_check,crossmnt,fsid=root,no_root_squash,insecure,async)")))))))) + (cons* + (static-networking-service "ens3" "10.0.0.15" + #:netmask "255.255.255.0") + (modify-services (operating-system-user-services %nfs-os) + (delete dhcp-client-service-type) + (nfs-service-type config => + (nfs-configuration + (exports '(("/export" + "*(rw,fsid=0,async,no_wdelay,no_root_squash,insecure,no_subtree_check,crossmnt)"))))))))) #:requirements '(nscd) #:imported-modules '((gnu services herd) (guix combinators)))) (define nfs-root-client-os - (marionette-operating-system - (operating-system - (inherit (simple-operating-system (service dhcp-client-service-type))) - (kernel-arguments '("ip=dhcp")) - (file-systems (cons - (file-system - (type "nfs") - (mount-point "/") - (device ":/export") - (options "addr=127.0.0.1,vers=4.2")) - %base-file-systems))) - #:requirements '(nscd) - #:imported-modules '((gnu services herd) - (guix combinators)))) + (let ((base-os (simple-operating-system (service dhcp-client-service-type)))) + (marionette-operating-system + (operating-system + (inherit base-os) + (host-name "nfs-client") + (kernel (modify-linux #:linux (operating-system-kernel base-os) + #:configs '("CONFIG_E1000=y" + "CONFIG_ROOT_NFS=y" + "CONFIG_NFS_FS=y" + "CONFIG_NFS_V4=y"))) + (kernel-arguments '("ip=10.0.0.16:::255.255.255.0:::off")) + (file-systems (cons + (file-system + (type "nfs") + (mount-point "/") + (device ":/") + (options "addr=10.0.0.15,vers=4")) + %base-file-systems)) + (services (cons* + (static-networking-service "eth0" "10.0.0.16" + #:netmask "255.255.255.0" + #:requirement '()) + %base-services))) + #:requirements '(nscd) + #:imported-modules '((gnu services herd) + (guix combinators))))) (define test (with-imported-modules '((gnu build marionette)) @@ -313,102 +324,61 @@ directories can be mounted.") (mkdir #$output) (chdir #$output) - (test-begin "start-nfs-boot-test") - - ;;; Start up NFS server host. + (test-begin "start nfs-root server") (mkdir "/tmp/server") (define server-marionette - (make-marionette (list #$(virtual-machine - nfs-root-server-os - ;(operating-system nfs-root-server-os) - ;(port-forwardings '( ; (111 . 111) - ; (2049 . 2049) - ; (20001 . 20001) - ; (20002 . 20002))) -)) - #:socket-directory "/tmp/server")) + (make-marionette + (cons* #$(virtual-machine nfs-root-server-os) + '("-nic" "socket,listen=127.0.0.1:37915,model=e1000,mac=52:54:98:76:54:32")) + #:socket-directory "/tmp/server")) - (marionette-eval - '(begin - (use-modules (gnu services herd)) - (current-output-port - (open-file "/dev/console" "w0")) - ;; FIXME: Instead statfs "/" and "/export" and wait until they - ;; are different file systems. But Guile doesn't seem to have - ;; statfs. - (sleep 5) - (chmod "/export" #o777) - (symlink "/gnu" "/export/gnu") - (start-service 'nscd) - (start-service 'networking) - (start-service 'nfs)) - server-marionette) - - ;;; Wait for the NFS services to be up and running. + (test-assert "nfs-root server boots" + (marionette-eval + '(begin + (use-modules (gnu services herd)) + (current-output-port + (open-file "/dev/console" "w0")) + (chmod "/export" #o777) + (mkdir-p "/export/var/run") + (mkdir-p "/export/run") + (call-with-output-file "/export/run/hello" + (lambda (port) + (display "Hello, World!" port))) + (and (start-service 'networking) + (start-service 'nscd) + (start-service 'nfs))) + server-marionette)) (test-assert "nfs services are running" - (wait-for-file "/var/run/rpc.statd.pid" server-marionette)) - - (test-assert "NFS port is ready" - (wait-for-tcp-port 2049 server-marionette)) + (and (wait-for-file "/var/run/rpc.statd.pid" server-marionette) + (marionette-eval + '(zero? (system* (string-append #$nfs-utils "/sbin/showmount") + "-e" "nfs-server")) + server-marionette))) - (test-assert "NFS statd port is ready" - (wait-for-tcp-port 20002 server-marionette)) - - (test-assert "NFS mountd port is ready" - (wait-for-tcp-port 20001 server-marionette)) - - ;;; FIXME: (test-assert "NFS portmapper port is ready" - ;;; FIXME: (wait-for-tcp-port 111 server-marionette)) + (test-end) - ;;; Start up NFS client host. + (test-begin "start nfs-root client") + (mkdir "/tmp/client") (define client-marionette - (make-marionette (list #$(virtual-machine - nfs-root-client-os - ;(port-forwardings '((111 . 111) - ; (2049 . 2049) - ; (20001 . 20001) - ; (20002 . 20002))) - )))) - - (marionette-eval - '(begin - (use-modules (gnu services herd)) - (use-modules (rnrs io ports)) - - (current-output-port - (open-file "/dev/console" "w0")) - (let ((content (call-with-input-file "/proc/mounts" get-string-all))) - (call-with-output-file "/mounts.new" - (lambda (port) - (display content port)))) - (chmod "/mounts.new" #o777) - (rename-file "/mounts.new" "/mounts")) - client-marionette) - - (test-assert "nfs-root-client booted") - - ;;; Check whether NFS client host communicated with NFS server host. + (make-marionette + (cons* #$(virtual-machine nfs-root-client-os) + '("-nic" "socket,connect=127.0.0.1:37915,model=e1000,mac=52:54:98:76:54:33")) + #:socket-directory "/tmp/client")) - (test-assert "nfs client deposited file" - (wait-for-file "/export/mounts" server-marionette)) - (marionette-eval - '(begin - (current-output-port - (open-file "/dev/console" "w0")) - (call-with-input-file "/export/mounts" display)) - server-marionette) + (test-assert "wait nfs-root client booted and sees hello file" + (wait-for-file "/run/hello" client-marionette)) (test-end) (exit (= (test-runner-fail-count (test-runner-current)) 0))))) - (gexp->derivation "nfs-root-fs-test" test)) + (gexp->derivation "nfs-root-test" test)) -(define %test-nfs-root-fs +(define %test-nfs-root (system-test - (name "nfs-root-fs") + (name "nfs-root") (description "Test that an NFS server can be started and the exported -directory can be used as root file system.") - (value (run-nfs-root-fs-test)))) +directory can be used as root file system of an NFS client.") + (value (run-nfs-root-test)))) ^ permalink raw reply related [flat|nested] 10+ messages in thread
* bug#47744: [PATCH] nfs-root test is failing 2021-09-27 22:34 ` Stefan @ 2021-12-23 10:01 ` Mathieu Othacehe 0 siblings, 0 replies; 10+ messages in thread From: Mathieu Othacehe @ 2021-12-23 10:01 UTC (permalink / raw) To: Stefan; +Cc: 47744-done Hello Stefan, > Sorry, that was not quite correct. I sent a patch months back, which did not > make it into this ticket, which solved this ticket but depends on > <http://issues.guix.gnu.org/48314>. I turned the nfs-root-fs test into an nfs-full test that is based on your patch. It spawns a VM with an NFS server and another VM that mounts it. It should now be possible to reintroduce an nfs-root-fs test so that the client VM uses an NFS as its root directory. Thanks, Mathieu ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2021-12-23 10:12 UTC | newest] Thread overview: 10+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2021-04-13 8:10 bug#47744: nfs-root-fs test is failing Mathieu Othacehe 2021-04-17 17:19 ` Stefan 2021-04-18 18:29 ` Stefan 2021-04-22 9:30 ` Mathieu Othacehe 2021-04-22 19:28 ` bug#47956: " Stefan 2021-04-25 17:30 ` Mathieu Othacehe 2021-05-31 20:03 ` Stefan 2021-09-26 12:57 ` bug#47744: " Stefan 2021-09-27 22:34 ` Stefan 2021-12-23 10:01 ` bug#47744: [PATCH] nfs-root " Mathieu Othacehe
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/guix.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).