Value in adding Shepherd requirements to file-systems entries?

Value in adding Shepherd requirements to file-systems entries?

[Richard Sent]

Hi Guix!

I wanted to ask the Guix community for their thoughts on improving the
support for adding networked file systems to an operating-system
declaration.

For some context, I started tackling adding CIFS support to file-system
declarations, but I've hit a snag. CIFS is a networked file system, but
Guix mounts all file systems specified in
(operating-system-file-systems) either when booting the system from the
initrd or as shepherd services after boot that depend on
'root-file-system and 'udev.

Either way, these run before any networking service has been
initialized. Ergo, a samba share like //192.168.1.102/share won't be
found. (At least, not on a wireless network. Perhaps the timing is
different for wired ones.)

Obviously, adding shepherd requirements to needed-at-boot? file systems
isn't possible. However, I think it should be possible to add shepherd
services to other file system entries.

(And yet, NFS is allegedly supported, although I can't figure out the
mechanism for that and don't have one set up on my LAN for testing.)

Before hacking away at this myself, I'd like to get other people's
thoughts on the best way to proceed. Do others agree that (file-system)
entries should support networked devices? Should this be transparent
depending on the type, or require explicit configuration?

e.g.

--8<---------------cut here---------------start------------->8---
(file-system
  (device "//192.168.1.102/share")
  (options "guest")
  (mount-point "/mnt/share")
  (type "cifs")
  ;; Should we explicitly require network, or implicitly add it from
  ;; the type? If the latter, what to do about Avahi?
  (requirement 'networking)
  (mount-may-fail? #t)
  (create-mount-point? #t))
--8<---------------cut here---------------end--------------->8---

I could see this being challenging since it's not immediately clear to
me what particular file-system-* service, if any, is provisioning
'file-systems, which other shepherd requirements the user may specify
can depend on. I imagine adding a requirement to the wrong file-system
could easily cause a deadlock.

I know a custom shepherd service could be used to run, say, mount.cifs
from userspace after networking is provisioned, but in my opinion it
would be cleaner to encapsulate within the existing file-system block of
the operating system.

-- 
Take it easy,
Richard Sent
Making my computer weirder one commit at a time.

Re: Value in adding Shepherd requirements to file-systems entries?

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Richard,

On Mon, Apr 22 2024, Richard Sent wrote:

> NFS is allegedly supported

Someone once gave me this service [1] to mount a file-system declared
with (mount? #f). [2] It's been working ever since.

Kind regards
Felix

[1] https://codeberg.org/lechner/system-config/src/commit/0131082ff0eb3f1262544f7799d291324ba08282/host/lechner-desktop/operating-system.scm#L209-L222
[2] https://codeberg.org/lechner/system-config/src/commit/0131082ff0eb3f1262544f7799d291324ba08282/host/lechner-desktop/operating-system.scm#L130-L139

Re: Value in adding Shepherd requirements to file-systems entries?

[Richard Sent]

Hi Felix,

> Someone once gave me this service [1] to mount a file-system declared
> with (mount? #f). [2] It's been working ever since.

Thanks! I know custom services can be made that can work on a
case-by-case basis. I was curious about the value of encapsulating that
logic within an operating-system file-systems field and reusing the
existing code of file-system-shepherd-service in (gnu services base) and
mount-file-system in (gnu build file-system).

My comment on NFS support is more about how mount-file-system supports
mounting NFS file-system records, but the existing code that actually
uses mount-file-system tries mounting all file systems before networking
has begun. Ergo, the fact that mount-file-system supports NFS seems a
bit extraneous at present, at least insofar as I can decipher.

I submitted a patch for what I'm thinking at
https://issues.guix.gnu.org/70542. If this winds up merged I believe
your code could be rewritten to remove [1] and replace [2] with

--8<---------------cut here---------------start------------->8---
(file-system
   (device "wallace-server.local:/acct")
   (mount-point "/acct")
   (type "nfs")
   (requirement '(avahi-daemon)) ;resolve .local
   ;; (flags '(no-atime no-dev no-exec read-only))
   ;; (options "proto=tcp6,timeo=300,nolock")
   (check? #f)
   (mount-may-fail? #t)
   (create-mount-point? #t))
--8<---------------cut here---------------end--------------->8---

(I don't have an NFS system on my LAN to test so no promises)

Hopefully that shows what I'm thinking. If anyone has thoughts I'd love
to hear it, either here or in the patch depending on what's appropriate.

-- 
Take it easy,
Richard Sent
Making my computer weirder one commit at a time.

Re: Value in adding Shepherd requirements to file-systems entries?

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Richard,

On Tue, Apr 23 2024, Richard Sent wrote:

> I submitted a patch [..] at https://issues.guix.gnu.org/70542. If this
> winds up merged I believe your code could be rewritten [as]
>
> --8<---------------cut here---------------start------------->8---
> (file-system
>    (device "wallace-server.local:/acct")
>    (mount-point "/acct")
>    (type "nfs")
>    (requirement '(avahi-daemon)) ;resolve .local
>    ;; (flags '(no-atime no-dev no-exec read-only))
>    ;; (options "proto=tcp6,timeo=300,nolock")
>    (check? #f)
>    (mount-may-fail? #t)
>    (create-mount-point? #t))
> --8<---------------cut here---------------end--------------->8---

Wow, that works!

Since this short form is optional and I can always switch back to the
previous version, v2 of that patch should be merged without further
delay.

Thank you for your most valuable contribution!

Kind regards,
Felix

P.S. The code above should read (requirements ...) in the plural.

Re: Value in adding Shepherd requirements to file-systems entries?

[Attila Lendvai]

> P.S. The code above should read (requirements ...) in the plural.

inside shepherd there's a bit of anomaly, but it's called requirement in the public API, and also in the guix side of the config; i.e. it's not plural.


-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“Moderation in temper is always a virtue; but moderation in principle is always a vice.”
	— Thomas Paine (1737–1809)

Re: Value in adding Shepherd requirements to file-systems entries?

[Ludovic Courtès]

Hi!

Richard Sent <richard@freakingpenguin.com> skribis:

> Before hacking away at this myself, I'd like to get other people's
> thoughts on the best way to proceed. Do others agree that (file-system)
> entries should support networked devices? Should this be transparent
> depending on the type, or require explicit configuration?
>
> e.g.
>
> (file-system
>   (device "//192.168.1.102/share")
>   (options "guest")
>   (mount-point "/mnt/share")
>   (type "cifs")
>   ;; Should we explicitly require network, or implicitly add it from
>   ;; the type? If the latter, what to do about Avahi?
>   (requirement 'networking)
>   (mount-may-fail? #t)
>   (create-mount-point? #t))

I think this makes sense.

The other option would be to allow for symbols in the ‘dependencies’
field, because it’s really the same thing.  That would only require a
new clause in the ‘dependency->shepherd-service-name’ procedure.

HTH!

Ludo’.

Re: Value in adding Shepherd requirements to file-systems entries?

[Richard Sent]

Hi Ludo!

> The other option would be to allow for symbols in the ‘dependencies’
> field, because it’s really the same thing.  That would only require a
> new clause in the ‘dependency->shepherd-service-name’ procedure.

Personally I prefer separating requirements and dependencies.
Dependencies adjusts the order of mounting file-systems /before/
provisioning 'file-systems, while requirements actually delays mounting
a file system until Shepherd services have started (by removing it as a
requirement for provisioning 'file-systems).

I think this distinction in behavior should be emphasized in the API and
manual.

An alternative to the requirement/requirements field is changing the
name to shepherd-requirement. That would be consistent with other
services and make the distinction between dependencies and requirements
unambiguous. (And sidestep the pluralization question.)

Happy to change to whatever the consensus is!

-- 
Take it easy,
Richard Sent
Making my computer weirder one commit at a time.

Re: Value in adding Shepherd requirements to file-systems entries?

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Richard,

On Tue, Apr 23 2024, Richard Sent wrote:

> I submitted a patch for what I'm thinking at
> https://issues.guix.gnu.org/70542.

I believe this stopped working for my NFS setup.

Kind regards
Felix

Re: Value in adding Shepherd requirements to file-systems entries?

[Richard Sent]

Hi Felix,

Felix Lechner <felix.lechner@lease-up.com> writes:

> Hi Richard,
>
> On Tue, Apr 23 2024, Richard Sent wrote:
>
>> I submitted a patch for what I'm thinking at
>> https://issues.guix.gnu.org/70542.
>
> I believe this stopped working for my NFS setup.

Interesting. It does still work with my CIFS/SMB share, so I suspect
this is an NFS-specific problem. Curiously I did have to reboot after
adding the file-system entry, but it works after that.

You could try invoking mount-file-system from (gnu build file-systems)
directly to try and narrow down what exactly is breaking.

>> --8<---------------cut here---------------start------------->8---
>> (file-system
>>    (device "wallace-server.local:/acct")
>>    (mount-point "/acct")
>>    (type "nfs")
>>    (requirement '(avahi-daemon)) ;resolve .local
>>    ;; (flags '(no-atime no-dev no-exec read-only))
>>    ;; (options "proto=tcp6,timeo=300,nolock")
>>    (check? #f)
>>    (mount-may-fail? #t)
>>    (create-mount-point? #t))
>> --8<---------------cut here---------------end--------------->8---
> Wow, that works!
> 
> Since this short form is optional and I can always switch back to the
> previous version, v2 of that patch should be merged without further
> delay.
> 
> Thank you for your most valuable contribution!
> 
> Kind regards,
> Felix
> 
> P.S. The code above should read (requirements ...) in the plural.

To confirm, you are using avahi-daemon as a shepherd-requirements entry
in your current code and not networking right? We'd need the former for
.local TLD name resolution.

-- 
Take it easy,
Richard Sent
Making my computer weirder one commit at a time.

Re: Value in adding Shepherd requirements to file-systems entries?

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Richard,

On Fri, Jun 07 2024, Richard Sent wrote:

> I did have to reboot

So did I. In fact, my issue is now that the file system fails to mount
on boot.

> You could try invoking mount-file-system from (gnu build file-systems)
> directly to try and narrow down what exactly is breaking.

How would I go about doing that, please?

> To confirm, you are using avahi-daemon as a shepherd-requirements

Yes, I am. [1]

Thanks!

Kind regards
Felix

P.S. Did you clear up the confusion about "requirements" being in the
plural?

[1] https://codeberg.org/lechner/system-config/src/commit/4d1f42cf1fc2d4ec6e8dd0434e1567e0d0bbfbf6/host/lechner-desktop/operating-system.scm#L134

Re: Value in adding Shepherd requirements to file-systems entries?

[Richard Sent]

Felix Lechner <felix.lechner@lease-up.com> writes:

> > You could try invoking mount-file-system from (gnu build file-systems)
> > directly to try and narrow down what exactly is breaking.
> How would I go about doing that, please?

1. $ guix repl
2. Import the module with ,use (gnu build file-systems)
3. Call (mount-file-system <args_from_method_signature>) in the REPL

Most likely you'd need to run guix repl as root to actually mount the
file system, so make sure the root user's Guix is up to date.

This method /just/ mounts the file-system, no shepherd services are
involved. If it mounts via mount-file-system, then there's a problem
with the service.

Also, in my experience shepherd sometimes gets upset (aka deadlocks) if
you try shutting down your system while you have an manually mounted
file system attached, so I advise making sure to unmount it after you
finish testing.

> Yes, I am. [1]
>
> Thanks!
>
> Kind regards
> Felix
>
> P.S. Did you clear up the confusion about "requirements" being in the
> plural?

Ah, V3 of the patch [1] changed it from requirements to
shepherd-requirements. (In part for consistency with other services, in
part to make it distinct from dependencies, and in part to dodge that
whole pluralization debate.) The code you linked still uses
requirements, so that could explain the problem.

I'm surprised (and mildly concerned) that there wasn't obvious breakage
when building the system. You should get an error like:

--8<---------------cut here---------------start------------->8---
ice-9/boot-9.scm:1685:16: In procedure raise-exception:
Syntax error:
unknown file:383:20: file-system: extraneous field initializers (requirements) in form (file-system (mount-point "/") (device "dummy") (requirements (quote (fake))) (type "dummy"))
--8<---------------cut here---------------end--------------->8---

That's from me trying to create a file-system record with the
requirements field instead of shepherd-requirements.

[1]: https://issues.guix.gnu.org/70542#19

-- 
Take it easy,
Richard Sent
Making my computer weirder one commit at a time.

Re: Value in adding Shepherd requirements to file-systems entries?

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Richard,

On Fri, Jun 07 2024, Richard Sent wrote:

> you are using avahi-daemon as a shepherd-requirements entry in your
> current code and not networking right?

Yes, here is my current stanza, which I adjusted after I received the
merged version of your patch:

--8<---------------cut here---------------start------------->8---
  (file-system
    (device "wallace-server.local:/acct")
    (mount-point "/acct")
    (type "nfs")
    (shepherd-requirements '(avahi-daemon)) ;resolve .local
    ;; (flags '(no-atime no-dev no-exec read-only))
    ;; (options "proto=tcp6,timeo=300,nolock")
    (check? #f)
    (mount-may-fail? #t)
    (create-mount-point? #t))
--8<---------------cut here---------------end--------------->8---

I can use the the /etc/fstab manually with

    mount /acct

> You could try invoking mount-file-system from (gnu build file-systems)
> directly to try and narrow down what exactly is breaking.

I have never used the Guix REPL.  Should I try that as the root user
immediately after booting?

Kind regards & thanks!
Felix