Herding file-systems

Herding file-systems

[Bruno Victal]

Co-authored-by: Felix Lechner


Some observations and potential plans of action to improve file-system handling
and proper NFS support within Guix.


Relaxing dependency field of file-system record type
===============================================================================

Guix currently envisions file systems to depend only on other file
systems, but that restriction should perhaps be relaxed. In some
cases, it may make sense to wait for any (shepherd) service,
i.e. not just for ones that mount directories. These could be used to place a
dependency on kerberos, etc.

A prominent example is NFS, which should depend on 'networking but
currently doesn't.


Networked file-systems (such as NFS)
-------------------------------------------------------------------------------

The prerequisite 'file-systems should probably be split into
'networked-file-systems and 'local-file-systems. That distinction is
already commonplace in other distributions. The stage
'networked-file-systems would in many respects be similar to
'file-systems but also depend on 'networking.

We would have to watch out for some inconsistencies, however. For
networked file systems, for example, the setting (needed-for-boot #t)
may be impossible---except perhaps for thin clients. In impossible
cases, the configuration would error out.


Ideas to implement NFS support
-------------------------------------------------------------------------------

1. Add a _netdev flag (à la systemd)
   Looks ugly?

2. Add a new prerequisite called 'networked-file-systems
   This serves as a generic catch point for network dependent filesystems.
   This is to be specified manually.  (3.1 notes apply here)

3. Partition file-systems according to field 'type into 'file-systems and 'networked-file-systems.
   As a drawback, it would only be able to handle known file-systems,
   because that list would have to be hard-coded.
   Cons: Automatic partition precludes the scenario that there could
   be two shepherd services, one providing regular 'networked-file-systems and a custom
   one providing some 'my-custom-service symbol, where the latter is specifically
   intended for the file-system, since the file-system is forcefully grouped into
   'networked-file-systems.

3.1 Alternatively forego partitioning between local and network filesystems
    and simply put a dependency on 'networking. The filesystem must be
    decoupled from the 'file-systems symbol to prevent circular dependencies.

Comments:
The decision criteria should be "which one is the most flexible/has the least
implicit assumptions" here.


Name collision for mounts on the same path
===============================================================================

One additional concern is the naming of Shepherd services for file systems.
We can no longer just create the name from mount point alone since those names
would collide with filesystems that can mount over the same path.

The concern is not as far fetched as it may initially appear. Examples
would be FUSE filesystems and OverlayFS, but there are probably others.
Some FUSE file systems are specifically mounted over
an existing mount point in order to provide additional services like
transparent access for compressed archives.

Regarding OverlayFS, it's worth noting that it can mount multiple
times over itself, though personally I don't know what the applications
are for this kind of scenario, merely that it can be done.


fstab serialization criteria
===============================================================================

It's worth revisiting why/what are the issues (and whether they're still relevant)
that made us exclude certain file-system declarations from being serialized to /etc/fstab. [1]
Excluding serialization for certain types isn't a problem but there should be an escape hatch option
to forcefully serialize them to /etc/fstab.


Footnotes
===============================================================================

[1]: https://issues.guix.gnu.org/60246


Cheers,
Bruno

Re: Herding file-systems

[Maxim Cournoyer]

Hi,

Bruno Victal <mirai@makinata.eu> writes:

> Co-authored-by: Felix Lechner
>
>
> Some observations and potential plans of action to improve file-system handling
> and proper NFS support within Guix.
>
>
> Relaxing dependency field of file-system record type
> ===============================================================================
>
> Guix currently envisions file systems to depend only on other file
> systems, but that restriction should perhaps be relaxed. In some
> cases, it may make sense to wait for any (shepherd) service,
> i.e. not just for ones that mount directories. These could be used to place a
> dependency on kerberos, etc.
>
> A prominent example is NFS, which should depend on 'networking but
> currently doesn't.

Sounds good.

> Networked file-systems (such as NFS)
> -------------------------------------------------------------------------------
>
> The prerequisite 'file-systems should probably be split into
> 'networked-file-systems and 'local-file-systems. That distinction is
> already commonplace in other distributions. The stage
> 'networked-file-systems would in many respects be similar to
> 'file-systems but also depend on 'networking.
>
> We would have to watch out for some inconsistencies, however. For
> networked file systems, for example, the setting (needed-for-boot #t)
> may be impossible---except perhaps for thin clients. In impossible
> cases, the configuration would error out.
>
>
> Ideas to implement NFS support
> -------------------------------------------------------------------------------
>
> 1. Add a _netdev flag (à la systemd)
>    Looks ugly?

Very :-).  Let's try to find a more elegant way.

> 2. Add a new prerequisite called 'networked-file-systems
>    This serves as a generic catch point for network dependent filesystems.
>    This is to be specified manually.  (3.1 notes apply here)

> 3. Partition file-systems according to field 'type into 'file-systems and 'networked-file-systems.
>    As a drawback, it would only be able to handle known file-systems,
>    because that list would have to be hard-coded.
>    Cons: Automatic partition precludes the scenario that there could
>    be two shepherd services, one providing regular 'networked-file-systems and a custom
>    one providing some 'my-custom-service symbol, where the latter is specifically
>    intended for the file-system, since the file-system is forcefully grouped into
>    'networked-file-systems.

I don't understand the cons, which perhaps suggests it'd be a bit of a
stretched use case?

> 3.1 Alternatively forego partitioning between local and network filesystems
>     and simply put a dependency on 'networking. The filesystem must be
>     decoupled from the 'file-systems symbol to prevent circular dependencies.
>
> Comments:
> The decision criteria should be "which one is the most flexible/has the least
> implicit assumptions" here.

Another related idea; perhaps we could introduce a new record called
'distributed-file-system' and use this at the time of declaring the file
system?  That'd would automatically depend on networking in the
underlying implementation; so I'd be able to specify it in my operating
system like:

    (file-systems (cons*
                   (file-system
                     (device (uuid "43582534-54a1-415a-a8ac-ecc20867f7a3"))
                     (mount-point "/boot")
                     (type "btrfs")
                     (options "compress-force=zstd"))
                   [...]
                   (distributed-file-system
                     (device "host:/srv/nfs/jami")
                     (mount-point "/var/cache/jami")
                     (create-mount-point? #t)
                     (type "nfs")
                     (options "soft,user"))

It'd allow to decouple the <file-system> and <distributed-file-system>
fully, providing flexibility to add new fields to the later in the
future which would make little sense for the former.  It's also more
declarative than the "auto-determine networkness from type" (point 2
above) heuristic.

Thanks for looking into this longstanding limitation!

-- 
Maxim

Re: Herding file-systems

[Liliana Marie Prikler]

Am Mittwoch, dem 08.03.2023 um 14:54 +0000 schrieb Bruno Victal:
> Co-authored-by: Felix Lechner
> 
> Some observations and potential plans of action to improve file-
> system handling and proper NFS support within Guix.
> 
> 
> Relaxing dependency field of file-system record type
> =====================================================================
> 
> Guix currently envisions file systems to depend only on other file
> systems, but that restriction should perhaps be relaxed. In some
> cases, it may make sense to wait for any (shepherd) service,
> i.e. not just for ones that mount directories. These could be used to
> place a dependency on kerberos, etc.
> 
> A prominent example is NFS, which should depend on 'networking but
> currently doesn't.
Sounds fair so far.

> 
> Networked file-systems (such as NFS)
> ---------------------------------------------------------------------
> 
> The prerequisite 'file-systems should probably be split into
> 'networked-file-systems and 'local-file-systems. That distinction is
> already commonplace in other distributions. The stage
> 'networked-file-systems would in many respects be similar to
> 'file-systems but also depend on 'networking.
> 
> We would have to watch out for some inconsistencies, however. For
> networked file systems, for example, the setting (needed-for-boot #t)
> may be impossible---except perhaps for thin clients. In impossible
> cases, the configuration would error out.
And here I see potential for possible over-engineering.  Since all our
file-systems are already handled by shepherd, I see no need in
"herding" them, as the title implies.  What is needed, OTOH, is a way
to inject shepherd dependencies, e.g. on networking, but possibly also
on other file systems (*cough* loop devices *cough*).

> Name collision for mounts on the same path
> =====================================================================
> One additional concern is the naming of Shepherd services for file
> systems.  We can no longer just create the name from mount point
> alone since those names would collide with filesystems that can mount
> over the same path.
Uhm, why should we allow this?

> The concern is not as far fetched as it may initially appear.
> Examples would be FUSE filesystems and OverlayFS, but there are
> probably others.  Some FUSE file systems are specifically mounted
> over an existing mount point in order to provide additional services
> like transparent access for compressed archives.
> 
> Regarding OverlayFS, it's worth noting that it can mount multiple
> times over itself, though personally I don't know what the
> applications are for this kind of scenario, merely that it can be
> done.
In which cases should these mounts be created by the operating system
rather than a cheeky user, though?

> fstab serialization criteria
> =====================================================================
> 
> It's worth revisiting why/what are the issues (and whether they're
> still relevant) that made us exclude certain file-system declarations
> from being serialized to /etc/fstab. [1]
> Excluding serialization for certain types isn't a problem but there
> should be an escape hatch option to forcefully serialize them to
> /etc/fstab.
This appears a somewhat separate concern to the rest here and I'd
suggest investigating it separately, as this escape hatch can easily be
added without thinking about the broader picture of network file
systems.

Cheers

Plan for NFS (was: Herding file-systems)

[Bruno Victal]

Hi all,

I've been thinking on a plan to address this old NFS question.

The plan:

  1. Allow the file-system `dependencies' field to accept arbitrary
shepherd symbols as well.

This can be done by having the `dependency->shepherd-service-name'
in (gnu services base) also accept matching against a symbol object.

  2. Allow excluding some file-system objects from being dependencies
of the `file-systems' shepherd service.

This is to avoid a cycle scenario where a (NFS) mount requires
'networking (that depends on 'file-systems) which in turn requires
a mount on some device that has the credentials stored for the network
connection. Another scenario to consider is loop mounts.

WDYT?

-- 
Furthermore, I consider that nonfree software must be eradicated.

Cheers,
Bruno.

Re: Plan for NFS (was: Herding file-systems)

[Liliana Marie Prikler]

Hi Bruno,

Am Dienstag, dem 26.09.2023 um 19:35 +0100 schrieb Bruno Victal:
> Hi all,
> 
> I've been thinking on a plan to address this old NFS question.
> 
> The plan:
> 
>   1. Allow the file-system `dependencies' field to accept arbitrary
> shepherd symbols as well.
Sure, why not?

> This can be done by having the `dependency->shepherd-service-name'
> in (gnu services base) also accept matching against a symbol object.
> 
>   2. Allow excluding some file-system objects from being dependencies
> of the `file-systems' shepherd service.
I think we could turn this one to yet another boolean in the file-
system record that's simply named "required?".

Cheers

Re: Plan for NFS

[Ludovic Courtès]

Hi!

Bruno Victal <mirai@makinata.eu> skribis:

> I've been thinking on a plan to address this old NFS question.

I’m all for something along the lines of what you suggest.

FWIW, comrades running an HPC computer started deploying Guix System on
the nodes, which requires among other things a store shared over NFS
(info "(guix-cookbook) Installing Guix on a Cluster").  I can’t find
their code right now but it exists somewhere.  If the comrades I’m
thinking of are reading this, please consider sharing your experience.
:-)

Ludo’.