Advanced network configuration

Advanced network configuration

[Alexey Abramov]

Hi Guix,

I have a couple of questions regarding our current network configuration
in Guix. With the latest changes to the static configuration, live is
indeed, became easier, but I have to say that it is still insufficient
to define a complete network configuration.

Please check out the following (my) use cases:

1. I have servers with multiple network cards. Let's say, the management
one and production. both network cards have multiple interfaces, which I
need to bond into 803.1ad mode. Management bond has a native vlan where
DHCP should be provided AND another vlan for the management traffic. The
production interface (after bonding) has another set of VLANs for a
different type of traffic as well. Currently, there is no way to define
it without writing a custom service with the iproute2 utility.

I was hacking on guile-netlink and did some progress with setting
bonds. However current static configuration won't let me build it
anyway. I have to not only 'link-add' and create a bond, but also amend
existing interfaces by 'link-set' them as a slave to my newly created
bond master for example.

2. Having a router with Guix at home. I have to run multiple services
that provision 'networking' which is not allowed right now. The DHCP
client service is greedy right now and binds to all available
interfaces. I sent a [1] patch to solve this. However, I cannot define
dhcp-client and static configuration at the same time anyway.

These two examples lead me to the following questions:

1) To configure the network, I have been happily using netplan app
(debain/ubuntu world). The specification they use can be found here
[2]. Here is an example:

--8<---------------cut here---------------start------------->8---
network:
  bonds:
    bond0:
      interfaces:
        - eno0
        - eno1
      parameters:
        mode: 802.3ad
    bond2:
      interfaces:
        - enp0
        - enp1
      parameters:
        mode: 802.3ad
  ethernets:
    eno0:
      match:
        macaddress: xx:xx:xx:xx:xx:xx
      set-name: eno0
    eno1:
      match:
        macaddress: xx:xx:xx:xx:xx:xx
      set-name: eno1
    eno2:
      match:
        macaddress: xx:xx:xx:xx:xx:xx
      set-name: eno2
    eno3:
      match:
        macaddress: xx:xx:xx:xx:xx:xx
      set-name: eno3
    enp0:
      match:
        macaddress: xx:xx:xx:xx:xx:xx
      set-name: enp0
    enp1:
      match:
        macaddress: xx:xx:xx:xx:xx:xx
      set-name: enp1
  renderer: networkd
  version: 2
  vlans:
    bond1.1055:
      addresses:
        - 10.0.0.1/24
      routes:
        - to: XX.XX.XX.XX/21
          via: 10.x.x.x
      gateway4: 10.0.0.1
      id: 1055
      link: bond1
    bond2.2055:
      addresses:
        - 10.8.0.1/24
      id: 2055
      link: bond2
    bond2.3055:
      addresses:
        - 192.168.0.0/16
      id: 3055
      link: bond2
--8<---------------cut here---------------end--------------->8---

From the example above, you can see that I rename interfaces (i do know
that udev can do it, but it is easier to do it here). I build bonds and
VLANs with their IPs. I also set routes and default gateway. Would
someone be interested in that? I don't see any problems in implementing
this with guile-netlink. This is basically my proposal on how we can
improve our static network configuration. What do you think?

2) In Systemd world there is a notion of 'target' which is used for
*grouping* units and *synchronization* points during the start-up.

At this moment *I* need to have dhcp-client and static
configuration. Seems like grouping is a key here. Were there any
ideas/attempts to make it happen? Or what should I do? 

Footnotes:
[1]  https://issues.guix.gnu.org/58223

[2]  https://netplan.io/reference

-- 
Alexey

Re: Advanced network configuration

[Ryan Sundberg]

[-- Attachment #1: Type: text/plain, Size: 4338 bytes --]

Hi Alexy, are you running netplan under Guix? I would find it useful to have a service using a netplan config, bypassing the guile-based networking service for such advanced use cases as you mention.


Sincerely,

Ryan Sundberg
Principal Software Engineer
Arctype Corporation



-------- Original Message --------
From: Alexey Abramov <levenson@mmer.org>
Sent: October 3, 2022 11:23:11 AM PDT
To: guix-devel@gnu.org
Subject: Advanced network configuration

Hi Guix,

I have a couple of questions regarding our current network configuration
in Guix. With the latest changes to the static configuration, live is
indeed, became easier, but I have to say that it is still insufficient
to define a complete network configuration.

Please check out the following (my) use cases:

1. I have servers with multiple network cards. Let's say, the management
one and production. both network cards have multiple interfaces, which I
need to bond into 803.1ad mode. Management bond has a native vlan where
DHCP should be provided AND another vlan for the management traffic. The
production interface (after bonding) has another set of VLANs for a
different type of traffic as well. Currently, there is no way to define
it without writing a custom service with the iproute2 utility.

I was hacking on guile-netlink and did some progress with setting
bonds. However current static configuration won't let me build it
anyway. I have to not only 'link-add' and create a bond, but also amend
existing interfaces by 'link-set' them as a slave to my newly created
bond master for example.

2. Having a router with Guix at home. I have to run multiple services
that provision 'networking' which is not allowed right now. The DHCP
client service is greedy right now and binds to all available
interfaces. I sent a [1] patch to solve this. However, I cannot define
dhcp-client and static configuration at the same time anyway.

These two examples lead me to the following questions:

1) To configure the network, I have been happily using netplan app
(debain/ubuntu world). The specification they use can be found here
[2]. Here is an example:

--8<---------------cut here---------------start------------->8---
network:
  bonds:
    bond0:
      interfaces:
        - eno0
        - eno1
      parameters:
        mode: 802.3ad
    bond2:
      interfaces:
        - enp0
        - enp1
      parameters:
        mode: 802.3ad
  ethernets:
    eno0:
      match:
        macaddress: xx:xx:xx:xx:xx:xx
      set-name: eno0
    eno1:
      match:
        macaddress: xx:xx:xx:xx:xx:xx
      set-name: eno1
    eno2:
      match:
        macaddress: xx:xx:xx:xx:xx:xx
      set-name: eno2
    eno3:
      match:
        macaddress: xx:xx:xx:xx:xx:xx
      set-name: eno3
    enp0:
      match:
        macaddress: xx:xx:xx:xx:xx:xx
      set-name: enp0
    enp1:
      match:
        macaddress: xx:xx:xx:xx:xx:xx
      set-name: enp1
  renderer: networkd
  version: 2
  vlans:
    bond1.1055:
      addresses:
        - 10.0.0.1/24
      routes:
        - to: XX.XX.XX.XX/21
          via: 10.x.x.x
      gateway4: 10.0.0.1
      id: 1055
      link: bond1
    bond2.2055:
      addresses:
        - 10.8.0.1/24
      id: 2055
      link: bond2
    bond2.3055:
      addresses:
        - 192.168.0.0/16
      id: 3055
      link: bond2
--8<---------------cut here---------------end--------------->8---

From the example above, you can see that I rename interfaces (i do know
that udev can do it, but it is easier to do it here). I build bonds and
VLANs with their IPs. I also set routes and default gateway. Would
someone be interested in that? I don't see any problems in implementing
this with guile-netlink. This is basically my proposal on how we can
improve our static network configuration. What do you think?

2) In Systemd world there is a notion of 'target' which is used for
*grouping* units and *synchronization* points during the start-up.

At this moment *I* need to have dhcp-client and static
configuration. Seems like grouping is a key here. Were there any
ideas/attempts to make it happen? Or what should I do? 

Footnotes:
[1]  https://issues.guix.gnu.org/58223

[2]  https://netplan.io/reference

-- 
Alexey

[-- Attachment #2: Type: text/html, Size: 5023 bytes --]

Re: Advanced network configuration

[Alexey Abramov]

Hi Ryan,

Ryan Sundberg <ryan@arctype.co> writes:

> Hi Alexy, are you running netplan under Guix? I would find it useful
> to have a service using a netplan config, bypassing the guile-based
> networking service for such advanced use cases as you mention.

No, I have never tried to do this actually. Netplan does support
NetworkManager as a backend. Maybe it is possible.

However, I would love to use a regular Guix service for that. 
guile-netlink with some small changes [1], can be used to build such a
configuration. 

I haven't implemented all the bond features, yet. Right now, I am
looking for ways to implement tests.

Footnotes:
[1]  https://gitlab.com/Levenson/guile-netlink/-/tree/wip-bonding

-- 
Alexey

Re: Advanced network configuration

[Ludovic Courtès]

Hi Alexey,

(Cc: Julien, author of Guile-Netlink.)

Alexey Abramov <levenson@mmer.org> skribis:

> 1. I have servers with multiple network cards. Let's say, the management
> one and production. both network cards have multiple interfaces, which I
> need to bond into 803.1ad mode. Management bond has a native vlan where
> DHCP should be provided AND another vlan for the management traffic. The
> production interface (after bonding) has another set of VLANs for a
> different type of traffic as well. Currently, there is no way to define
> it without writing a custom service with the iproute2 utility.
>
> I was hacking on guile-netlink and did some progress with setting
> bonds. However current static configuration won't let me build it
> anyway. I have to not only 'link-add' and create a bond, but also amend
> existing interfaces by 'link-set' them as a slave to my newly created
> bond master for example.

I’m sure your improvements to Guile-Netlink would be welcome.

Regarding ‘static-networking’ in Guix, the goal was to allow it to be as
expressive as the underlying netlink interface, but clearly we focused
on the most common use cases.

If you can think of how you’d like to represent these setups in
‘static-networking’ (perhaps a ‘bonds’ field similar to the netplan YAML
snippet you showed?), we (or you :-)) can try and implement it.

> 2. Having a router with Guix at home. I have to run multiple services
> that provision 'networking' which is not allowed right now. The DHCP
> client service is greedy right now and binds to all available
> interfaces. I sent a [1] patch to solve this. However, I cannot define
> dhcp-client and static configuration at the same time anyway.

OK, we could allow users to change the Shepherd service name used by the
DHCP client then.

> These two examples lead me to the following questions:
>
> 1) To configure the network, I have been happily using netplan app
> (debain/ubuntu world). The specification they use can be found here
> [2]. Here is an example:

[...]

> From the example above, you can see that I rename interfaces (i do know
> that udev can do it, but it is easier to do it here). I build bonds and
> VLANs with their IPs. I also set routes and default gateway. Would
> someone be interested in that? I don't see any problems in implementing
> this with guile-netlink. This is basically my proposal on how we can
> improve our static network configuration. What do you think?

I’m all for it!  This should be done in a way that preserves
compatibility with the existing <static-networking> record as much as
possible.

> 2) In Systemd world there is a notion of 'target' which is used for
> *grouping* units and *synchronization* points during the start-up.
>
> At this moment *I* need to have dhcp-client and static
> configuration. Seems like grouping is a key here. Were there any
> ideas/attempts to make it happen? Or what should I do? 

I’m not sure.  IIUC, a “networking target” here could translate to a
Shepherd service that depends on all the relevant DHCP and static
networking services.  The question the becomes how to express that
grouping conveniently.

Thanks,
Ludo’.

Re: Advanced network configuration

[Alexey Abramov]

Hi Ludo, Julien

Ludovic Courtès <ludo@gnu.org> writes:

> Hi Alexey,
>
> (Cc: Julien, author of Guile-Netlink.)
>

[...]

>
> I’m sure your improvements to Guile-Netlink would be welcome.

> Regarding ‘static-networking’ in Guix, the goal was to allow it to be as
> expressive as the underlying netlink interface, but clearly we focused
> on the most common use cases.
>
> If you can think of how you’d like to represent these setups in
> ‘static-networking’ (perhaps a ‘bonds’ field similar to the netplan YAML
> snippet you showed?), we (or you :-)) can try and implement it.

Yeah, that was my intention =). @Julien Could you tell me how can I
collaborate? Shall I send patches to you directly or maybe debbugs, or
guix-patches?

>> 2. Having a router with Guix at home. I have to run multiple services
>> that provision 'networking' which is not allowed right now. The DHCP
>> client service is greedy right now and binds to all available
>> interfaces. I sent a [1] patch to solve this. However, I cannot define
>> dhcp-client and static configuration at the same time anyway.
>
> OK, we could allow users to change the Shepherd service name used by the
> DHCP client then.

That would indeed help for now. I can prepare yet another patch for
that.

[...]

> I’m not sure.  IIUC, a “networking target” here could translate to a
> Shepherd service that depends on all the relevant DHCP and static
> networking services.  The question the becomes how to express that
> grouping conveniently.

Yes, I also would like to point out that their must be a way to
establish a firewall, for example, *before* any network interface is up
(After=network-pre.target in systemd [1]). And the same thing during the
shutdown procedure (Before=network-pre.target in systemd).  Applications
have to be able to gracefully shutdown their network connections.  Is it
the case right now, I don't know?

I am checking (shepherd services) where `shutdown-services' defined, and
seems like it just walks across %services hash table. Am I missing
something?

Footnotes:
[1]  https://www.freedesktop.org/software/systemd/man/systemd.special.html#network-pre.target

-- 
Alexey

Re: Advanced network configuration

[Julien Lepiller]

[-- Attachment #1: Type: text/plain, Size: 2585 bytes --]

I guess using debbugs would give other people a chance to have a look at your patches and comment, but I'm the only one who can push anyway. If you decide to use debbugs, make sure to CC me too.

Le 6 octobre 2022 15:11:30 GMT+02:00, Alexey Abramov <levenson@mmer.org> a écrit :
>Hi Ludo, Julien
>
>Ludovic Courtès <ludo@gnu.org> writes:
>
>> Hi Alexey,
>>
>> (Cc: Julien, author of Guile-Netlink.)
>>
>
>[...]
>
>>
>> I’m sure your improvements to Guile-Netlink would be welcome.
>
>> Regarding ‘static-networking’ in Guix, the goal was to allow it to be as
>> expressive as the underlying netlink interface, but clearly we focused
>> on the most common use cases.
>>
>> If you can think of how you’d like to represent these setups in
>> ‘static-networking’ (perhaps a ‘bonds’ field similar to the netplan YAML
>> snippet you showed?), we (or you :-)) can try and implement it.
>
>Yeah, that was my intention =). @Julien Could you tell me how can I
>collaborate? Shall I send patches to you directly or maybe debbugs, or
>guix-patches?
>
>>> 2. Having a router with Guix at home. I have to run multiple services
>>> that provision 'networking' which is not allowed right now. The DHCP
>>> client service is greedy right now and binds to all available
>>> interfaces. I sent a [1] patch to solve this. However, I cannot define
>>> dhcp-client and static configuration at the same time anyway.
>>
>> OK, we could allow users to change the Shepherd service name used by the
>> DHCP client then.
>
>That would indeed help for now. I can prepare yet another patch for
>that.
>
>[...]
>
>> I’m not sure.  IIUC, a “networking target” here could translate to a
>> Shepherd service that depends on all the relevant DHCP and static
>> networking services.  The question the becomes how to express that
>> grouping conveniently.
>
>Yes, I also would like to point out that their must be a way to
>establish a firewall, for example, *before* any network interface is up
>(After=network-pre.target in systemd [1]). And the same thing during the
>shutdown procedure (Before=network-pre.target in systemd).  Applications
>have to be able to gracefully shutdown their network connections.  Is it
>the case right now, I don't know?
>
>I am checking (shepherd services) where `shutdown-services' defined, and
>seems like it just walks across %services hash table. Am I missing
>something?
>
>Footnotes:
>[1]  https://www.freedesktop.org/software/systemd/man/systemd.special.html#network-pre.target
>
>-- 
>Alexey

[-- Attachment #2: Type: text/html, Size: 3797 bytes --]

Re: Advanced network configuration

[Alexey Abramov]

Hi Julien,

I sent patches to guile-netlink [1].


Footnotes:
[1]  https://issues.guix.gnu.org/58382

-- 
Alexey

Re: Advanced network configuration

[Ludovic Courtès]

Hi!

Alexey Abramov <levenson@mmer.org> skribis:

[...]

>> I’m not sure.  IIUC, a “networking target” here could translate to a
>> Shepherd service that depends on all the relevant DHCP and static
>> networking services.  The question the becomes how to express that
>> grouping conveniently.
>
> Yes, I also would like to point out that their must be a way to
> establish a firewall, for example, *before* any network interface is up
> (After=network-pre.target in systemd [1]). And the same thing during the
> shutdown procedure (Before=network-pre.target in systemd).

I would do that by having ‘networking’ depend on ‘firewall’ (say).

Does that make sense?

It’d be interesting to see whether we need something beyond this.

> Applications have to be able to gracefully shutdown their network
> connections.  Is it the case right now, I don't know?

What do you mean?

> I am checking (shepherd services) where `shutdown-services' defined, and
> seems like it just walks across %services hash table. Am I missing
> something?

Correct, there’s nothing fancy going on there.

Thanks,
Ludo’.

Re: Advanced network configuration

[Julien Lepiller]

Le 10 octobre 2022 17:17:16 GMT+02:00, "Ludovic Courtès" <ludo@gnu.org> a écrit :
>Hi!
>
>Alexey Abramov <levenson@mmer.org> skribis:
>
>[...]
>
>>> I’m not sure.  IIUC, a “networking target” here could translate to a
>>> Shepherd service that depends on all the relevant DHCP and static
>>> networking services.  The question the becomes how to express that
>>> grouping conveniently.
>>
>> Yes, I also would like to point out that their must be a way to
>> establish a firewall, for example, *before* any network interface is up
>> (After=network-pre.target in systemd [1]). And the same thing during the
>> shutdown procedure (Before=network-pre.target in systemd).
>
>I would do that by having ‘networking’ depend on ‘firewall’ (say).
>
>Does that make sense?

Wouldn't there be an issue if firewall rules reference interfaces created by the networking service?

>
>It’d be interesting to see whether we need something beyond this.
>

Re: Advanced network configuration

[Alexey Abramov]

Hi,

Ludovic Courtès <ludo@gnu.org> writes:

> Hi!
>
> Alexey Abramov <levenson@mmer.org> skribis:
>
> [...]
>

[...]

>
> I would do that by having ‘networking’ depend on ‘firewall’ (say).
>
> Does that make sense?
>
> It’d be interesting to see whether we need something beyond this.

But what if I just want to stop a firewall? Won't that trigger
network to restart in that case?

>> Applications have to be able to gracefully shutdown their network
>> connections.  Is it the case right now, I don't know?
>
> What do you mean?

If you run a simple VM with openssh, login there via ssh client and run
'herd power-off root' from the QEMU serial console, the VM will be
shutdown, but your ssh connection won't know anything about it and the
socket will be left in a ESTABLISHED state! By typing there system will
keep trying to send data over there wire (Send-Q).

--8<---------------cut here---------------start------------->8---
root@delta ~# ss -ta -A all '( dport = :ssh )' dst 192.168.10.100/24
Netid             State             Recv-Q             Send-Q                         Local Address:Port                            Peer Address:Port            Process             
tcp               ESTAB             0                  288                             192.168.10.1:48106                         192.168.10.100:ssh                                 
--8<---------------cut here---------------end--------------->8---

There is no such a problem with shutdown command by the way. 

>> I am checking (shepherd services) where `shutdown-services' defined, and
>> seems like it just walks across %services hash table. Am I missing
>> something?
>
> Correct, there’s nothing fancy going on there.
>
> Thanks,
> Ludo’.

-- 
Alexey

Re: Advanced network configuration

[Ludovic Courtès]

Hi,

Alexey Abramov <levenson@mmer.org> skribis:

>> I would do that by having ‘networking’ depend on ‘firewall’ (say).
>>
>> Does that make sense?
>>
>> It’d be interesting to see whether we need something beyond this.
>
> But what if I just want to stop a firewall? Won't that trigger
> network to restart in that case?

Yes it will, you’re right.

>>> Applications have to be able to gracefully shutdown their network
>>> connections.  Is it the case right now, I don't know?
>>
>> What do you mean?
>
> If you run a simple VM with openssh, login there via ssh client and run
> 'herd power-off root' from the QEMU serial console, the VM will be
> shutdown, but your ssh connection won't know anything about it and the
> socket will be left in a ESTABLISHED state! By typing there system will
> keep trying to send data over there wire (Send-Q).
>
> root@delta ~# ss -ta -A all '( dport = :ssh )' dst 192.168.10.100/24
> Netid             State             Recv-Q             Send-Q                         Local Address:Port                            Peer Address:Port            Process             
> tcp               ESTAB             0                  288                             192.168.10.1:48106                         192.168.10.100:ssh                                 
>
> There is no such a problem with shutdown command by the way. 

Hmm I see.

So I guess I don’t know what the correct solution would be, but it’d be
great to see what ideas you have and how systemd approaches these
issues.  Then, if needed, we can adjust the service model in the
Shepherd.

Thanks,
Ludo’.