dhcp-client default gateway with multiple interfaces, race condition?

dhcp-client default gateway with multiple interfaces, race condition?

[Richard Sent]

Hi Guix!

I'm having trouble where my virtual private server (VPS) [1] has
incorrect routing information configured. From a recovery console:

--8<---------------cut here---------------start------------->8---
root@droplet-base ~# ip route
default dev eth1
10.108.0.0/20 dev eth1 proto kernel scope link src 10.108.0.2
157.245.0.0/20 dev eth0 proto kernel scope link src <public_ip_address>
--8<---------------cut here---------------end--------------->8---

From above, the default gateway isn't pointing towards the
internet-routable gateway (157.245.0.1). If I were to manually run $ ip
route replace default default via 157.245.0.1, I can reach public
addresses again (e.g. $ ping 8.8.8.8).

This error seems to occur randomly on reboot, maybe 4/5 times. My guess
is there is some sort of race condition in the DHCP configuration that
causes eth1 to be set as the default gateway.

I believe I could resolve this by changing dhcp-client-service-type to

--8<---------------cut here---------------start------------->8---
(service dhcp-client-service-type ;make droplet know its own ip
         (dhcp-client-configuration
         (interfaces '("eth0"))))
--8<---------------cut here---------------end--------------->8---

i.e. only listen on eth0 and give up on configuring the private IP range
on eth1 entirely. I haven't tested this to confirm yet, but will
shortly.

However, I don't want to remove the private IP range entirely. From what
I read, it may be possible to configure isc-dhcp to set "nogateway" for
a specific interface. However, it looks like this isn't possible in Guix
at present. Is there another method to resolve my issue? Or is my fancy
theory about race-conditions in dhcp configuration inaccurate and
something else is to blame?

Thanks.

[1]: https://git.sr.ht/~freakingpenguin/rsent/tree/edde39fbb0585db243ac5369548867bbdc765fe8/item/rsent/system/digitalocean.scm
[2]: https://unix.stackexchange.com/a/333610
-- 
Take it easy,
Richard Sent
Making my computer weirder one commit at a time.

Re: dhcp-client default gateway with multiple interfaces, race condition?

[Felix Lechner via]

Hi Richard,

On Fri, Jun 14 2024, Richard Sent wrote:

> the default gateway isn't pointing towards the internet-routable
> gateway

The traditional answer has been that our dhclient service doesn't work
very well.  I had similar issues two years ago and, lacking time, simply
went with network-manager or connman.  Now, however, I took a look.

I'm not sure what causes the discrepancy.  Guix reads the contents of
/proc/net/dev (in syscalls.scm) in a deterministic fashion so the error
isn't in how we invoke 'dhclient'.

I believe the 'dhcp-configuration' is not quite sophisticated enough so
you may have to use the config-file option and include this content:

    # Assign this interface an IP address only.
    # (This disables setting the gateway router, DNS, domain, etc.)
    interface "eth1" {
      request subnet-mask, broadcast-address, interface-mtu;
    }

That idea came from here. [1] It could also be enough to use the
'nogateway' option as suggested in the link you posted already. [2]

I am just trying to help and did not try any of this. No warranties.

Kind regards
Felix

[1] https://unix.stackexchange.com/questions/399659/how-to-avoid-dhclient-default-gateway-on-an-interface
[2] https://unix.stackexchange.com/a/333610

Re: dhcp-client default gateway with multiple interfaces, race condition?

[Richard Sent]

Hi Felix!

Felix Lechner <felix.lechner@lease-up.com> writes:

> I believe the 'dhcp-configuration' is not quite sophisticated enough so
> you may have to use the config-file option and include this content:
>
>     # Assign this interface an IP address only.
>     # (This disables setting the gateway router, DNS, domain, etc.)
>     interface "eth1" {
>       request subnet-mask, broadcast-address, interface-mtu;
>     }
>
> That idea came from here. [1] It could also be enough to use the
> 'nogateway' option as suggested in the link you posted already. [2]

Thanks for helping out! I think the existing code only supports
providing a config file to dhcpd (dhcp server), not dhcp-client (aka
dhclient, dhcp client).

I submitted a patch for specifying a dhclient config file just now in
dhcp-client-configuration [1]. Unfortunately while the patch itself
seems fine, the config file doesn't seem to have an effect in my case.

--8<---------------cut here---------------start------------->8---
# Problem still occurs if we run the command manually like this
$ dhclient -nw -I -pf /var/run/dhclient.pid \
    -cf /gnu/store/blah-dhclient.conf
--8<---------------cut here---------------end--------------->8---

Annoyingly that nogateway option I mentioned seems to be a feature of
dhcpcd [2], which has no organizational relationship to dhclient or
dhcpd.

I have confirmed the issue is solved if I manually invoke dhclient,
don't provide a config file, and only specify eth0 as an interface. e.g.

--8<---------------cut here---------------start------------->8---
$ dhclient -nw -I -pf /var/run/dhclient.pid eth0
--8<---------------cut here---------------end--------------->8---

Curiously providing the config file causes the error to appear again
even if eth0 is the only interface provided at the command line.
Presumably the fact that the config file references eth1 is enough for
dhclient to scan it.

For completeness, this is the contents of dhclient.conf:

--8<---------------cut here---------------start------------->8---
# Assign this interface an IP address only.
# (This disables setting the gateway router, DNS, domain, etc.)
interface \"eth1\" {
  request subnet-mask, broadcast-address, interface-mtu;
}
--8<---------------cut here---------------end--------------->8---

[1]: https://issues.guix.gnu.org/71561
[2]: https://github.com/NetworkConfiguration/dhcpcd

-- 
Take it easy,
Richard Sent
Making my computer weirder one commit at a time.

Re: dhcp-client default gateway with multiple interfaces, race condition?

[Richard Sent]

I managed to find an alternative that seems to work without disabling
eth0 entirely or bloating my barebones server with NetworkManager.
Connman. The configuration and the routing tables are much more
complicated unfortunately, but it works.

(Curiously I see a new IP in the routing table that never appeared
before and doesn't seem to be related to anything beyond being owned by
DigitalOcean. Oh well, at least the default route works.)

While the cause of the problem isn't known, I can guess it's somehow
related to ISC dhcp-client being EOL'd back in 2022. At some point we
should consider deprecating it, particularly if issues like what I
encountered become more common.

(I also think any replacement should probably avoid being called
"dhcp-client". Most other services don't name themselves according to
their role, but the software.)

-- 
Take it easy,
Richard Sent
Making my computer weirder one commit at a time.