bug#59515: nginx: Fails to start on boot while upstream service is not yet running

bug#59515: nginx: Fails to start on boot while upstream service is not yet running

[Jonathan Brielmaier]

When I start my personal server with a radicale service behind a nginx
reverse proxy, nginx fails to start.

The relevant part in the log:
Nov 23 16:02:56 localhost shepherd[1]: Service networking has been started.
Nov 23 16:02:56 localhost shepherd[1]: Service radicale has been started.
Nov 23 16:02:56 localhost shepherd[1]: Service ssh-daemon has been started.
Nov 23 16:02:56 localhost shepherd[1]: [nginx] nginx: [emerg] host not
found in upstream "localhost:5232" in
/gnu/store/y29zl57pprwxbcxfx593s16456kxk99y-nginx.conf:15
Nov 23 16:02:56 localhost shepherd[1]: Failed to start nginx in the
background.

The config can be found here:
https://gitlab.com/jonsger/jonsger-guix/-/blob/master/config/baebia.scm#L116
```
(upstream-blocks (list
   (nginx-upstream-configuration
     (name "radicale")
     (servers (list "localhost:5232")))))
```

I wonder whats going wrong here. Is there a way to define that nginx
service should wait until radicale service is started?

~Jonathan

bug#59515: nginx: Fails to start on boot while upstream service is not yet running

[mirai]

nginx-configuration has a 'shepherd-requirement' parameter that can be used here

bug#59515: nginx: Fails to start on boot while upstream service is not yet running

[Jonathan Brielmaier]

Ah thanks.

I added `(shepherd-requirement '(radicale))` to my nginx-configuration.
Sadly the issue is not fixed, the situation is the same as before.

I assume that radicale takes a fracture of a second to start. And nginx
starts faster up. See this small experiment:

$ sudo herd stop radicale
$ sudo herd start radicale; curl localhost:5232
Service radicale has been started.
curl: (7) Failed to connect to localhost port 5232 after 0 ms:
Verbindungsaufbau abgelehnt

$ sudo herd stop radicale
$ sudo herd start radicale; sleep 0.2; curl localhost:5232
Service radicale has been started.
Redirected to /.web

bug#59515: nginx: Fails to start on boot while upstream service is not yet running

[Ludovic Courtès]

Jonathan Brielmaier <jonathan.brielmaier@web.de> skribis:

> I added `(shepherd-requirement '(radicale))` to my nginx-configuration.
> Sadly the issue is not fixed, the situation is the same as before.
>
> I assume that radicale takes a fracture of a second to start. And nginx
> starts faster up. See this small experiment:

That looks like a bug in the radicale service: there’s no
synchronization upon startup:

--8<---------------cut here---------------start------------->8---
     (list (shepherd-service
            (provision '(radicale))
            (documentation "Run the radicale daemon.")
            (requirement '(networking))
            (start #~(make-forkexec-constructor
                      (list #$(file-append package "/bin/radicale")
                        "-C" #$config-file)
                      #:user "radicale"
                      #:group "radicale"))
            (stop #~(make-kill-destructor))))
--8<---------------cut here---------------end--------------->8---

It should wait for a PID file or be started as a systemd or inetd
service.  That way, we’d be sure that shepherd considers it started once
it’s really listening for input.  A quick grep in the source suggests it
creates a PID file somewhere.

HTH!

Ludo’.

bug#59515: nginx: Fails to start on boot while upstream service is not yet running

[Jonathan Brielmaier]

On 27.11.22 22:14, Ludovic Courtès wrote:> That looks like a bug in the
radicale service: there’s no
> synchronization upon startup:

I think its rather an issue with nscd, networking or some other base
service.

Changing the nginx configuration from
(servers (list "localhost:5232"))
to
(servers (list "127.0.0.1:5232"))
resolves the issue for me :)

> It should wait for a PID file or be started as a systemd or inetd
> service.  That way, we’d be sure that shepherd considers it started once
> it’s really listening for input.  A quick grep in the source suggests it
> creates a PID file somewhere.

I couldn't find any PID file on my server of radicale. But there seems
to by a .lock file: /var/lib/radicale/collections/.Radicale.lock
No idea if that would be helpful...

~Jonathan

bug#59515: nginx: Fails to start on boot while upstream service is not yet running

[Jonathan Brielmaier]

Closing.
Workaround with using "127.0.0.1" instead of "localhost" provided.