* bug#36731: shepherd lost track of nginx
@ 2019-07-19 16:49 Robert Vollmert
2019-07-19 22:49 ` Ludovic Courtès
0 siblings, 1 reply; 6+ messages in thread
From: Robert Vollmert @ 2019-07-19 16:49 UTC (permalink / raw)
To: 36731
Not sure who’s at fault here, but without doing anything weird,
I ended up with a system where shepherd thought that nginx was
stopped, while there was still an nginx process around. I
certainly didn’t start it by hand.
The result was this:
$ sudo herd restart nginx
Service nginx is not running.
herd: exception caught while executing 'start' on service 'nginx':
Throw to key `srfi-34' with args `("#<condition &invoke-error [program: \"/gnu/store/mlg0xfbiq03s812rm3v7mrlhyngas4xp-nginx-1.17.1/sbin/nginx\" arguments: (\"-c\" \"/gnu/store/r6gl9n7pwf4npiri05qxr40vdihdm2yy-nginx.conf\" \"-p\" \"/var/run/nginx\") exit-status: 1 term-signal: #f stop-signal: #f] 147e000>")’.
That error message could also be clearer about what’s going on. At any
rate, after I killed the nginx process, “herd start nginx” worked fine.
I should add that nginx was still doing its job fine before I killed it.
^ permalink raw reply [flat|nested] 6+ messages in thread* bug#36731: shepherd lost track of nginx
2019-07-19 16:49 bug#36731: shepherd lost track of nginx Robert Vollmert
@ 2019-07-19 22:49 ` Ludovic Courtès
2019-07-20 7:42 ` Robert Vollmert
2019-07-20 23:07 ` Mark H Weaver
0 siblings, 2 replies; 6+ messages in thread
From: Ludovic Courtès @ 2019-07-19 22:49 UTC (permalink / raw)
To: Robert Vollmert; +Cc: 36731
Hello,
Robert Vollmert <rob@vllmrt.net> skribis:
> Not sure who’s at fault here, but without doing anything weird,
> I ended up with a system where shepherd thought that nginx was
> stopped, while there was still an nginx process around. I
> certainly didn’t start it by hand.
Did you try “herd status nginx” to see shepherd’s notion of the nginx
process?
> The result was this:
>
> $ sudo herd restart nginx
> Service nginx is not running.
> herd: exception caught while executing 'start' on service 'nginx':
> Throw to key `srfi-34' with args `("#<condition &invoke-error [program: \"/gnu/store/mlg0xfbiq03s812rm3v7mrlhyngas4xp-nginx-1.17.1/sbin/nginx\" arguments: (\"-c\" \"/gnu/store/r6gl9n7pwf4npiri05qxr40vdihdm2yy-nginx.conf\" \"-p\" \"/var/run/nginx\") exit-status: 1 term-signal: #f stop-signal: #f] 147e000>")’.
Do you use an “opaque” nginx config file, or do you use <nginx-...>
records?
In the former case, the ‘start’ method won’t attempt to read the PID
file (because it cannot be sure it’ll exist), so it’s effectively unable
to track the process. See comment in ‘nginx-shepherd-service’.
> That error message could also be clearer about what’s going on. At any
> rate, after I killed the nginx process, “herd start nginx” worked fine.
I agree that we could and should improve the error message. Redirecting
nginx’s stderr so that shepherd clients can see it would be best.
Thanks,
Ludo’.
^ permalink raw reply [flat|nested] 6+ messages in thread* bug#36731: shepherd lost track of nginx
2019-07-19 22:49 ` Ludovic Courtès
@ 2019-07-20 7:42 ` Robert Vollmert
2019-07-20 13:51 ` Ludovic Courtès
2019-07-20 23:07 ` Mark H Weaver
1 sibling, 1 reply; 6+ messages in thread
From: Robert Vollmert @ 2019-07-20 7:42 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 36731
> On 20. Jul 2019, at 00:49, Ludovic Courtès <ludo@gnu.org> wrote:
>
> Hello,
>
> Robert Vollmert <rob@vllmrt.net> skribis:
>
>> Not sure who’s at fault here, but without doing anything weird,
>> I ended up with a system where shepherd thought that nginx was
>> stopped, while there was still an nginx process around. I
>> certainly didn’t start it by hand.
>
> Did you try “herd status nginx” to see shepherd’s notion of the nginx
> process?
Not at the time, no.
>
>> The result was this:
>>
>> $ sudo herd restart nginx
>> Service nginx is not running.
>> herd: exception caught while executing 'start' on service 'nginx':
>> Throw to key `srfi-34' with args `("#<condition &invoke-error [program: \"/gnu/store/mlg0xfbiq03s812rm3v7mrlhyngas4xp-nginx-1.17.1/sbin/nginx\" arguments: (\"-c\" \"/gnu/store/r6gl9n7pwf4npiri05qxr40vdihdm2yy-nginx.conf\" \"-p\" \"/var/run/nginx\") exit-status: 1 term-signal: #f stop-signal: #f] 147e000>")’.
>
> Do you use an “opaque” nginx config file, or do you use <nginx-...>
> records?
The latter I think:
(service nginx-service-type
(nginx-configuration
(extra-content “…”)))
^ permalink raw reply [flat|nested] 6+ messages in thread* bug#36731: shepherd lost track of nginx
2019-07-20 7:42 ` Robert Vollmert
@ 2019-07-20 13:51 ` Ludovic Courtès
0 siblings, 0 replies; 6+ messages in thread
From: Ludovic Courtès @ 2019-07-20 13:51 UTC (permalink / raw)
To: Robert Vollmert; +Cc: 36731
Hi,
Robert Vollmert <rob@vllmrt.net> skribis:
>>> $ sudo herd restart nginx
>>> Service nginx is not running.
>>> herd: exception caught while executing 'start' on service 'nginx':
>>> Throw to key `srfi-34' with args `("#<condition &invoke-error [program: \"/gnu/store/mlg0xfbiq03s812rm3v7mrlhyngas4xp-nginx-1.17.1/sbin/nginx\" arguments: (\"-c\" \"/gnu/store/r6gl9n7pwf4npiri05qxr40vdihdm2yy-nginx.conf\" \"-p\" \"/var/run/nginx\") exit-status: 1 term-signal: #f stop-signal: #f] 147e000>")’.
>>
>> Do you use an “opaque” nginx config file, or do you use <nginx-...>
>> records?
>
> The latter I think:
>
> (service nginx-service-type
> (nginx-configuration
> (extra-content “…”)))
That’s actually the non-opaque variant, so shepherd should read the PID
file and it shouldn’t get it wrong. Not sure what happened.
If you can reproduce it, it would be great to gather the output of “herd
status nginx” at the time shepherd is confused.
Thanks,
Ludo’.
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#36731: shepherd lost track of nginx
2019-07-19 22:49 ` Ludovic Courtès
2019-07-20 7:42 ` Robert Vollmert
@ 2019-07-20 23:07 ` Mark H Weaver
2019-07-22 10:31 ` Ludovic Courtès
1 sibling, 1 reply; 6+ messages in thread
From: Mark H Weaver @ 2019-07-20 23:07 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 36731, Robert Vollmert
Hello,
Ludovic Courtès <ludo@gnu.org> writes:
> Robert Vollmert <rob@vllmrt.net> skribis:
>
>> The result was this:
>>
>> $ sudo herd restart nginx
>> Service nginx is not running.
>> herd: exception caught while executing 'start' on service 'nginx':
>> Throw to key `srfi-34' with args `("#<condition &invoke-error
>> [program:
>> \"/gnu/store/mlg0xfbiq03s812rm3v7mrlhyngas4xp-nginx-1.17.1/sbin/nginx\"
>> arguments: (\"-c\"
>> \"/gnu/store/r6gl9n7pwf4npiri05qxr40vdihdm2yy-nginx.conf\" \"-p\"
>> \"/var/run/nginx\") exit-status: 1 term-signal: #f stop-signal: #f]
>> 147e000>")’.
[…]
>> That error message could also be clearer about what’s going on. At any
>> rate, after I killed the nginx process, “herd start nginx” worked fine.
>
> I agree that we could and should improve the error message.
On the subject of this error message, why was the &invoke-error
condition serialized to a string before apparently being embedded within
another exception? In other words, why did it print:
Throw to key `srfi-34' with args `("#<condition &invoke-error [program: \"/gnu/store/mlg0xfbiq03s812rm3v7mrlhyngas4xp-nginx-1.17.1/sbin/nginx\" arguments: (\"-c\" \"/gnu/store/r6gl9n7pwf4npiri05qxr40vdihdm2yy-nginx.conf\" \"-p\" \"/var/run/nginx\") exit-status: 1 term-signal: #f stop-signal: #f] 147e000>")’.
instead of something closer to:
Throw to key `srfi-34' with args `(#<condition &invoke-error [program: "/gnu/store/mlg0xfbiq03s812rm3v7mrlhyngas4xp-nginx-1.17.1/sbin/nginx" arguments: ("-c" "/gnu/store/r6gl9n7pwf4npiri05qxr40vdihdm2yy-nginx.conf" "-p" "/var/run/nginx") exit-status: 1 term-signal: #f stop-signal: #f] 147e000>)’.
We may want to go further in this specific case to make a user-friendly
error message, but in the more general case of printing arbitrary
exceptions, eliminating that second layer of string serialization would
help make the error reports a bit nicer to read.
What do you think?
Mark
^ permalink raw reply [flat|nested] 6+ messages in thread* bug#36731: shepherd lost track of nginx
2019-07-20 23:07 ` Mark H Weaver
@ 2019-07-22 10:31 ` Ludovic Courtès
0 siblings, 0 replies; 6+ messages in thread
From: Ludovic Courtès @ 2019-07-22 10:31 UTC (permalink / raw)
To: Mark H Weaver; +Cc: 36731, Robert Vollmert
Hi Mark,
Mark H Weaver <mhw@netris.org> skribis:
> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Robert Vollmert <rob@vllmrt.net> skribis:
>>
>>> The result was this:
>>>
>>> $ sudo herd restart nginx
>>> Service nginx is not running.
>>> herd: exception caught while executing 'start' on service 'nginx':
>>> Throw to key `srfi-34' with args `("#<condition &invoke-error
>>> [program:
>>> \"/gnu/store/mlg0xfbiq03s812rm3v7mrlhyngas4xp-nginx-1.17.1/sbin/nginx\"
>>> arguments: (\"-c\"
>>> \"/gnu/store/r6gl9n7pwf4npiri05qxr40vdihdm2yy-nginx.conf\" \"-p\"
>>> \"/var/run/nginx\") exit-status: 1 term-signal: #f stop-signal: #f]
>>> 147e000>")’.
>
> […]
>
>>> That error message could also be clearer about what’s going on. At any
>>> rate, after I killed the nginx process, “herd start nginx” worked fine.
>>
>> I agree that we could and should improve the error message.
>
> On the subject of this error message, why was the &invoke-error
> condition serialized to a string before apparently being embedded within
> another exception?
That serialization comes from the Shepherd when it talks to its clients
(see ‘write-reply’ in (shepherd comm)).
Normally service methods should write a human-readable message instead
of throwing an exception, but when that happens, shepherd serializes
those things so that one can at least diagnose the problem.
In this case we could use ‘report-invoke-error’ from (guix build utils)
on ‘core-updates’.
Thanks,
Ludo’.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2019-07-22 10:32 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-07-19 16:49 bug#36731: shepherd lost track of nginx Robert Vollmert
2019-07-19 22:49 ` Ludovic Courtès
2019-07-20 7:42 ` Robert Vollmert
2019-07-20 13:51 ` Ludovic Courtès
2019-07-20 23:07 ` Mark H Weaver
2019-07-22 10:31 ` Ludovic Courtès
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).