bug#37631: service zabbix-server (and zabbix-agent) fails starting (cannot run as root!)

bug#37631: service zabbix-server (and zabbix-agent) fails starting (cannot run as root!)

[Giovanni Biscuolo]

Hi Guix,

executive summary: do we really need to start zabbix_server in
foreground mode?

I have a Guix System in which I defined this services (thanks to the
work in guix-maintenance):

--8<---------------cut here---------------start------------->8---
	   ;; For the Zabbix database.  It was created by manually
	   ;; following the instructions here:
	   ;; https://www.zabbix.com/documentation/4.2/manual/appendix/install/db_scripts
           (postgresql-service)

	   ;; Monitoring

	   (service zabbix-agent-service-type)

           (service zabbix-server-service-type
		    (zabbix-server-configuration
		     (include-files '("/root/secrets/zabbix-server-dbpass"))
		     (log-type "file")))

           (service zabbix-front-end-service-type
                    (zabbix-front-end-configuration
                     (nginx (list
                             (nginx-server-configuration
                              (root #~(string-append #$zabbix-server:front-end "/share/zabbix/php"))
                              (listen '("7878"))
			      (index '("index.php"))
			      (locations
			       (let ((php-location (nginx-php-location)))
				 (list (nginx-location-configuration
					(inherit php-location)
					(body (append (nginx-location-configuration-body php-location)
						      (list "
fastcgi_param PHP_VALUE \"post_max_size = 16M 
                          max_execution_time = 300\";
"))))))))))
                       (db-secret-file "/root/secrets/zabbix-front-end-dbpass"))))
--8<---------------cut here---------------end--------------->8---

The zabbix frontend service is running well but the zabbix-server
refuses to start

--8<---------------cut here---------------start------------->8---
$ herd start zabbix-server
Service zabbix-server could not be started.
herd: failed to start service zabbix-server
--8<---------------cut here---------------end--------------->8---

looking in the current system profile (built with a guix master branch
on 27 Sept)

--8<---------------cut here---------------start------------->8---
Generation 12	Sep 27 2019 21:18:26	(current)
  file name: /var/guix/profiles/system-12-link
  canonical file name: /gnu/store/h03qdv70sgndclgp04dpkka4rqlk9fg3-system
  label: GNU with Linux-Libre 5.2.17
  bootloader: grub
  root device: UUID: 9862e534-946d-4323-b7ce-9937661bdb7d
  kernel: /gnu/store/bjs8k11phqhn39n7cs1wix5x147fwhnn-linux-libre-5.2.17/bzImage
--8<---------------cut here---------------end--------------->8---

I found the shepherd uses
/gnu/store/lm1d60d0kra3z86hcjmav828cfxjcgi8-shepherd-zabbix-server.scm
with this (partial) parameters:

--8<---------------cut here---------------start------------->8---
#:start (make-forkexec-constructor (list "/gnu/store/qcm5j0wk8rs6ykn6b10vg8awf2v6kvx1-zabbix-server-4.2.0/sbin/zabbix_server" "--config" "/gnu/store/w1vgvlbzs3jks014r5dra7ih6g7r26n7-zabbix_server.conf" "--foreground") #:user "zabbix" #:group "zabbix" #:pid-file "/var/run/zabbix/zabbix_server.pid"
--8<---------------cut here---------------end--------------->8---

and if I try to start it from the command line:

--8<---------------cut here---------------start------------->8---
/gnu/store/qcm5j0wk8rs6ykn6b10vg8awf2v6kvx1-zabbix-server-4.2.0/sbin/zabbix_server --config /gnu/store/w1vgvlbzs3jks014r5dra7ih6g7r26n7-zabbix_server.conf --foreground
--8<---------------cut here---------------end--------------->8---

I get:

--8<---------------cut here---------------start------------->8---
zabbix_server [879]: cannot run as root!
--8<---------------cut here---------------end--------------->8---

I had a look in upstream bug reports but was not able to find nothing
strictly related to zabbix_server, but I was able fo find this for
zabbix_agentd https://support.zabbix.com/browse/ZBX-10611 (fixed since
4.2.1rc1)

actually if I start zabbix_server without ``--foreground'' the server
starts without problems

I thought upgrading to the last stable release of zabbix was the
solution, so I submitted a patch (bug#37629) to upgrade to 4.2.7 and now
I'm using a custom channel with that patch applied:

--8<---------------cut here---------------start------------->8---
(list (channel
        (name 'guix)
	(url "https://gitlab.com/gbiscuolo/guix.git")
	(branch "wip-zabbix-update")))
--8<---------------cut here---------------end--------------->8---

but if I switch to my last system generation (built with the above channel):

--8<---------------cut here---------------start------------->8---
Generation 13	Oct 05 2019 10:24:28
  file name: /var/guix/profiles/system-13-link
  canonical file name: /gnu/store/bmmjbk6sidqjahq0i53mgp38b342lnda-system
  label: GNU with Linux-Libre 5.3.2
  bootloader: grub
  root device: UUID: 9862e534-946d-4323-b7ce-9937661bdb7d
  kernel: /gnu/store/b28yv4dww1fha0hdhxibfg0k1f50sy0f-linux-libre-5.3.2/bzImage
--8<---------------cut here---------------end--------------->8---

and reboot...

--8<---------------cut here---------------start------------->8---
$ sudo herd status zabbix-server
Status of zabbix-server:
It is stopped.
It is enabled.
Provides (zabbix-server).
Requires ().
Conflicts with ().
Will be respawned.
--8<---------------cut here---------------end--------------->8---

this time if I start zabbix_server from the command line:

--8<---------------cut here---------------start------------->8---
 $ /gnu/store/94w5smlc86xac7v1vz4wbqkq43b2fia8-zabbix-server-4.2.7/sbin/zabbix_server --config /gnu/store/w1vgvlbzs3jks014r5dra7ih6g7r26n7-zabbix_server.conf --foreground
Starting Zabbix Server. Zabbix 4.2.7 (revision 624fb7497b).
Press Ctrl+C to exit.

--8<---------------cut here---------------end--------------->8---

the server starts but the console remains open (as expected by using of
foreground) and if I press Ctrl+C obviously the server is terminated

AFAIU the foreground process never exits and shepherd times it out:
right?

the same thing **almost** apply to zabbix-agent that is also
started with "--foregroud": it is started by shepherd, the console gets
busy (so I cannot login for exemple) but if I press Ctrl+C the process
keeps running

--8<---------------cut here---------------start------------->8---
$ sudo herd status zabbix-agent
Status of zabbix-agent:
  It is started.
  Running value is 277.
  It is enabled.
  Provides (zabbix-agent).
  Requires ().
  Conflicts with ().
  Will be respawned.
--8<---------------cut here---------------end--------------->8---

strange behaviour

anyway back to the question: do we need to start zabbix_server and
zabbix_agentd in foreground?

Thanks! Gio'

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

bug#37631: service zabbix-server (and zabbix-agent) fails starting (cannot run as root!)

[Giovanni Biscuolo]

Giovanni Biscuolo <g@xelera.eu> writes:

> executive summary: do we really need to start zabbix_server in
> foreground mode?

executive answer: I don't know **but** this is not the cause of my issue
:)

> --8<---------------cut here---------------start------------->8---

[...]

>            (service zabbix-server-service-type
> 		    (zabbix-server-configuration
> 		     (include-files '("/root/secrets/zabbix-server-dbpass"))
> 		     (log-type "file")))

ouch!... looking at the console (it's a remote VM so I usually connect
via ssh only, but today I also connected via SPICE):

--8<---------------cut here---------------start------------->8---
zabbix_server [1942]: /root/secrets/zabbix-server-dbpass: [13] Permission denied
--8<---------------cut here---------------end--------------->8---

unfortunately shepherd did not catch this error (due to foreground
mode?) in syslog :-(

I just had to adjust the permissions to allow zabbix (I allowed the
zabbix group to traverse /root/secrets and read the file) to read the
included file

this now works with both zabbix 4.2.0 and zabbix 4.2.7

[...]

Thanks! Gio'

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

bug#37631: service zabbix-server (and zabbix-agent) fails starting (cannot run as root!)

[Gábor Boskovits]

[-- Attachment #1: Type: text/plain, Size: 1574 bytes --]

Hello Giovanni,

Giovanni Biscuolo <g@xelera.eu> ezt írta (időpont: 2019. okt. 5., Szo,
14:51):

> Giovanni Biscuolo <g@xelera.eu> writes:
>
> > executive summary: do we really need to start zabbix_server in
> > foreground mode?
>
> executive answer: I don't know **but** this is not the cause of my issue
> :)
>
> > --8<---------------cut here---------------start------------->8---
>
> [...]
>
> >            (service zabbix-server-service-type
> >                   (zabbix-server-configuration
> >                    (include-files
> '("/root/secrets/zabbix-server-dbpass"))
> >                    (log-type "file")))
>
> ouch!... looking at the console (it's a remote VM so I usually connect
> via ssh only, but today I also connected via SPICE):
>
> --8<---------------cut here---------------start------------->8---
> zabbix_server [1942]: /root/secrets/zabbix-server-dbpass: [13] Permission
> denied
> --8<---------------cut here---------------end--------------->8---
>
> unfortunately shepherd did not catch this error (due to foreground
> mode?) in syslog :-(
>
> I just had to adjust the permissions to allow zabbix (I allowed the
> zabbix group to traverse /root/secrets and read the file) to read the
> included file
>
> this now works with both zabbix 4.2.0 and zabbix 4.2.7
>
> [...]
>
> Thanks! Gio'
>
> --
> Giovanni Biscuolo
>
> Xelera IT Infrastructures
>
>
>
> Can we consider this resolved then?

Best regards,
g_bor
-- 
OpenPGP Key Fingerprint: 7988:3B9F:7D6A:4DBF:3719:0367:2506:A96C:CF63:0B21

[-- Attachment #2: Type: text/html, Size: 2259 bytes --]

bug#37631: service zabbix-server (and zabbix-agent) fails starting (cannot run as root!)

[Giovanni Biscuolo]

Gábor Boskovits <boskovits@gmail.com> writes:

[...]

>> Can we consider this resolved then?

oh yes sorry, forgot to (auto) close this bug as done: this message
should do it

Thanks! Gio'

[...]

-- 
Giovanni Biscuolo

Xelera IT Infrastructures