From: Edouard Klein <edou@rdklein.fr>
To: Maxime Devos <maximedevos@telenet.be>
Cc: help-guix@gnu.org
Subject: Re: Environment of a shepherd service
Date: Tue, 27 Apr 2021 22:03:08 +0200 [thread overview]
Message-ID: <877dko5mgc.fsf@rdklein.fr> (raw)
In-Reply-To: <75a95dd5315d1633452694f824b42141989d9c03.camel@telenet.be>
Thank you Maxime for your answer :)
Maxime Devos writes:
> edk@beaver-labs.com schreef op zo 11-04-2021 om 21:31 [+0200]:
>> Dear fellow Guixers,
>
>> [...]
>> But, when I try to run it with shepherd, it fails because it can't find
>> flask (a dependency of the software, which I've put as a
>> propagated-input, and is indeed installed in the container).
>
> Propagated inputs can be inconvenient at times. I would advise
> looking where requisomatic is referring to flask, and replacing
> flask --> (string-append (assoc-ref inputs "flask") "/bin/flask")
> using substitute*.
OK, I understand this part :) The executable is gunicorn, so I'll just
use its full path in the script that launches the service.
My previous solution was to source the profile beforehand:
https://gitlab.com/edouardklein/requisomatic/-/blob/baf3fe51ad8bbabbcbc467dff92ec02c43e6daf1/guix.scm#L200
I do agree that a rich profile feels dirty and I too don't like
propagated inputs too much, but with Python I don't understand what the
best way to do it is ?
For example, in order to package a Python application in a way that does
not break the host system, I had to put all the search-paths in a
wrapper script and change all propagated inputs as just inputs. I find
the resulting code quite ugly, and I don't think I'm doing things right:
https://gitlab.com/edouardklein/gendscraper/-/blob/980f2597cc36bc79a9be7af5814e0fcf2313b677/gendscraper.scm#L1176
I wouldn't like to have to this for every service. If propagated inputs
are inconvenient, and if services run in an environment where the
search-paths of the installed packages are not available, how can we
easily tell the software where to find its dynamically-loaded parts ?
>
>> But, when I try to run it with shepherd, it fails because it can't find
>> flask (a dependency of the software, which I've put as a
>> propagated-input, and is indeed installed in the container).
>> [...]
>
> Some advice (warning: I'm not familiar with gunicorn or requisomatic
> at all).
I used flask and gunicorn in this project, but I'm not familiar with
them either ;)
>
>>
>> -----extract from my operating-system declaration file-------
>> (define requisomatic-shepherd-service
>> ([...] (shepherd-service
>> [...]
> (documentation "Run the requisomatic server")
>> (start #~((make-forkexec-constructor
>> ;; (append
>> ;; (if db-file
>> ;; `("env"
>> ;; ,(string-append "REQUISOMATIC_DB_FILE=" db-file))
>> ;; '())
>> '("gunicorn" "requisomatic:app")
>
> Normally, services refer by absolute path to the binary to run, and not rely
> on the PATH (the latter would require polluting the system profile).
> Idiomatically, one would write
>
> '(#$(file-append gunicorn "/bin/gunicorn")
> #$(file-append requisomatic) "/wherever/the/binary/is")
>
I did not know about file-append even though it is in the manual. Thanks :)
>> #:directory (string-append #$requisomatic "/bin/requisomatic/")
>
> Why are you changing the working directory to
> (string-append #$requisomatic "/bin/requisomatic/"),
because . is by default where flask will look for the code it needs.
>and why is "/bin/requisomatic" a
> directory and not an executable?
Because flask/jinja/etc. are normative about where they want stuff to be
and I put everything in a single dir. As most of the assets is code
(some are html templates) I put it in bin/... but I could put them in
/lib of wherever. This is not the cleanest, but I really wanted to get a
guix operating system declaration to work before I did things cleanly.
> Is that a gunicorn thing?
More like flask, but yeah, it's a framework thing.
>
>> Why is the PYTHONPATH (and the other env vars, for that matter) not
>> propagated from the package to the shepherd service by default ?
>
> How is the shepherd service supposed to automagically know which packages
> to include in the environment variables?
>
I realised afterwards that by extending the profile-service-type,
services can say what packages they need (I currently add them to the
operating-system definition). I would expect the shepherd
command to run in an environment where the search-paths of those package
is set. I have no idea if this is complicated to do, but as a service
writer I would enjoy it very much.
>> And how can I make it so ?
>
> Use the #:environment-variables option, see e.g.
> bitlbee-shepherd-service
I see the concept, but the list is hardcoded, and for even moderately
complex Python application (or any other dynamic language) the list of
env variables to set will become huge.
> Or create a wrapper. See e.g. wrapped-dbus-service.
This looks like a clean version of what I did for gendscraper, but the
problem remains of which variable to set. This information exists in
each individual package. Some will need additions to e.g. PYTHONPATH,
others to LD_LIBRARY_PATH, etc. We need to walk the entire dependency
DAG somehow.
>
>> Follow up question, can shepherd services be specified to run in a
>> specific profile ?
>
> IIUC, currently shepherd services aren't run in *any* profile at all.
> It would be useful to have a function manifest->environment-gexp
> though.
Or package->search-paths ?
>
>> So that I can have two services with incompatible
>> dependencies running at the same time in the same operating-system ?
> Yes, it with "dependencies" you mean packages, and not other services.
>
I meant packages :)
Thank you very much for your help, it seems you know services inside
out. I can now pinpoint way better what it is that I don't understand. I
hope I was more clear in expressing it.
Cheers !
Edouard.
> Greetings,
> Maxime.
prev parent reply other threads:[~2021-04-27 20:05 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-11 19:31 Environment of a shepherd service edk
2021-04-13 20:12 ` Edouard Klein
2021-04-13 20:47 ` Jonathan McHugh
2021-04-14 7:59 ` Edouard Klein
2021-04-25 15:36 ` Maxime Devos
2021-04-27 20:03 ` Edouard Klein [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=877dko5mgc.fsf@rdklein.fr \
--to=edou@rdklein.fr \
--cc=help-guix@gnu.org \
--cc=maximedevos@telenet.be \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).