From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:8:6d80::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id +C6oAfCgdmBcHwEAgWs5BA (envelope-from ) for ; Wed, 14 Apr 2021 09:59:44 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id WFTrNe+gdmC5XAAAB5/wlQ (envelope-from ) for ; Wed, 14 Apr 2021 07:59:43 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 1D6C0296B4 for ; Wed, 14 Apr 2021 09:59:43 +0200 (CEST) Received: from localhost ([::1]:43322 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lWaR3-0008SR-By for larch@yhetil.org; Wed, 14 Apr 2021 03:59:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47376) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lWaQt-0008SH-RH for help-guix@gnu.org; Wed, 14 Apr 2021 03:59:31 -0400 Received: from sender4-op-o10.zoho.com ([136.143.188.10]:17076) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lWaQr-0001RR-8U for help-guix@gnu.org; Wed, 14 Apr 2021 03:59:31 -0400 ARC-Seal: i=1; a=rsa-sha256; t=1618387165; cv=none; d=zohomail.com; s=zohoarc; b=ND007P0wQwEtiznuJ/mNf6/cS7bHd/8t+U4h/bcn0qbhoiXhEcGwC4aPdUgbmew0ONOrBFRjtHQurcrnsAb5TTg+qmVp4lx4F4fuft5HeKQrVbmFR152TZ1rvDuSrD+v7gzussBRH0fAvCFclhdbNyHfrYjoq7USDXBaOXSHLA8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1618387165; h=Content-Type:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; bh=Pj1T5/iYdne9VCJUj37egzGkRD8laF1It5zhlpGjlJM=; b=frjyVSrmwKHiBFqOjhlJHjElH0ywgUAsExz7KOJw9osMkHK6m9++nFO+VRGa7rixWZK4StoMvtN+JFjSWMXAsRjoPqUcWbH35Y0CSXQykBfog7ViIf6VfBNs5xBIn5d3vndJg/1xWpEKZm/iqpaV8INvVjx5ELn9OR65Q5LQTo8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=rdklein.fr; spf=pass smtp.mailfrom=edou@rdklein.fr; dmarc=pass header.from= header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1618387165; s=zoho; d=rdklein.fr; i=edou@rdklein.fr; h=References:From:To:Subject:In-reply-to:Message-ID:Date:MIME-Version:Content-Type; bh=Pj1T5/iYdne9VCJUj37egzGkRD8laF1It5zhlpGjlJM=; b=CGBQR7npOTk8rVTTgN7I2woqOD2I+FLDmm1tWAAEimz5nb0PVLI+qhsne4RGKRC7 EjdkndNQyXHu0QOQzVgRa7xvZwv2QcJjL1eC11sHXWjPBGYIJxo+L4+HLedOjwAF2mF IVwDKif3I+g0oE6C97pz0HDBCzaCtA+1uAOvem7s= Received: from Rasoir (lfbn-idf3-1-808-29.w90-3.abo.wanadoo.fr [90.3.133.29]) by mx.zohomail.com with SMTPS id 1618387162266257.44319342471; Wed, 14 Apr 2021 00:59:22 -0700 (PDT) References: <87r1jgsjme.fsf@rdklein.fr> <87lf9mosf4.fsf@rdklein.fr> <87k0p5lxmp.fsf@libre.brussels> User-agent: mu4e 1.4.15; emacs 27.1 From: Edouard Klein To: help-guix@gnu.org Subject: Re: Environment of a shepherd service In-reply-to: <87k0p5lxmp.fsf@libre.brussels> Message-ID: <87im4ppa9k.fsf@rdklein.fr> Date: Wed, 14 Apr 2021 09:59:04 +0200 MIME-Version: 1.0 Content-Type: text/plain X-ZohoMailClient: External Received-SPF: pass client-ip=136.143.188.10; envelope-from=edou@rdklein.fr; helo=sender4-op-o10.zoho.com X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: help-guix@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: help-guix-bounces+larch=yhetil.org@gnu.org Sender: "Help-Guix" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1618387183; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=Pj1T5/iYdne9VCJUj37egzGkRD8laF1It5zhlpGjlJM=; b=lYDRYledfQWj2OFeO39AqeZSXkg/A+VkX8HywtoWVWshOd2XQObEwroyCjHcByfes7Lwet Rp/JAKvPZuHqMJCnLbz8Ji7weMb2BzpgyzQv03xVecUw0CjqTvFqdcuuu4mEhgPLTKtV+r nxDFU3glhl/OgpWDVa2xUF3j++B6T2JWduxqrNw6Vw9IMywIawVpwE2B49r5yNX5lFhVOq CNFbh1in+3Gfw2w264G73WNcNqaO8Kmbt00oxStCIl+b2HfIxhLEpi17YZJ3jCitrCilFC h/7rEm5073MiZRTRh9b0EoEL5v+Pa5A0xCFbUC4bYlFdCYl+WsgH8a+6f4a2wQ== ARC-Seal: i=2; s=key1; d=yhetil.org; t=1618387183; a=rsa-sha256; cv=pass; b=M7xXQrRsLHd9HFXknWq6xD/bRctlhOPnMr5tIVDdKLZHOYGNRRNgvAqfJO7RP03T7iuC5Y eoDBGz18c87oC6Yzxs02mqh6ycSSaHRgyo3HpWUtOe+CXoLvLBqYfk6/OB3mW/2GnYytiD XFIlRTV5fywav93ryc5HfzjkMy0+SPPPhhU/z8VEbaogHz8qg7OjNZIo88V5yHq/fvUKnX GCpRYcC+2sME92yC4vPi8YBc195/DNoZ8hdj4OWZF8aNMcsKDNYT0YlxZmUZnrbXN6oaOU Fa4GpQc25i/BLG6zVyZiXOfqRVbUNRcvWdgcWD1drKHwwyTN2tOCjupu93U1Rw== ARC-Authentication-Results: i=2; aspmx1.migadu.com; dkim=none ("invalid DKIM record") header.d=rdklein.fr header.s=zoho header.b=CGBQR7np; arc=pass ("zohomail.com:s=zohoarc:i=1"); dmarc=none; spf=pass (aspmx1.migadu.com: domain of help-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=help-guix-bounces@gnu.org X-Migadu-Spam-Score: -3.44 Authentication-Results: aspmx1.migadu.com; dkim=none ("invalid DKIM record") header.d=rdklein.fr header.s=zoho header.b=CGBQR7np; arc=pass ("zohomail.com:s=zohoarc:i=1"); dmarc=none; spf=pass (aspmx1.migadu.com: domain of help-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=help-guix-bounces@gnu.org X-Migadu-Queue-Id: 1D6C0296B4 X-Spam-Score: -3.44 X-Migadu-Scanner: scn0.migadu.com X-TUID: YxA0u8F8fXa5 Hi ! I did, thanks :) But this post talks only about shepherd, not its integration with guix. It is indeed a very good resource for creating shepherd services, and I was able thanks to it to write a shepherd service. What I would like to do is create a shepherd service, using some facilities offered by the "operating-system declaration" concept of guix. In this instance I have trouble setting the env of the service to the search paths of the installed packages. The final goal is to orchestrate multiple services (typically multiple web services and an instance of nginx to reverse-proxy them all) in one operating-system declaration. Here are the resources I have found on this topic - https://www.gnu.org/software/shepherd/manual/shepherd.html#Services - https://guix.gnu.org/en/blog/2020/gnu-shepherd-user-services/ - https://archive.fosdem.org/2017/schedule/event/composingsystemservicesinguixsd/attachments/slides/1794/export/events/attachments/composingsystemservicesinguixsd/slides/1794/guix_service_composition.pdf - https://www.mndet.net/2016/05/04/guixsd-system-service.html - https://guix.gnu.org/en/manual/en/guix.html#Defining-Services The documentation is scarce (not an attack, I understand writing docs is harder than writing code), and I have trouble getting used to the source because I'm unable to get a guile repl at the level at which I'm trying to run code, so I have to make edits, build the container, try, repeat. If there is an error I have no stack trace or debug information. Cheers, Edouard. Jonathan McHugh writes: > Did you read this blog post?: > https://guix.gnu.org/en/blog/2020/gnu-shepherd-user-services/ > > ====== START ======== > The GNU Shepherd manual suggests putting all the services inside a monolithic init.scm file, located by default at $XDG_CONFIG_DIR/shepherd/init.scm. While this does make it easy to keep everything in one place, it does create one glaring issue: any changes to the file mean that all the services need to be stopped and restarted in order for any changes to take place. > > Luckily there's a nice function called scandir hiding in ice-9 ftw which returns a list of all files in a specified directory (with options for narrowing down the list or sorting it). This means that our init.scm can contain a minimum of code and all actual services can be loaded from individual files. > > First the minimal init.scm: > > (use-modules (shepherd service) > ((ice-9 ftw) #:select (scandir))) > > ;; Load all the files in the directory 'init.d' with a suffix '.scm'. > (for-each > (lambda (file) > (load (string-append "init.d/" file))) > (scandir (string-append (dirname (current-filename)) "/init.d") > (lambda (file) > (string-suffix? ".scm" file)))) > > ;; Send shepherd into the background > (action 'shepherd 'daemonize) > > Let's take a sample service for running syncthing, as defined in $XDG_CONFIG_DIR/shepherd/init.d/syncthing.scm: > > (define syncthing > (make > #:provides '(syncthing) > #:docstring "Run `syncthing' without calling the browser" > #:start (make-forkexec-constructor > '("syncthing" "-no-browser") > #:log-file (string-append (getenv "HOME") > "/log/syncthing.log")) > #:stop (make-kill-destructor) > #:respawn? #t)) > (register-services syncthing) > > (start syncthing) > > As with any other shepherd service it is defined and registered, and in > this case it will start automatically. When the file is loaded by > shepherd after being discovered by scandir everything works exactly as > though the service definition were located directly inside the init.scm. > > ======== END ======= > > > HTH > > > Edouard Klein writes: > >> Hi, >> >> So I looked at the source and I understand that there's no way around >> having only PATH=/run/current-system/profile/bin as the sole environment >> of a service (which makes me wonder how anyone is running any service in >> GuixSD, don't you need any env variables ?). >> >> I tried to define a trivial package that would use wrap-program to >> create a script that would set the environment variables to all the >> search-paths of my requisomatic package, but I don't know how to access >> those ! >> >> In the code that is executed by the daemon, all references to the >> package are lost, it is not in the same strata as the package. >> >> I can get the store path to the package but that does not help me. >> >> I really could use some guidance here. >> >> Cheers, >> >> Edouard. >> edk@beaver-labs.com writes: >> >>> Dear fellow Guixers, >>> >>> I'm trying to create an operating system declaration, so that I can run >>> a piece of software of mine in a container with =guix system container=. >>> >>> I wrote a package for the software. The package works: the tests pass >>> and when the package is installed I can run the software. >>> >>> I wrote a shepherd service for the software (it's called requisomatic). >>> I copied the relevant part at the end of the email. >>> >>> When I run the container script created by =guix system container=, and >>> get a shell in the container, I can run the software (I added the software's >>> package to the globally installed packages in the operating-system definition). >>> >>> But, when I try to run it with shepherd, it fails because it can't find >>> flask (a dependency of the software, which I've put as a >>> propagated-input, and is indeed installed in the container). >>> >>> I replaced the software invocation in the shepherd service with just >>> "env", and saw that the whole env in the service is: >>> >>> PATH=/run/current-system/profile/bin >>> >>> whereas in the shell I get when I connect to the container, the env >>> contains many other variables, including a correctly set PYTHONPATH, >>> which allows the finding of flask. >>> >>> So I now know why my software is not starting, but my question is: >>> >>> Why is the PYTHONPATH (and the other env vars, for that matter) not >>> propagated from the package to the shepherd service by default ? And how >>> can I make it so ? I would have expected the shepherd service to run >>> with the global profile active. >>> >>> Follow up question, can shepherd services be specified to run in a >>> specific profile ? So that I can have two services with incompatible >>> dependencies running at the same time in the same operating-system ? >>> >>> Thanks in advance, >>> >>> Cheers, >>> >>> Edouard. >>> >>> >>> >>> -----extract from my operating-system declaration file------- >>> (define requisomatic-shepherd-service >>> (match-lambda >>> (($ user group db-file) >>> (list (shepherd-service >>> (provision '(requisomatic)) >>> (requirement '(user-processes networking)) >>> (documentation "Run the requisomatic server") >>> (start #~((make-forkexec-constructor >>> ;; (append >>> ;; (if db-file >>> ;; `("env" >>> ;; ,(string-append "REQUISOMATIC_DB_FILE=" db-file)) >>> ;; '()) >>> '("gunicorn" "requisomatic:app") >>> ;; '("env") >>> ;;) >>> #:directory (string-append #$requisomatic "/bin/requisomatic/") >>> #:log-file "/var/log/requisomatic.log"))) >>> (stop #~(make-kill-destructor)))))))