* [bug#56677] [PATCH 0/2] environment: Add --emulate-fhs option.
@ 2022-07-21 4:14 John Kehayias via Guix-patches via
2022-07-21 4:19 ` [bug#56677] [PATCH 1/2] gnu: Add glibc-for-fhs John Kehayias via Guix-patches via
` (2 more replies)
0 siblings, 3 replies; 10+ messages in thread
From: John Kehayias via Guix-patches via @ 2022-07-21 4:14 UTC (permalink / raw)
To: 56677
Hello Guix,
As discussed on guix-devel here (please see for more detailed discussion and design aims): https://lists.gnu.org/r/guix-devel/2022-07/msg00161.html this is a patch to add an FHS (Filesystem Hierarchy Standard) emulation option for environments.
The overall goal is to mimic typical GNU/Linux distributions in following FHS (/bin, /etc, and /usr in particular) as well as a glibc that reads a global /etc/ld.so.cache and PATH with /bin, and so on. The idea is that following instructions for setting up a development environment, building software, running something, and so on in "typical" Linux environments, should "just work" with 'guix shell --container --emulate-fhs ...', provided the right inputs and other options are set.
For testers, this can be used by using pre-inst-env (outside of the pure shell used to build a local guix) to run guix shell with this patch. Please see the mailing list discussion for particular examples as well.
For review, in particular:
1. On the mailing list there was discussion about the necessity or not of glibc-for-fhs (added in the first patch). I find this useful and a big piece of making this FHS option work, but open to discussion or if it should be a further option.
2. Right now I used a script written to the containers /tmp/fhs.sh to generate the ld cache, supplement $PATH (somewhat optional, but I found useful for less tinkering), and finally launch the given command or shell. I found that when not providing a command the prompt for /bin/sh is not the same as when not using --emulate-fhs. So I'm not sure if this is the correct way to launch the default /bin/sh if no command is given. Open to ideas of a better way to implement these actions for a container start up as well.
3. This is my first time touching a guix script and the documentation, so please do check the commit message and guix.texi.
4. I decided to link the second level FHS directories, like /usr/bin, as well as optional ones like /lib64 (or /lib32), to the top level /bin, /lib, and so on. These could just be bind mounted to profile/bin and so on as well, but again tried to mimic an FHS distribution like Arch where the files only live in one place. While perhaps making the code a little more involved, I hope this makes the container look tidier.
I may be forgetting other elements in the implementation decisions I made, but I have been testing these patches along the way and have gotten good usage of them. Please test further too!
Thanks,
John
^ permalink raw reply [flat|nested] 10+ messages in thread
* [bug#56677] [PATCH 1/2] gnu: Add glibc-for-fhs.
2022-07-21 4:14 [bug#56677] [PATCH 0/2] environment: Add --emulate-fhs option John Kehayias via Guix-patches via
@ 2022-07-21 4:19 ` John Kehayias via Guix-patches via
2022-07-21 4:19 ` [bug#56677] [PATCH 2/2] environment: Add '--emulate-fhs' John Kehayias via Guix-patches via
2022-08-04 10:36 ` [bug#56677] [PATCH 0/2] environment: Add --emulate-fhs option Ludovic Courtès
2 siblings, 0 replies; 10+ messages in thread
From: John Kehayias via Guix-patches via @ 2022-07-21 4:19 UTC (permalink / raw)
To: 56677@debbugs.gnu.org
[-- Attachment #1: Type: text/plain, Size: 13 bytes --]
Empty Message
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-gnu-Add-glibc-for-fhs.patch --]
[-- Type: text/x-patch; name=0001-gnu-Add-glibc-for-fhs.patch, Size: 1832 bytes --]
From ac94f4203423c625c06018212677a798ea46b417 Mon Sep 17 00:00:00 2001
From: John Kehayias <john.kehayias@protonmail.com>
Date: Wed, 20 Jul 2022 23:26:53 -0400
Subject: [PATCH 1/2] gnu: Add glibc-for-fhs.
* gnu/packages/base.scm (glibc-for-fhs): New variable. Hidden package for use
in forthcoming --emulate-fhs option for guix containers.
---
gnu/packages/base.scm | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index 4bdc3e7792..b566057b41 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -20,6 +20,7 @@
;;; Copyright © 2021 Maxime Devos <maximedevos@telenet.be>
;;; Copyright © 2021 Guillaume Le Vaillant <glv@posteo.net>
;;; Copyright © 2021 Maxim Cournoyer <maxim.cournoyer@gmail.com>
+;;; Copyright © 2022 John Kehayias <john.kehayias@protonmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -928,6 +929,20 @@ (define-public glibc
(license lgpl2.0+)
(home-page "https://www.gnu.org/software/libc/")))
+;; Define a variation of glibc which uses the default /etc/ld.so.cache, useful
+;; in FHS containers.
+(define-public glibc-for-fhs
+ (hidden-package
+ (package
+ (inherit glibc)
+ (name "glibc-for-fhs")
+ (source (origin (inherit (package-source glibc))
+ ;; Remove Guix's patch to read ld.so.cache from /gnu/store
+ ;; directories, re-enabling the default /etc/ld.so.cache
+ ;; behavior.
+ (patches (delete (car (search-patches "glibc-dl-cache.patch"))
+ (origin-patches (package-source glibc)))))))))
+
;; Below are old libc versions, which we use mostly to build locale data in
;; the old format (which the new libc cannot cope with.)
(define-public glibc-2.32
--
2.37.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [bug#56677] [PATCH 2/2] environment: Add '--emulate-fhs'.
2022-07-21 4:14 [bug#56677] [PATCH 0/2] environment: Add --emulate-fhs option John Kehayias via Guix-patches via
2022-07-21 4:19 ` [bug#56677] [PATCH 1/2] gnu: Add glibc-for-fhs John Kehayias via Guix-patches via
@ 2022-07-21 4:19 ` John Kehayias via Guix-patches via
2022-08-02 14:27 ` [bug#56677] [PATCH 2/2 v2] " John Kehayias via Guix-patches via
2022-08-04 10:36 ` [bug#56677] [PATCH 0/2] environment: Add --emulate-fhs option Ludovic Courtès
2 siblings, 1 reply; 10+ messages in thread
From: John Kehayias via Guix-patches via @ 2022-07-21 4:19 UTC (permalink / raw)
To: 56677@debbugs.gnu.org
[-- Attachment #1: Type: text/plain, Size: 13 bytes --]
Empty Message
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0002-environment-Add-emulate-fhs.patch --]
[-- Type: text/x-patch; name=0002-environment-Add-emulate-fhs.patch, Size: 16457 bytes --]
From 27d40cbd60471b79dc1692f7db4aef495b93493d Mon Sep 17 00:00:00 2001
From: John Kehayias <john.kehayias@protonmail.com>
Date: Wed, 20 Jul 2022 23:46:45 -0400
Subject: [PATCH 2/2] environment: Add '--emulate-fhs'.
* guix/scripts/environment.scm (show-environment-options-help)
(%options): Add '--emulate-fhs'.
* guix/scripts/environment.scm (launch-environment/container): Add
'emulate-fhs?' key and implement it. Define and use FHS-MAPPINGS,
FHS-SYMLINKS, and LINK-CONTENTS to set up the container to follow the
Filesystem Hierarchy Standard (FHS) for /bin, /etc, and /usr. Generate
/etc/ld.so.cache in the container from /etc/ld.so.conf by running the script
/tmp/fhs.sh to launch the container.
(guix-environment*): Add glibc-for-fhs to the container packages when
'emulate-fhs?' key is in OPTS.
* doc/guix.texi (Invoking guix shell): Document '--emulate-fhs'.
(Invoking guix environment): Document '--emulate-fhs'.
---
doc/guix.texi | 33 ++++++++
guix/scripts/environment.scm | 156 ++++++++++++++++++++++++++++++-----
2 files changed, 167 insertions(+), 22 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 3c5864ec1a..ab3967b2e5 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -106,6 +106,7 @@
Copyright @copyright{} 2022 Karl Hallsby@*
Copyright @copyright{} 2022 Justin Veilleux@*
Copyright @copyright{} 2022 Reily Siegel@*
+Copyright @copyright{} 2022 John Kehayias@*
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3 or
@@ -6155,6 +6156,22 @@ Invoking guix shell
guix shell --container --expose=$HOME=/exchange guile -- guile
@end example
+@item --emulate-fhs
+@item -F
+For containers, emulate a Filesystem Hierarchy Standard (FHS)
+configuration within the container, see
+@uref{https://refspecs.linuxfoundation.org/fhs.shtml, the official
+specification}. As Guix deviates from the FHS specification, this
+option sets up the container to more closely mimic that of other
+GNU/Linux distributions. This is useful for reproducing other
+development environments, testing, and using programs which expect the
+FHS specification to be followed. With this option, the container will
+include a version of @code{glibc} which will read
+@code{/etc/ld.so.cache} within the container for the shared library
+cache (contrary to @code{glibc} in regular Guix usage) and set up the
+expected FHS directories: @code{/bin}, @code{/etc}, @code{/lib}, and
+@code{/usr} from the container's profile.
+
@item --rebuild-cache
@cindex caching, of profiles
@cindex caching, in @command{guix shell}
@@ -6574,6 +6591,22 @@ Invoking guix environment
@end table
+@item --emulate-fhs
+@item -F
+For containers, emulate a Filesystem Hierarchy Standard (FHS)
+configuration within the container, see
+@uref{https://refspecs.linuxfoundation.org/fhs.shtml, the official
+specification}. As Guix deviates from the FHS specification, this
+option sets up the container to more closely mimic that of other
+GNU/Linux distributions. This is useful for reproducing other
+development environments, testing, and using programs which expect the
+FHS specification to be followed. With this option, the container will
+include a version of @code{glibc} which will read
+@code{/etc/ld.so.cache} within the container for the shared library
+cache (contrary to @code{glibc} in regular Guix usage) and set up the
+expected FHS directories: @code{/bin}, @code{/etc}, @code{/lib}, and
+@code{/usr} from the container's profile.
+
@command{guix environment}
also supports all of the common build options that @command{guix
build} supports (@pxref{Common Build Options}) as well as package
diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scm
index 3216235937..f943cd89d9 100644
--- a/guix/scripts/environment.scm
+++ b/guix/scripts/environment.scm
@@ -2,6 +2,7 @@
;;; Copyright © 2014, 2015, 2018 David Thompson <davet@gnu.org>
;;; Copyright © 2015-2022 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2018 Mike Gerwitz <mtg@gnu.org>
+;;; Copyright © 2022 John Kehayias <john.kehayias@protonmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -120,6 +121,9 @@ (define (show-environment-options-help)
--expose=SPEC for containers, expose read-only host file system
according to SPEC"))
(display (G_ "
+ -F, --emulate-fhs for containers, emulate the Filesystem Hierarchy
+ Standard (FHS)"))
+ (display (G_ "
-v, --verbosity=LEVEL use the given verbosity LEVEL"))
(display (G_ "
--bootstrap use bootstrap binaries to build the environment")))
@@ -256,6 +260,9 @@ (define %options
(alist-cons 'file-system-mapping
(specification->file-system-mapping arg #f)
result)))
+ (option '(#\F "emulate-fhs") #f #f
+ (lambda (opt name arg result)
+ (alist-cons 'emulate-fhs? #t result)))
(option '(#\r "root") #t #f
(lambda (opt name arg result)
(alist-cons 'gc-root arg result)))
@@ -608,16 +615,18 @@ (define* (launch-environment/fork command profile manifest
(define* (launch-environment/container #:key command bash user user-mappings
profile manifest link-profile? network?
- map-cwd? (white-list '()))
+ map-cwd? emulate-fhs? (white-list '()))
"Run COMMAND within a container that features the software in PROFILE.
-Environment variables are set according to the search paths of MANIFEST.
-The global shell is BASH, a file name for a GNU Bash binary in the
-store. When NETWORK?, access to the host system network is permitted.
-USER-MAPPINGS, a list of file system mappings, contains the user-specified
-host file systems to mount inside the container. If USER is not #f, each
-target of USER-MAPPINGS will be re-written relative to '/home/USER', and USER
-will be used for the passwd entry. LINK-PROFILE? creates a symbolic link from
-~/.guix-profile to the environment profile.
+Environment variables are set according to the search paths of MANIFEST. The
+global shell is BASH, a file name for a GNU Bash binary in the store. When
+NETWORK?, access to the host system network is permitted. USER-MAPPINGS, a
+list of file system mappings, contains the user-specified host file systems to
+mount inside the container. If USER is not #f, each target of USER-MAPPINGS
+will be re-written relative to '/home/USER', and USER will be used for the
+passwd entry. When EMULATE-FHS?, set up the container to follow the
+Filesystem Hierarchy Standard and provide a glibc that reads the cache from
+/etc/ld.so.cache. LINK-PROFILE? creates a symbolic link from ~/.guix-profile
+to the environment profile.
Preserve environment variables whose name matches the one of the regexps in
WHILE-LIST."
@@ -625,6 +634,40 @@ (define* (launch-environment/container #:key command bash user user-mappings
(and (file-exists? (file-system-mapping-source mapping))
(file-system-mapping->bind-mount mapping)))
+ ;; File system mappings for an FHS container, where the entire directory can
+ ;; be mapped. Others (bin and etc) will already have contents and need to
+ ;; use LINK-CONTENTS to symlink the directory contents.
+ (define fhs-mappings
+ (map (lambda (mapping)
+ (file-system-mapping
+ (source (string-append profile (car mapping)))
+ (target (cdr mapping))))
+ '(("/lib" . "/lib")
+ ("/include" . "/usr/include")
+ ("/sbin" . "/sbin")
+ ("/libexec" . "/usr/libexec")
+ ("/share" . "/usr/share"))))
+
+ ;; Additional (optional) symlinks for an FHS container.
+ (define fhs-symlinks
+ `(("/lib" . "/usr/lib")
+ ,(if (target-64bit?)
+ '("/lib" . "/lib64")
+ '("/lib" . "/lib32"))
+ ("/bin" . "/usr/bin")
+ ("/sbin" . "/usr/sbin")))
+
+ ;; A procedure to symlink the contents (at the top level) of a directory,
+ ;; excluding the directory itself and parent, along with any others provided
+ ;; in EXCLUDE.
+ (define* (link-contents dir #:key (exclude '()))
+ (for-each (lambda (file)
+ (symlink (string-append profile dir "/" file)
+ (string-append dir "/" file)))
+ (scandir (string-append profile dir)
+ (negate (cut member <>
+ (append exclude '("." ".." )))))))
+
(define (exit/status* status)
(exit/status (validate-exit-status profile command status)))
@@ -682,6 +725,11 @@ (define* (launch-environment/container #:key command bash user user-mappings
(filter-map optional-mapping->fs
%network-file-mappings)
'())
+ ;; Mappings for an FHS container.
+ (if emulate-fhs?
+ (filter-map optional-mapping->fs
+ fhs-mappings)
+ '())
(map file-system-mapping->bind-mount
mappings))))
(exit/status*
@@ -709,6 +757,53 @@ (define* (launch-environment/container #:key command bash user user-mappings
(mkdir-p home-dir)
(setenv "HOME" home-dir)
+ ;; Set up an FHS container.
+ (when emulate-fhs?
+ ;; The FHS container sets up the expected filesystem through
+ ;; MAPPINGS above, the optional symlinks, and linking the
+ ;; contents of profile/bin and profile/etc, as these both have
+ ;; or will have contents for a non-FHS container so must be
+ ;; handled separately.
+ (mkdir-p "/usr")
+ (for-each (lambda (link)
+ (if (file-exists? (car link))
+ (symlink (car link) (cdr link))))
+ fhs-symlinks)
+ (link-contents "/bin" #:exclude '("sh"))
+ (mkdir-p "/etc")
+ (link-contents "/etc")
+
+ ;; Provide a frequently expected 'cc' symlink to gcc (in case it
+ ;; is in the container), though this could also be done by the
+ ;; user in the container, e.g. in $HOME/.local/bin and adding
+ ;; that to $PATH. Note: we do this in /bin since that already
+ ;; has the sh symlink and the other (optional) FHS bin
+ ;; directories will link to /bin.
+ (symlink (string-append profile "/bin/gcc") "/bin/cc")
+
+ ;; Guix's ldconfig doesn't seem to search in FHS default
+ ;; locations, so provide a minimal ld.so.conf.
+ (call-with-output-file "/etc/ld.so.conf"
+ (lambda (port)
+ (for-each (lambda (directory)
+ (display directory port)
+ (newline port))
+ ;; /lib/nss is needed as Guix's nss puts libraries
+ ;; there rather than in the lib directory.
+ '("/lib" "/lib/nss"))))
+
+ ;; Define an entry script to start the container: generate
+ ;; ld.so.cache, supplement $PATH (optional, but to better match
+ ;; FHS expectations), and include COMMAND.
+ (call-with-output-file "/tmp/fhs.sh"
+ (lambda (port)
+ (display "ldconfig -X" port)
+ (newline port)
+ (display "export PATH=/bin:/usr/bin:/sbin:/usr/sbin:$PATH" port)
+ (newline port)
+ (display (car command) port)
+ (newline port))))
+
;; If requested, link $GUIX_ENVIRONMENT to $HOME/.guix-profile;
;; this allows programs expecting that path to continue working as
;; expected within a container.
@@ -746,7 +841,10 @@ (define* (launch-environment/container #:key command bash user user-mappings
(primitive-exit/status
;; A container's environment is already purified, so no need to
;; request it be purified again.
- (launch-environment command
+ (launch-environment (if emulate-fhs?
+ ;; Use the FHS start script.
+ '("/bin/sh" "/tmp/fhs.sh")
+ command)
(if link-profile?
(string-append home-dir "/.guix-profile")
profile)
@@ -874,16 +972,17 @@ (define (guix-environment* opts)
"Run the 'guix environment' command on OPTS, an alist resulting for
command-line option processing with 'parse-command-line'."
(with-error-handling
- (let* ((pure? (assoc-ref opts 'pure))
- (container? (assoc-ref opts 'container?))
- (link-prof? (assoc-ref opts 'link-profile?))
- (network? (assoc-ref opts 'network?))
- (no-cwd? (assoc-ref opts 'no-cwd?))
- (user (assoc-ref opts 'user))
- (bootstrap? (assoc-ref opts 'bootstrap?))
- (system (assoc-ref opts 'system))
- (profile (assoc-ref opts 'profile))
- (command (or (assoc-ref opts 'exec)
+ (let* ((pure? (assoc-ref opts 'pure))
+ (container? (assoc-ref opts 'container?))
+ (link-prof? (assoc-ref opts 'link-profile?))
+ (network? (assoc-ref opts 'network?))
+ (no-cwd? (assoc-ref opts 'no-cwd?))
+ (emulate-fhs? (assoc-ref opts 'emulate-fhs?))
+ (user (assoc-ref opts 'user))
+ (bootstrap? (assoc-ref opts 'bootstrap?))
+ (system (assoc-ref opts 'system))
+ (profile (assoc-ref opts 'profile))
+ (command (or (assoc-ref opts 'exec)
;; Spawn a shell if the user didn't specify
;; anything in particular.
(if container?
@@ -922,12 +1021,24 @@ (define (guix-environment* opts)
(leave (G_ "'--user' cannot be used without '--container'~%")))
(when (and (not container?) no-cwd?)
(leave (G_ "--no-cwd cannot be used without --container~%")))
+ (when (and (not container?) emulate-fhs?)
+ (leave (G_ "'--emulate-fhs' cannot be used without '--container~'%")))
(with-store/maybe store
(with-status-verbosity (assoc-ref opts 'verbosity)
(define manifest-from-opts
- (options/resolve-packages store opts))
+ (options/resolve-packages store
+ ;; For an FHS-container, add the
+ ;; (hidden) package glibc-for-fhs which
+ ;; uses the global cache at
+ ;; /etc/ld.so.cache.
+ (if emulate-fhs?
+ (alist-cons 'expression
+ '(ad-hoc-package
+ "(@@ (gnu packages base) glibc-for-fhs)")
+ opts)
+ opts)))
(define manifest
(if profile
@@ -1001,7 +1112,8 @@ (define (guix-environment* opts)
#:white-list white-list
#:link-profile? link-prof?
#:network? network?
- #:map-cwd? (not no-cwd?))))
+ #:map-cwd? (not no-cwd?)
+ #:emulate-fhs? emulate-fhs?)))
(else
(return
--
2.37.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [bug#56677] [PATCH 2/2 v2] environment: Add '--emulate-fhs'.
2022-07-21 4:19 ` [bug#56677] [PATCH 2/2] environment: Add '--emulate-fhs' John Kehayias via Guix-patches via
@ 2022-08-02 14:27 ` John Kehayias via Guix-patches via
0 siblings, 0 replies; 10+ messages in thread
From: John Kehayias via Guix-patches via @ 2022-08-02 14:27 UTC (permalink / raw)
To: 56677@debbugs.gnu.org
[-- Attachment #1: Type: text/plain, Size: 913 bytes --]
Here is a tiny update to this patch; I noticed after I sent it originally that there was a formatting typo in the guix.texi changes. Update attached.
Has anyone had a chance to try out the patch or take a look at it?
I do use this in some daily work, for a project that uses Python's poetry shell and the Playwright libraries (uses browser binaries it downloads to do web automation). Here is the command I use, where since I need to use browsers in the container just getting the development inputs for ungoogled-chromium is a nice shortcut. The share/expose is to get all graphical/hardware acceleration working.
~/path/to/guix-source/pre-inst-env guix shell -C -F -N poetry coreutils gcc:lib -D ungoogled-chromium --share=$HOME/temphome=$HOME --preserve='^DISPLAY$' --preserve='^XAUTHORITY$' --share=$XAUTHORITY -- "poetry shell"
All works great and is very handy for doing this on my Guix machine.
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0002-environment-Add-emulate-fhs.patch --]
[-- Type: text/x-patch; name=0002-environment-Add-emulate-fhs.patch, Size: 16470 bytes --]
From 6b5e3931bb83d589ff47263cc3bfd5eb236a3954 Mon Sep 17 00:00:00 2001
From: John Kehayias <john.kehayias@protonmail.com>
Date: Wed, 20 Jul 2022 23:46:45 -0400
Subject: [PATCH] environment: Add '--emulate-fhs'.
* guix/scripts/environment.scm (show-environment-options-help)
(%options): Add '--emulate-fhs'.
* guix/scripts/environment.scm (launch-environment/container): Add
'emulate-fhs?' key and implement it. Define and use FHS-MAPPINGS,
FHS-SYMLINKS, and LINK-CONTENTS to set up the container to follow the
Filesystem Hierarchy Standard (FHS) for /bin, /etc, and /usr. Generate
/etc/ld.so.cache in the container from /etc/ld.so.conf by running the script
/tmp/fhs.sh to launch the container.
(guix-environment*): Add glibc-for-fhs to the container packages when
'emulate-fhs?' key is in OPTS.
* doc/guix.texi (Invoking guix shell): Document '--emulate-fhs'.
(Invoking guix environment): Document '--emulate-fhs'.
---
doc/guix.texi | 33 ++++++++
guix/scripts/environment.scm | 157 ++++++++++++++++++++++++++++++-----
2 files changed, 168 insertions(+), 22 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 3c5864ec1a..03a65f26f4 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -106,6 +106,7 @@
Copyright @copyright{} 2022 Karl Hallsby@*
Copyright @copyright{} 2022 Justin Veilleux@*
Copyright @copyright{} 2022 Reily Siegel@*
+Copyright @copyright{} 2022 John Kehayias@*
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3 or
@@ -6155,6 +6156,22 @@ Invoking guix shell
guix shell --container --expose=$HOME=/exchange guile -- guile
@end example
+@item --emulate-fhs
+@item -F
+For containers, emulate a Filesystem Hierarchy Standard (FHS)
+configuration within the container, see
+@uref{https://refspecs.linuxfoundation.org/fhs.shtml, the official
+specification}. As Guix deviates from the FHS specification, this
+option sets up the container to more closely mimic that of other
+GNU/Linux distributions. This is useful for reproducing other
+development environments, testing, and using programs which expect the
+FHS specification to be followed. With this option, the container will
+include a version of @code{glibc} which will read
+@code{/etc/ld.so.cache} within the container for the shared library
+cache (contrary to @code{glibc} in regular Guix usage) and set up the
+expected FHS directories: @code{/bin}, @code{/etc}, @code{/lib}, and
+@code{/usr} from the container's profile.
+
@item --rebuild-cache
@cindex caching, of profiles
@cindex caching, in @command{guix shell}
@@ -6572,6 +6589,22 @@ Invoking guix environment
guix environment --container --expose=$HOME=/exchange --ad-hoc guile -- guile
@end example
+@item --emulate-fhs
+@item -F
+For containers, emulate a Filesystem Hierarchy Standard (FHS)
+configuration within the container, see
+@uref{https://refspecs.linuxfoundation.org/fhs.shtml, the official
+specification}. As Guix deviates from the FHS specification, this
+option sets up the container to more closely mimic that of other
+GNU/Linux distributions. This is useful for reproducing other
+development environments, testing, and using programs which expect the
+FHS specification to be followed. With this option, the container will
+include a version of @code{glibc} which will read
+@code{/etc/ld.so.cache} within the container for the shared library
+cache (contrary to @code{glibc} in regular Guix usage) and set up the
+expected FHS directories: @code{/bin}, @code{/etc}, @code{/lib}, and
+@code{/usr} from the container's profile.
+
@end table
@command{guix environment}
diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scm
index 3216235937..c80f5f28af 100644
--- a/guix/scripts/environment.scm
+++ b/guix/scripts/environment.scm
@@ -2,6 +2,7 @@
;;; Copyright © 2014, 2015, 2018 David Thompson <davet@gnu.org>
;;; Copyright © 2015-2022 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2018 Mike Gerwitz <mtg@gnu.org>
+;;; Copyright © 2022 John Kehayias <john.kehayias@protonmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -120,6 +121,9 @@ (define (show-environment-options-help)
--expose=SPEC for containers, expose read-only host file system
according to SPEC"))
(display (G_ "
+ -F, --emulate-fhs for containers, emulate the Filesystem Hierarchy
+ Standard (FHS)"))
+ (display (G_ "
-v, --verbosity=LEVEL use the given verbosity LEVEL"))
(display (G_ "
--bootstrap use bootstrap binaries to build the environment")))
@@ -256,6 +260,9 @@ (define %options
(alist-cons 'file-system-mapping
(specification->file-system-mapping arg #f)
result)))
+ (option '(#\F "emulate-fhs") #f #f
+ (lambda (opt name arg result)
+ (alist-cons 'emulate-fhs? #t result)))
(option '(#\r "root") #t #f
(lambda (opt name arg result)
(alist-cons 'gc-root arg result)))
@@ -608,16 +615,18 @@ (define* (launch-environment/fork command profile manifest
(define* (launch-environment/container #:key command bash user user-mappings
profile manifest link-profile? network?
- map-cwd? (white-list '()))
+ map-cwd? emulate-fhs? (white-list '()))
"Run COMMAND within a container that features the software in PROFILE.
-Environment variables are set according to the search paths of MANIFEST.
-The global shell is BASH, a file name for a GNU Bash binary in the
-store. When NETWORK?, access to the host system network is permitted.
-USER-MAPPINGS, a list of file system mappings, contains the user-specified
-host file systems to mount inside the container. If USER is not #f, each
-target of USER-MAPPINGS will be re-written relative to '/home/USER', and USER
-will be used for the passwd entry. LINK-PROFILE? creates a symbolic link from
-~/.guix-profile to the environment profile.
+Environment variables are set according to the search paths of MANIFEST. The
+global shell is BASH, a file name for a GNU Bash binary in the store. When
+NETWORK?, access to the host system network is permitted. USER-MAPPINGS, a
+list of file system mappings, contains the user-specified host file systems to
+mount inside the container. If USER is not #f, each target of USER-MAPPINGS
+will be re-written relative to '/home/USER', and USER will be used for the
+passwd entry. When EMULATE-FHS?, set up the container to follow the
+Filesystem Hierarchy Standard and provide a glibc that reads the cache from
+/etc/ld.so.cache. LINK-PROFILE? creates a symbolic link from ~/.guix-profile
+to the environment profile.
Preserve environment variables whose name matches the one of the regexps in
WHILE-LIST."
@@ -625,6 +634,40 @@ (define* (launch-environment/container #:key command bash user user-mappings
(and (file-exists? (file-system-mapping-source mapping))
(file-system-mapping->bind-mount mapping)))
+ ;; File system mappings for an FHS container, where the entire directory can
+ ;; be mapped. Others (bin and etc) will already have contents and need to
+ ;; use LINK-CONTENTS to symlink the directory contents.
+ (define fhs-mappings
+ (map (lambda (mapping)
+ (file-system-mapping
+ (source (string-append profile (car mapping)))
+ (target (cdr mapping))))
+ '(("/lib" . "/lib")
+ ("/include" . "/usr/include")
+ ("/sbin" . "/sbin")
+ ("/libexec" . "/usr/libexec")
+ ("/share" . "/usr/share"))))
+
+ ;; Additional symlinks for an FHS container.
+ (define fhs-symlinks
+ `(("/lib" . "/usr/lib")
+ ,(if (target-64bit?)
+ '("/lib" . "/lib64")
+ '("/lib" . "/lib32"))
+ ("/bin" . "/usr/bin")
+ ("/sbin" . "/usr/sbin")))
+
+ ;; A procedure to symlink the contents (at the top level) of a directory,
+ ;; excluding the directory itself and parent, along with any others provided
+ ;; in EXCLUDE.
+ (define* (link-contents dir #:key (exclude '()))
+ (for-each (lambda (file)
+ (symlink (string-append profile dir "/" file)
+ (string-append dir "/" file)))
+ (scandir (string-append profile dir)
+ (negate (cut member <>
+ (append exclude '("." ".." )))))))
+
(define (exit/status* status)
(exit/status (validate-exit-status profile command status)))
@@ -682,6 +725,11 @@ (define* (launch-environment/container #:key command bash user user-mappings
(filter-map optional-mapping->fs
%network-file-mappings)
'())
+ ;; Mappings for an FHS container.
+ (if emulate-fhs?
+ (filter-map optional-mapping->fs
+ fhs-mappings)
+ '())
(map file-system-mapping->bind-mount
mappings))))
(exit/status*
@@ -709,6 +757,54 @@ (define* (launch-environment/container #:key command bash user user-mappings
(mkdir-p home-dir)
(setenv "HOME" home-dir)
+ ;; Set up an FHS container.
+ (when emulate-fhs?
+ ;; The FHS container sets up the expected filesystem through
+ ;; MAPPINGS with FHS-MAPPINGS above, the symlinks through
+ ;; FHS-SYMLINKS, and linking the contents of profile/bin and
+ ;; profile/etc using LINK-CONTENTS, as these both have or will
+ ;; have contents for a non-FHS container so must be handled
+ ;; separately.
+ (mkdir-p "/usr")
+ (for-each (lambda (link)
+ (if (file-exists? (car link))
+ (symlink (car link) (cdr link))))
+ fhs-symlinks)
+ (link-contents "/bin" #:exclude '("sh"))
+ (mkdir-p "/etc")
+ (link-contents "/etc")
+
+ ;; Provide a frequently expected 'cc' symlink to gcc (in case it
+ ;; is in the container), though this could also be done by the
+ ;; user in the container, e.g. in $HOME/.local/bin and adding
+ ;; that to $PATH. Note: we do this in /bin since that already
+ ;; has the sh symlink and the other (optional) FHS bin
+ ;; directories will link to /bin.
+ (symlink (string-append profile "/bin/gcc") "/bin/cc")
+
+ ;; Guix's ldconfig doesn't seem to search in FHS default
+ ;; locations, so provide a minimal ld.so.conf.
+ (call-with-output-file "/etc/ld.so.conf"
+ (lambda (port)
+ (for-each (lambda (directory)
+ (display directory port)
+ (newline port))
+ ;; /lib/nss is needed as Guix's nss puts libraries
+ ;; there rather than in the lib directory.
+ '("/lib" "/lib/nss"))))
+
+ ;; Define an entry script to start the container: generate
+ ;; ld.so.cache, supplement $PATH (optional, but to better match
+ ;; FHS expectations), and include COMMAND.
+ (call-with-output-file "/tmp/fhs.sh"
+ (lambda (port)
+ (display "ldconfig -X" port)
+ (newline port)
+ (display "export PATH=/bin:/usr/bin:/sbin:/usr/sbin:$PATH" port)
+ (newline port)
+ (display (car command) port)
+ (newline port))))
+
;; If requested, link $GUIX_ENVIRONMENT to $HOME/.guix-profile;
;; this allows programs expecting that path to continue working as
;; expected within a container.
@@ -746,7 +842,10 @@ (define* (launch-environment/container #:key command bash user user-mappings
(primitive-exit/status
;; A container's environment is already purified, so no need to
;; request it be purified again.
- (launch-environment command
+ (launch-environment (if emulate-fhs?
+ ;; Use the FHS start script.
+ '("/bin/sh" "/tmp/fhs.sh")
+ command)
(if link-profile?
(string-append home-dir "/.guix-profile")
profile)
@@ -874,16 +973,17 @@ (define (guix-environment* opts)
"Run the 'guix environment' command on OPTS, an alist resulting for
command-line option processing with 'parse-command-line'."
(with-error-handling
- (let* ((pure? (assoc-ref opts 'pure))
- (container? (assoc-ref opts 'container?))
- (link-prof? (assoc-ref opts 'link-profile?))
- (network? (assoc-ref opts 'network?))
- (no-cwd? (assoc-ref opts 'no-cwd?))
- (user (assoc-ref opts 'user))
- (bootstrap? (assoc-ref opts 'bootstrap?))
- (system (assoc-ref opts 'system))
- (profile (assoc-ref opts 'profile))
- (command (or (assoc-ref opts 'exec)
+ (let* ((pure? (assoc-ref opts 'pure))
+ (container? (assoc-ref opts 'container?))
+ (link-prof? (assoc-ref opts 'link-profile?))
+ (network? (assoc-ref opts 'network?))
+ (no-cwd? (assoc-ref opts 'no-cwd?))
+ (emulate-fhs? (assoc-ref opts 'emulate-fhs?))
+ (user (assoc-ref opts 'user))
+ (bootstrap? (assoc-ref opts 'bootstrap?))
+ (system (assoc-ref opts 'system))
+ (profile (assoc-ref opts 'profile))
+ (command (or (assoc-ref opts 'exec)
;; Spawn a shell if the user didn't specify
;; anything in particular.
(if container?
@@ -922,12 +1022,24 @@ (define (guix-environment* opts)
(leave (G_ "'--user' cannot be used without '--container'~%")))
(when (and (not container?) no-cwd?)
(leave (G_ "--no-cwd cannot be used without --container~%")))
+ (when (and (not container?) emulate-fhs?)
+ (leave (G_ "'--emulate-fhs' cannot be used without '--container~'%")))
(with-store/maybe store
(with-status-verbosity (assoc-ref opts 'verbosity)
(define manifest-from-opts
- (options/resolve-packages store opts))
+ (options/resolve-packages store
+ ;; For an FHS-container, add the
+ ;; (hidden) package glibc-for-fhs which
+ ;; uses the global cache at
+ ;; /etc/ld.so.cache.
+ (if emulate-fhs?
+ (alist-cons 'expression
+ '(ad-hoc-package
+ "(@@ (gnu packages base) glibc-for-fhs)")
+ opts)
+ opts)))
(define manifest
(if profile
@@ -1001,7 +1113,8 @@ (define (guix-environment* opts)
#:white-list white-list
#:link-profile? link-prof?
#:network? network?
- #:map-cwd? (not no-cwd?))))
+ #:map-cwd? (not no-cwd?)
+ #:emulate-fhs? emulate-fhs?)))
(else
(return
--
2.37.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [bug#56677] [PATCH 0/2] environment: Add --emulate-fhs option.
2022-07-21 4:14 [bug#56677] [PATCH 0/2] environment: Add --emulate-fhs option John Kehayias via Guix-patches via
2022-07-21 4:19 ` [bug#56677] [PATCH 1/2] gnu: Add glibc-for-fhs John Kehayias via Guix-patches via
2022-07-21 4:19 ` [bug#56677] [PATCH 2/2] environment: Add '--emulate-fhs' John Kehayias via Guix-patches via
@ 2022-08-04 10:36 ` Ludovic Courtès
2022-08-17 21:43 ` John Kehayias via Guix-patches via
2 siblings, 1 reply; 10+ messages in thread
From: Ludovic Courtès @ 2022-08-04 10:36 UTC (permalink / raw)
To: John Kehayias; +Cc: 56677
Hello,
John Kehayias <john.kehayias@protonmail.com> skribis:
> As discussed on guix-devel here (please see for more detailed discussion and design aims): https://lists.gnu.org/r/guix-devel/2022-07/msg00161.html this is a patch to add an FHS (Filesystem Hierarchy Standard) emulation option for environments.
Wo0t!
> 1. On the mailing list there was discussion about the necessity or not of glibc-for-fhs (added in the first patch). I find this useful and a big piece of making this FHS option work, but open to discussion or if it should be a further option.
I would prefer to keep complexity as low as possible, and thus not have
this glibc variant.
Now, I don’t know for this use case how much it matters that libc honors
/etc/ld.so.cache. Intuitively, like I wrote on guix-devel, I’d think
ld.so.cache doesn’t matter, but you encountered counterexamples.
So I guess that if in practice presence of /etc/ld.so.cache *and* having
glibc honor it is necessary often enough, we can do that.
It seems that ‘glibc-for-fhs’ is merely added to the environment though,
and not actually used?
+ ;; For an FHS-container, add the
+ ;; (hidden) package glibc-for-fhs which
+ ;; uses the global cache at
+ ;; /etc/ld.so.cache.
+ (if emulate-fhs?
+ (alist-cons 'expression
+ '(ad-hoc-package
+ "(@@ (gnu packages base) glibc-for-fhs)")
+ opts)
+ opts)))
Or rather it’s only used when running ‘ldconfig’, right?
> 2. Right now I used a script written to the containers /tmp/fhs.sh to generate the ld cache, supplement $PATH (somewhat optional, but I found useful for less tinkering), and finally launch the given command or shell. I found that when not providing a command the prompt for /bin/sh is not the same as when not using --emulate-fhs. So I'm not sure if this is the correct way to launch the default /bin/sh if no command is given. Open to ideas of a better way to implement these actions for a container start up as well.
>
> 3. This is my first time touching a guix script and the documentation, so please do check the commit message and guix.texi.
>
> 4. I decided to link the second level FHS directories, like /usr/bin, as well as optional ones like /lib64 (or /lib32), to the top level /bin, /lib, and so on. These could just be bind mounted to profile/bin and so on as well, but again tried to mimic an FHS distribution like Arch where the files only live in one place. While perhaps making the code a little more involved, I hope this makes the container look tidier.
>
> I may be forgetting other elements in the implementation decisions I made, but I have been testing these patches along the way and have gotten good usage of them. Please test further too!
At first sight, I find it pretty cool! I would have two grievances:
1. Can we make the implementation more orthogonal and less entangled
in the already-long ‘launch-environment/container’?
Maybe that can be accomplished by moving all the code conditional
on ‘emulate-fhs?’ out of the way in a separate procedure, and
possibly by adding a generic hook in ‘launch-environment/container’
that would call said procedure.
2. Please add tests. You can probably augment
‘tests/guix-environment-container.sh’ for that. Let us know if
you’re not sure how to do that.
Thanks for all the work, and sorry for the delay: it seems to be summer
time for many of us. :-)
Ludo’.
^ permalink raw reply [flat|nested] 10+ messages in thread
* [bug#56677] [PATCH 0/2] environment: Add --emulate-fhs option.
2022-08-04 10:36 ` [bug#56677] [PATCH 0/2] environment: Add --emulate-fhs option Ludovic Courtès
@ 2022-08-17 21:43 ` John Kehayias via Guix-patches via
2022-09-08 20:58 ` Ludovic Courtès
2022-09-08 20:58 ` Ludovic Courtès
0 siblings, 2 replies; 10+ messages in thread
From: John Kehayias via Guix-patches via @ 2022-08-17 21:43 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 56677
Hello,
Took me a while to respond as well, summer meaning more time to hack on things and then also let them get away...
------- Original Message -------
On Thursday, August 4th, 2022 at 6:36 AM, Ludovic Courtès <ludo@gnu.org> wrote:
>
> Hello,
>
> John Kehayias john.kehayias@protonmail.com skribis:
>
> > As discussed on guix-devel here (please see for more detailed discussion and design aims): https://lists.gnu.org/r/guix-devel/2022-07/msg00161.html this is a patch to add an FHS (Filesystem Hierarchy Standard) emulation option for environments.
>
> Wo0t!
>
> > 1. On the mailing list there was discussion about the necessity or not of glibc-for-fhs (added in the first patch). I find this useful and a big piece of making this FHS option work, but open to discussion or if it should be a further option.
>
> I would prefer to keep complexity as low as possible, and thus not have
> this glibc variant.
>
> Now, I don’t know for this use case how much it matters that libc honors
> /etc/ld.so.cache. Intuitively, like I wrote on guix-devel, I’d think
> ld.so.cache doesn’t matter, but you encountered counterexamples.
>
> So I guess that if in practice presence of /etc/ld.so.cache and having
> glibc honor it is necessary often enough, we can do that.
>
Right, though as I said, happy to hear of alternatives or what other use cases come up. This seems rather robust to any "usual" assumptions though.
> It seems that ‘glibc-for-fhs’ is merely added to the environment though,
> and not actually used?
>
Well, it is added to the environment which here means the glibc-for-fhs lib directory ends up in the container's global /lib. This may be useful for anything expecting a more "typical" glibc to be found in the typical location. I can't say I know the particulars here, other than binaries and an example of other nested containers (used in non-free software, but the containers are bwrap and friends) expecting glibc to default to a global ld cache. Again, there may be other workarounds or ways to reduce this, but for now I followed the "emulate" part of the flag :)
> + ;; For an FHS-container, add the
> + ;; (hidden) package glibc-for-fhs which
> + ;; uses the global cache at
> + ;; /etc/ld.so.cache.
> + (if emulate-fhs?
> + (alist-cons 'expression
> + '(ad-hoc-package
> + "(@@ (gnu packages base) glibc-for-fhs)")
> + opts)
> + opts)))
>
> Or rather it’s only used when running ‘ldconfig’, right?
>
Yes, since that is the glibc in the container. Though actually generating a cache shouldn't matter, right? Guix's glibc will do that as well, as the only patch removed is the one that changed where glibc reads the ld cache from. So the cache could be generated with Guix's glibc, but then likely won't be read otherwise? Sorry, a bit out of what I know for all the details; I'm mostly at "this is what is often expected" and "this makes it work."
> > 2. Right now I used a script written to the containers /tmp/fhs.sh to generate the ld cache, supplement $PATH (somewhat optional, but I found useful for less tinkering), and finally launch the given command or shell. I found that when not providing a command the prompt for /bin/sh is not the same as when not using --emulate-fhs. So I'm not sure if this is the correct way to launch the default /bin/sh if no command is given. Open to ideas of a better way to implement these actions for a container start up as well.
> >
> > 3. This is my first time touching a guix script and the documentation, so please do check the commit message and guix.texi.
> >
> > 4. I decided to link the second level FHS directories, like /usr/bin, as well as optional ones like /lib64 (or /lib32), to the top level /bin, /lib, and so on. These could just be bind mounted to profile/bin and so on as well, but again tried to mimic an FHS distribution like Arch where the files only live in one place. While perhaps making the code a little more involved, I hope this makes the container look tidier.
> >
> > I may be forgetting other elements in the implementation decisions I made, but I have been testing these patches along the way and have gotten good usage of them. Please test further too!
>
> At first sight, I find it pretty cool! I would have two grievances:
>
Thanks! It is serving me in some day-to-day work nicely.
> 1. Can we make the implementation more orthogonal and less entangled
> in the already-long ‘launch-environment/container’?
>
> Maybe that can be accomplished by moving all the code conditional
> on ‘emulate-fhs?’ out of the way in a separate procedure, and
> possibly by adding a generic hook in ‘launch-environment/container’
> that would call said procedure.
>
Sure, this sounds like a good idea. I can certainly separate out the FHS setup to a separate function and call it. But I'm not sure what you mean by a "generic hook" here. Do you mean that launch-environment/container would have as an argument say a list of functions it would call?
> 2. Please add tests. You can probably augment
> ‘tests/guix-environment-container.sh’ for that. Let us know if
> you’re not sure how to do that.
>
Thanks, definitely forgot about that. In looking at that, I've just ran it with "./pre-inst-env sh tests/guix-environment-container.sh" and see that the exit code is 0. Is that the correct way to run these?
Secondly, I'm trying to think of what tests to add. I could of course run the same tests already, but with the --emulate-fhs option, to check that there are no regressions. Other than that, maybe checking that e.g. there's /etc/ld.so.cache, /lib, and so on?
> Thanks for all the work, and sorry for the delay: it seems to be summer
> time for many of us. :-)
>
> Ludo’.
No worries, summer is a good time to get away, or dig in :)
John
^ permalink raw reply [flat|nested] 10+ messages in thread
* [bug#56677] [PATCH 0/2] environment: Add --emulate-fhs option.
2022-08-17 21:43 ` John Kehayias via Guix-patches via
@ 2022-09-08 20:58 ` Ludovic Courtès
2022-09-08 20:58 ` Ludovic Courtès
1 sibling, 0 replies; 10+ messages in thread
From: Ludovic Courtès @ 2022-09-08 20:58 UTC (permalink / raw)
To: John Kehayias; +Cc: 56677
Howdy,
John Kehayias <john.kehayias@protonmail.com> skribis:
>> I would prefer to keep complexity as low as possible, and thus not have
>> this glibc variant.
>>
>> Now, I don’t know for this use case how much it matters that libc honors
>> /etc/ld.so.cache. Intuitively, like I wrote on guix-devel, I’d think
>> ld.so.cache doesn’t matter, but you encountered counterexamples.
>>
>> So I guess that if in practice presence of /etc/ld.so.cache and having
>> glibc honor it is necessary often enough, we can do that.
>>
>
> Right, though as I said, happy to hear of alternatives or what other use cases come up. This seems rather robust to any "usual" assumptions though.
>
>> It seems that ‘glibc-for-fhs’ is merely added to the environment though,
>> and not actually used?
>>
>
> Well, it is added to the environment which here means the glibc-for-fhs lib directory ends up in the container's global /lib. This may be useful for anything expecting a more "typical" glibc to be found in the typical location. I can't say I know the particulars here, other than binaries and an example of other nested containers (used in non-free software, but the containers are bwrap and friends) expecting glibc to default to a global ld cache. Again, there may be other workarounds or ways to reduce this, but for now I followed the "emulate" part of the flag :)
Oh I got it; that /lib/libc.so *is* used, but only by binaries that were
built on an FHS distro and that you’d bring in (that’s the whole point,
I guess). It’s not used by Guix packages.
>> 1. Can we make the implementation more orthogonal and less entangled
>> in the already-long ‘launch-environment/container’?
>>
>> Maybe that can be accomplished by moving all the code conditional
>> on ‘emulate-fhs?’ out of the way in a separate procedure, and
>> possibly by adding a generic hook in ‘launch-environment/container’
>> that would call said procedure.
>>
>
> Sure, this sounds like a good idea. I can certainly separate out the FHS setup to a separate function and call it. But I'm not sure what you mean by a "generic hook" here. Do you mean that launch-environment/container would have as an argument say a list of functions it would call?
Yes, or an argument with a single procedure to call at a specific
point. That would default to a no-op.
>> 2. Please add tests. You can probably augment
>> ‘tests/guix-environment-container.sh’ for that. Let us know if
>> you’re not sure how to do that.
>>
>
> Thanks, definitely forgot about that. In looking at that, I've just ran it with "./pre-inst-env sh tests/guix-environment-container.sh" and see that the exit code is 0. Is that the correct way to run these?
The correct way is:
make check TESTS=tests/guix-environment-container.sh
Compared to what you wrote, it uses ./test-env (which spawns a daemon
that uses the local store, not /gnu/sore) and sets a bunch of
environment variables.
See
<https://guix.gnu.org/manual/devel/en/html_node/Running-the-Test-Suite.html>.
> Secondly, I'm trying to think of what tests to add. I could of course run the same tests already, but with the --emulate-fhs option, to check that there are no regressions. Other than that, maybe checking that e.g. there's /etc/ld.so.cache, /lib, and so on?
Right, at least you’d want to check for these files/directories.
Note that since the test relies on ‘glibc-for-fhs’, it cannot be done
the “normal way” (that is, using the local store rather than /gnu/store)
because it would end up building the world.
The solution here is to use /gnu/store, if available, and to otherwise
skip the test (return 77). See ‘tests/guix-pack-relocatable.sh’ up to
line 40 on how to do that.
HTH!
Ludo’.
^ permalink raw reply [flat|nested] 10+ messages in thread
* [bug#56677] [PATCH 0/2] environment: Add --emulate-fhs option.
2022-08-17 21:43 ` John Kehayias via Guix-patches via
2022-09-08 20:58 ` Ludovic Courtès
@ 2022-09-08 20:58 ` Ludovic Courtès
2022-10-04 18:32 ` [bug#56677] [PATCH v2 1/2] " John Kehayias via Guix-patches via
1 sibling, 1 reply; 10+ messages in thread
From: Ludovic Courtès @ 2022-09-08 20:58 UTC (permalink / raw)
To: John Kehayias; +Cc: 56677
Howdy,
John Kehayias <john.kehayias@protonmail.com> skribis:
>> I would prefer to keep complexity as low as possible, and thus not have
>> this glibc variant.
>>
>> Now, I don’t know for this use case how much it matters that libc honors
>> /etc/ld.so.cache. Intuitively, like I wrote on guix-devel, I’d think
>> ld.so.cache doesn’t matter, but you encountered counterexamples.
>>
>> So I guess that if in practice presence of /etc/ld.so.cache and having
>> glibc honor it is necessary often enough, we can do that.
>>
>
> Right, though as I said, happy to hear of alternatives or what other use cases come up. This seems rather robust to any "usual" assumptions though.
>
>> It seems that ‘glibc-for-fhs’ is merely added to the environment though,
>> and not actually used?
>>
>
> Well, it is added to the environment which here means the glibc-for-fhs lib directory ends up in the container's global /lib. This may be useful for anything expecting a more "typical" glibc to be found in the typical location. I can't say I know the particulars here, other than binaries and an example of other nested containers (used in non-free software, but the containers are bwrap and friends) expecting glibc to default to a global ld cache. Again, there may be other workarounds or ways to reduce this, but for now I followed the "emulate" part of the flag :)
Oh I got it; that /lib/libc.so *is* used, but only by binaries that were
built on an FHS distro and that you’d bring in (that’s the whole point,
I guess). It’s not used by Guix packages.
>> 1. Can we make the implementation more orthogonal and less entangled
>> in the already-long ‘launch-environment/container’?
>>
>> Maybe that can be accomplished by moving all the code conditional
>> on ‘emulate-fhs?’ out of the way in a separate procedure, and
>> possibly by adding a generic hook in ‘launch-environment/container’
>> that would call said procedure.
>>
>
> Sure, this sounds like a good idea. I can certainly separate out the FHS setup to a separate function and call it. But I'm not sure what you mean by a "generic hook" here. Do you mean that launch-environment/container would have as an argument say a list of functions it would call?
Yes, or an argument with a single procedure to call at a specific
point. That would default to a no-op.
>> 2. Please add tests. You can probably augment
>> ‘tests/guix-environment-container.sh’ for that. Let us know if
>> you’re not sure how to do that.
>>
>
> Thanks, definitely forgot about that. In looking at that, I've just ran it with "./pre-inst-env sh tests/guix-environment-container.sh" and see that the exit code is 0. Is that the correct way to run these?
The correct way is:
make check TESTS=tests/guix-environment-container.sh
Compared to what you wrote, it uses ./test-env (which spawns a daemon
that uses the local store, not /gnu/sore) and sets a bunch of
environment variables.
See
<https://guix.gnu.org/manual/devel/en/html_node/Running-the-Test-Suite.html>.
> Secondly, I'm trying to think of what tests to add. I could of course run the same tests already, but with the --emulate-fhs option, to check that there are no regressions. Other than that, maybe checking that e.g. there's /etc/ld.so.cache, /lib, and so on?
Right, at least you’d want to check for these files/directories.
Note that since the test relies on ‘glibc-for-fhs’, it cannot be done
the “normal way” (that is, using the local store rather than /gnu/store)
because it would end up building the world.
The solution here is to use /gnu/store, if available, and to otherwise
skip the test (return 77). See ‘tests/guix-pack-relocatable.sh’ up to
line 40 on how to do that.
HTH!
Ludo’.
^ permalink raw reply [flat|nested] 10+ messages in thread
* [bug#56677] [PATCH v2 1/2] environment: Add --emulate-fhs option.
2022-09-08 20:58 ` Ludovic Courtès
@ 2022-10-04 18:32 ` John Kehayias via Guix-patches via
2022-10-13 7:37 ` [bug#56677] [PATCH 0/2] " Ludovic Courtès
0 siblings, 1 reply; 10+ messages in thread
From: John Kehayias via Guix-patches via @ 2022-10-04 18:32 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 56677
[-- Attachment #1: Type: text/plain, Size: 5181 bytes --]
Hi Ludo’!
Apologies for the long delay! I've done some more rewriting than expected, so let me outline some changes in the v2 patch (attached, while path 1/2 for glibc-for-fhs remains unchanged).
On Thu, Sep 08, 2022 at 10:58 PM, Ludovic Courtès wrote:
> Howdy,
>
> John Kehayias <john.kehayias@protonmail.com> skribis:
>
[snip]
>>> 1. Can we make the implementation more orthogonal and less entangled
>>> in the already-long ‘launch-environment/container’?
>>>
>>> Maybe that can be accomplished by moving all the code conditional
>>> on ‘emulate-fhs?’ out of the way in a separate procedure, and
>>> possibly by adding a generic hook in ‘launch-environment/container’
>>> that would call said procedure.
>>>
>>
>> Sure, this sounds like a good idea. I can certainly separate out the FHS setup to a
>> separate function and call it. But I'm not sure what you mean by a "generic hook" here.
>> Do you mean that launch-environment/container would have as an argument say a list of
>> functions it would call?
>
> Yes, or an argument with a single procedure to call at a specific
> point. That would default to a no-op.
>
I've done some rearranging to address this and along the way fixed the command passing to the container to work the same as without --emulate-fhs (previously I hacked together a startup script that didn't properly capture the command given by the user).
Unfortunately, since some of the needed directories are bind mounted and others can be linked after creating filesystems, not all of the FHS directory setup can be done in the same place. To be explicit, bind mounts for all filesystems are set up in the early mlet of launch-environment/container, before being able to call a procedure to do other setup needed here.
The end result:
launch-environment/container now takes a key 'setup-hook' which defaults to #f. For now this only handles a single list with the function name and arguments (could extend this to be more general, but didn't have any use cases for it off hand). For the FHS container this is set, by the entry point guix-environment*, to the new function setup-fhs which sets up symlinks and ld.so.conf. Handling arguments was necessary as setup-fhs needs the profile used in the container.
So then launch-environment also gets a new 'emulate-fhs?' option so it can run some additional setup before running the command given by the user. What was the (hacky) /tmp/fhs.sh is now a setenv and invoke, before the final execlp for the given command.
A little more complicated perhaps, but I think functionally separates the different steps (bind mounts, symlinks and file creation, and running in the container) at least.
>>> 2. Please add tests. You can probably augment
>>> ‘tests/guix-environment-container.sh’ for that. Let us know if
>>> you’re not sure how to do that.
>>>
>>
>> Thanks, definitely forgot about that. In looking at that, I've just ran it with
>> "./pre-inst-env sh tests/guix-environment-container.sh" and see that the exit code is 0.
>> Is that the correct way to run these?
>
> The correct way is:
>
> make check TESTS=tests/guix-environment-container.sh
>
> Compared to what you wrote, it uses ./test-env (which spawns a daemon
> that uses the local store, not /gnu/sore) and sets a bunch of
> environment variables.
>
> See
> <https://guix.gnu.org/manual/devel/en/html_node/Running-the-Test-Suite.html>.
>
Thanks, missed that somehow.
>> Secondly, I'm trying to think of what tests to add. I could of course run the same tests
>> already, but with the --emulate-fhs option, to check that there are no regressions.
>> Other than that, maybe checking that e.g. there's /etc/ld.so.cache, /lib, and so on?
>
> Right, at least you’d want to check for these files/directories.
>
> Note that since the test relies on ‘glibc-for-fhs’, it cannot be done
> the “normal way” (that is, using the local store rather than /gnu/store)
> because it would end up building the world.
>
> The solution here is to use /gnu/store, if available, and to otherwise
> skip the test (return 77). See ‘tests/guix-pack-relocatable.sh’ up to
> line 40 on how to do that.
>
> HTH!
>
Very helpful, thanks!
I added in two tests to the end of tests/guix-environment-container.sh. One checks for the FHS file structure in the container and the other tries to read the ld cache (using 'ldconfig -p').
If we wanted to run all the non-fhs tests with --emulate-fhs, then maybe we'd want to make it so the FHS specific tests live in a new file and guix-environment-container.sh can be called in a way to enable that option? (A quick guess would be to just set an alias so that guix environment is always called with --emulate-fhs, but not sure if that works in the test environment.)
Wasn't sure if all that is necessary so I just went with the FHS-specific tests for now. I checked that they pass for me.
The commit changelog has gotten a bit more complicated with these changes, hopefully I got everything in there.
Thanks for your help on this and I'll make sure any future revisions are more timely!
John
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0002-environment-Add-emulate-fhs-v2.patch --]
[-- Type: text/x-patch; name=0002-environment-Add-emulate-fhs-v2.patch, Size: 20483 bytes --]
From de7ae9b203a65e1ec1b1429ca4076f3f91b1ae33 Mon Sep 17 00:00:00 2001
From: John Kehayias <john.kehayias@protonmail.com>
Date: Wed, 20 Jul 2022 23:46:45 -0400
Subject: [PATCH] environment: Add '--emulate-fhs'.
* guix/scripts/environment.scm (show-environment-options-help, %options): Add
'--emulate-fhs'.
(setup-fhs): New procedure. Setup for the Filesystem Hierarchy Standard (FHS)
container. Defines and uses FHS-SYMLINKS and LINK-CONTENTS to create FHS
expected directories and creates /etc/ld.so.conf.
(launch-environment): Add 'emulate-fhs?' key and implement it to set $PATH and
generate /etc/ld.so.cache before calling COMMAND.
(launch-environment/container): Add 'emulate-fhs?' and 'setup-hook' keys and
implement them. Define and use FHS-MAPPINGS, to set up additional bind mounts
in the container to follow FHS expectations.
(guix-environment*): Add glibc-for-fhs to the container packages when
'emulate-fhs?' key is in OPTS.
* doc/guix.texi (Invoking guix shell): Document '--emulate-fhs'.
(Invoking guix environment): Document '--emulate-fhs'.
* tests/guix-environment-container.sh: Add tests for '--emulate-fhs'.
---
doc/guix.texi | 33 ++++++
guix/scripts/environment.scm | 167 ++++++++++++++++++++++++----
tests/guix-environment-container.sh | 34 ++++++
3 files changed, 211 insertions(+), 23 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 535c8cdfc3..99dea2e3c1 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -107,6 +107,7 @@
Copyright @copyright{} 2022 Justin Veilleux@*
Copyright @copyright{} 2022 Reily Siegel@*
Copyright @copyright{} 2022 Simon Streit@*
+Copyright @copyright{} 2022 John Kehayias@*
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3 or
@@ -6187,6 +6188,22 @@ Invoking guix shell
guix shell --container --expose=$HOME=/exchange guile -- guile
@end example
+@item --emulate-fhs
+@item -F
+For containers, emulate a Filesystem Hierarchy Standard (FHS)
+configuration within the container, see
+@uref{https://refspecs.linuxfoundation.org/fhs.shtml, the official
+specification}. As Guix deviates from the FHS specification, this
+option sets up the container to more closely mimic that of other
+GNU/Linux distributions. This is useful for reproducing other
+development environments, testing, and using programs which expect the
+FHS specification to be followed. With this option, the container will
+include a version of @code{glibc} which will read
+@code{/etc/ld.so.cache} within the container for the shared library
+cache (contrary to @code{glibc} in regular Guix usage) and set up the
+expected FHS directories: @code{/bin}, @code{/etc}, @code{/lib}, and
+@code{/usr} from the container's profile.
+
@item --rebuild-cache
@cindex caching, of profiles
@cindex caching, in @command{guix shell}
@@ -6604,6 +6621,22 @@ Invoking guix environment
guix environment --container --expose=$HOME=/exchange --ad-hoc guile -- guile
@end example
+@item --emulate-fhs
+@item -F
+For containers, emulate a Filesystem Hierarchy Standard (FHS)
+configuration within the container, see
+@uref{https://refspecs.linuxfoundation.org/fhs.shtml, the official
+specification}. As Guix deviates from the FHS specification, this
+option sets up the container to more closely mimic that of other
+GNU/Linux distributions. This is useful for reproducing other
+development environments, testing, and using programs which expect the
+FHS specification to be followed. With this option, the container will
+include a version of @code{glibc} which will read
+@code{/etc/ld.so.cache} within the container for the shared library
+cache (contrary to @code{glibc} in regular Guix usage) and set up the
+expected FHS directories: @code{/bin}, @code{/etc}, @code{/lib}, and
+@code{/usr} from the container's profile.
+
@end table
@command{guix environment}
diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scm
index 2493134470..f5b417457d 100644
--- a/guix/scripts/environment.scm
+++ b/guix/scripts/environment.scm
@@ -2,6 +2,7 @@
;;; Copyright © 2014, 2015, 2018 David Thompson <davet@gnu.org>
;;; Copyright © 2015-2022 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2018 Mike Gerwitz <mtg@gnu.org>
+;;; Copyright © 2022 John Kehayias <john.kehayias@protonmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -120,6 +121,9 @@ (define (show-environment-options-help)
--expose=SPEC for containers, expose read-only host file system
according to SPEC"))
(display (G_ "
+ -F, --emulate-fhs for containers, emulate the Filesystem Hierarchy
+ Standard (FHS)"))
+ (display (G_ "
-v, --verbosity=LEVEL use the given verbosity LEVEL"))
(display (G_ "
--bootstrap use bootstrap binaries to build the environment")))
@@ -256,6 +260,9 @@ (define %options
(alist-cons 'file-system-mapping
(specification->file-system-mapping arg #f)
result)))
+ (option '(#\F "emulate-fhs") #f #f
+ (lambda (opt name arg result)
+ (alist-cons 'emulate-fhs? #t result)))
(option '(#\r "root") #t #f
(lambda (opt name arg result)
(alist-cons 'gc-root arg result)))
@@ -375,6 +382,65 @@ (define (inputs->requisites inputs)
input->requisites inputs)))
(return (delete-duplicates (concatenate reqs)))))
+(define (setup-fhs profile)
+ "Setup the FHS container by creating and linking expected directories from
+PROFILE (other bind mounts are done in LAUNCH-ENVIRONMENT/CONTAINER),
+providing a symlink for CC if GCC is in the container PROFILE, and writing
+/etc/ld.so.conf."
+ ;; Additional symlinks for an FHS container.
+ (define fhs-symlinks
+ `(("/lib" . "/usr/lib")
+ ,(if (target-64bit?)
+ '("/lib" . "/lib64")
+ '("/lib" . "/lib32"))
+ ("/bin" . "/usr/bin")
+ ("/sbin" . "/usr/sbin")))
+
+ ;; A procedure to symlink the contents (at the top level) of a directory,
+ ;; excluding the directory itself and parent, along with any others provided
+ ;; in EXCLUDE.
+ (define* (link-contents dir #:key (exclude '()))
+ (for-each (lambda (file)
+ (symlink (string-append profile dir "/" file)
+ (string-append dir "/" file)))
+ (scandir (string-append profile dir)
+ (negate (cut member <>
+ (append exclude '("." ".." )))))))
+
+ ;; The FHS container sets up the expected filesystem through MAPPINGS with
+ ;; FHS-MAPPINGS (in LAUNCH-ENVIRONMENT/CONTAINER), the symlinks through
+ ;; FHS-SYMLINKS, and linking the contents of PROFILE/bin and PROFILE/etc
+ ;; using LINK-CONTENTS, as these both have or will have contents for a
+ ;; non-FHS container so must be handled separately.
+ (mkdir-p "/usr")
+ (for-each (lambda (link)
+ (if (file-exists? (car link))
+ (symlink (car link) (cdr link))))
+ fhs-symlinks)
+ (link-contents "/bin" #:exclude '("sh"))
+ (mkdir-p "/etc")
+ (link-contents "/etc")
+
+ ;; Provide a frequently expected 'cc' symlink to gcc (in case it is in
+ ;; PROFILE), though this could also be done by the user in the container,
+ ;; e.g. in $HOME/.local/bin and adding that to $PATH. Note: we do this in
+ ;; /bin since that already has the sh symlink and the other (optional) FHS
+ ;; bin directories will link to /bin.
+ (let ((gcc-path (string-append profile "/bin/gcc")))
+ (if (file-exists? gcc-path)
+ (symlink gcc-path "/bin/cc")))
+
+ ;; Guix's ldconfig doesn't search in FHS default locations, so provide a
+ ;; minimal ld.so.conf.
+ (call-with-output-file "/etc/ld.so.conf"
+ (lambda (port)
+ (for-each (lambda (directory)
+ (display directory port)
+ (newline port))
+ ;; /lib/nss is needed as Guix's nss puts libraries
+ ;; there rather than in the lib directory.
+ '("/lib" "/lib/nss")))))
+
(define (status->exit-code status)
"Compute the exit code made from STATUS, a value as returned by 'waitpid',
and suitable for 'exit'."
@@ -386,11 +452,13 @@ (define exit/status (compose exit status->exit-code))
(define primitive-exit/status (compose primitive-exit status->exit-code))
(define* (launch-environment command profile manifest
- #:key pure? (white-list '()))
+ #:key pure? (white-list '())
+ emulate-fhs?)
"Run COMMAND in a new environment containing INPUTS, using the native search
paths defined by the list PATHS. When PURE?, pre-existing environment
variables are cleared before setting the new ones, except those matching the
-regexps in WHITE-LIST."
+regexps in WHITE-LIST. When EMULATE-FHS?, first set up an FHS environment
+with $PATH and generate the LD cache."
;; Properly handle SIGINT, so pressing C-c in an interactive terminal
;; application works.
(sigaction SIGINT SIG_DFL)
@@ -406,6 +474,12 @@ (define* (launch-environment command profile manifest
((program . args)
(catch 'system-error
(lambda ()
+ (when emulate-fhs?
+ ;; When running in a container with EMULATE-FHS?, supplement $PATH
+ ;; (optional, but to better match FHS expectations), and generate
+ ;; /etc/ld.so.cache.
+ (setenv "PATH" "/bin:/usr/bin:/sbin:/usr/sbin:$PATH")
+ (invoke "ldconfig" "-X"))
(apply execlp program program args))
(lambda _
;; Report the error from here because the parent process cannot
@@ -604,15 +678,20 @@ (define* (launch-environment/fork command profile manifest
(define* (launch-environment/container #:key command bash user user-mappings
profile manifest link-profile? network?
- map-cwd? (white-list '()))
+ map-cwd? emulate-fhs? (setup-hook #f)
+ (white-list '()))
"Run COMMAND within a container that features the software in PROFILE.
-Environment variables are set according to the search paths of MANIFEST.
-The global shell is BASH, a file name for a GNU Bash binary in the
-store. When NETWORK?, access to the host system network is permitted.
-USER-MAPPINGS, a list of file system mappings, contains the user-specified
-host file systems to mount inside the container. If USER is not #f, each
-target of USER-MAPPINGS will be re-written relative to '/home/USER', and USER
-will be used for the passwd entry. LINK-PROFILE? creates a symbolic link from
+Environment variables are set according to the search paths of MANIFEST. The
+global shell is BASH, a file name for a GNU Bash binary in the store. When
+NETWORK?, access to the host system network is permitted. USER-MAPPINGS, a
+list of file system mappings, contains the user-specified host file systems to
+mount inside the container. If USER is not #f, each target of USER-MAPPINGS
+will be re-written relative to '/home/USER', and USER will be used for the
+passwd entry. When EMULATE-FHS?, set up the container to follow the
+Filesystem Hierarchy Standard and provide a glibc that reads the cache from
+/etc/ld.so.cache. SETUP-HOOK is an additional setup procedure to be called, as a
+list with the function name and arguments, currently only used with the
+EMULATE-FHS? option. LINK-PROFILE? creates a symbolic link from
~/.guix-profile to the environment profile.
Preserve environment variables whose name matches the one of the regexps in
@@ -621,6 +700,21 @@ (define* (launch-environment/container #:key command bash user user-mappings
(and (file-exists? (file-system-mapping-source mapping))
(file-system-mapping->bind-mount mapping)))
+ ;; File system mappings for an FHS container, where the entire directory can
+ ;; be mapped. Others (bin and etc) will already have contents and need to
+ ;; use LINK-CONTENTS (defined in SETUP-FHS) to symlink the directory
+ ;; contents.
+ (define fhs-mappings
+ (map (lambda (mapping)
+ (file-system-mapping
+ (source (string-append profile (car mapping)))
+ (target (cdr mapping))))
+ '(("/lib" . "/lib")
+ ("/include" . "/usr/include")
+ ("/sbin" . "/sbin")
+ ("/libexec" . "/usr/libexec")
+ ("/share" . "/usr/share"))))
+
(mlet %store-monad ((reqs (inputs->requisites
(list (direct-store-path bash) profile))))
(return
@@ -675,6 +769,11 @@ (define* (launch-environment/container #:key command bash user user-mappings
(filter-map optional-mapping->fs
%network-file-mappings)
'())
+ ;; Mappings for an FHS container.
+ (if emulate-fhs?
+ (filter-map optional-mapping->fs
+ fhs-mappings)
+ '())
(map file-system-mapping->bind-mount
mappings))))
(exit/status
@@ -702,6 +801,12 @@ (define* (launch-environment/container #:key command bash user user-mappings
(mkdir-p home-dir)
(setenv "HOME" home-dir)
+ ;; Call an additional setup procedure, if provided. Currently
+ ;; this is only used with the EMULATE-FHS? option, but could be
+ ;; expanded to a general list of functions to be called.
+ (if setup-hook
+ (apply (car setup-hook) (cdr setup-hook)))
+
;; If requested, link $GUIX_ENVIRONMENT to $HOME/.guix-profile;
;; this allows programs expecting that path to continue working as
;; expected within a container.
@@ -743,7 +848,8 @@ (define* (launch-environment/container #:key command bash user user-mappings
(if link-profile?
(string-append home-dir "/.guix-profile")
profile)
- manifest #:pure? #f)))
+ manifest #:pure? #f
+ #:emulate-fhs? emulate-fhs?)))
#:guest-uid uid
#:guest-gid gid
#:namespaces (if network?
@@ -867,16 +973,17 @@ (define (guix-environment* opts)
"Run the 'guix environment' command on OPTS, an alist resulting for
command-line option processing with 'parse-command-line'."
(with-error-handling
- (let* ((pure? (assoc-ref opts 'pure))
- (container? (assoc-ref opts 'container?))
- (link-prof? (assoc-ref opts 'link-profile?))
- (network? (assoc-ref opts 'network?))
- (no-cwd? (assoc-ref opts 'no-cwd?))
- (user (assoc-ref opts 'user))
- (bootstrap? (assoc-ref opts 'bootstrap?))
- (system (assoc-ref opts 'system))
- (profile (assoc-ref opts 'profile))
- (command (or (assoc-ref opts 'exec)
+ (let* ((pure? (assoc-ref opts 'pure))
+ (container? (assoc-ref opts 'container?))
+ (link-prof? (assoc-ref opts 'link-profile?))
+ (network? (assoc-ref opts 'network?))
+ (no-cwd? (assoc-ref opts 'no-cwd?))
+ (emulate-fhs? (assoc-ref opts 'emulate-fhs?))
+ (user (assoc-ref opts 'user))
+ (bootstrap? (assoc-ref opts 'bootstrap?))
+ (system (assoc-ref opts 'system))
+ (profile (assoc-ref opts 'profile))
+ (command (or (assoc-ref opts 'exec)
;; Spawn a shell if the user didn't specify
;; anything in particular.
(if container?
@@ -915,12 +1022,22 @@ (define (guix-environment* opts)
(leave (G_ "'--user' cannot be used without '--container'~%")))
(when (and (not container?) no-cwd?)
(leave (G_ "--no-cwd cannot be used without --container~%")))
+ (when (and (not container?) emulate-fhs?)
+ (leave (G_ "'--emulate-fhs' cannot be used without '--container~'%")))
(with-store/maybe store
(with-status-verbosity (assoc-ref opts 'verbosity)
(define manifest-from-opts
- (options/resolve-packages store opts))
+ (options/resolve-packages
+ store
+ ;; For an FHS-container, add the (hidden) package glibc-for-fhs
+ ;; which uses the global cache at /etc/ld.so.cache.
+ (if emulate-fhs?
+ (alist-cons 'expression
+ '(ad-hoc-package "(@@ (gnu packages base) glibc-for-fhs)")
+ opts)
+ opts)))
(define manifest
(if profile
@@ -994,7 +1111,11 @@ (define (guix-environment* opts)
#:white-list white-list
#:link-profile? link-prof?
#:network? network?
- #:map-cwd? (not no-cwd?))))
+ #:map-cwd? (not no-cwd?)
+ #:emulate-fhs? emulate-fhs?
+ #:setup-hook (if emulate-fhs?
+ (list setup-fhs profile)
+ #f))))
(else
(return
diff --git a/tests/guix-environment-container.sh b/tests/guix-environment-container.sh
index 2e238c501d..31e409420f 100644
--- a/tests/guix-environment-container.sh
+++ b/tests/guix-environment-container.sh
@@ -197,3 +197,37 @@ then false;
else
test $? -gt 127
fi
+
+# Test the Filesystem Hierarchy Standard (FHS) container option, --emulate-fhs (-F)
+
+# As this option requires a glibc package (glibc-for-fhs), try to run these
+# tests with the user's global store to make it easier to build or download a
+# substitute.
+storedir="`guile -c '(use-modules (guix config))(display %storedir)'`"
+localstatedir="`guile -c '(use-modules (guix config))(display %localstatedir)'`"
+NIX_STORE_DIR="$storedir"
+GUIX_DAEMON_SOCKET="$localstatedir/guix/daemon-socket/socket"
+export NIX_STORE_DIR GUIX_DAEMON_SOCKET
+
+if ! guile -c '(use-modules (guix)) (exit (false-if-exception (open-connection)))'
+then
+ exit 77
+fi
+
+# Test that the container has FHS specific files/directories. Note that /bin
+# exists in a non-FHS container as it will contain sh, a symlink to the bash
+# package, so we don't test for it.
+guix environment -C --emulate-fhs --ad-hoc --bootstrap guile-bootstrap \
+ -- guile -c '(exit (and (file-exists? "/etc/ld.so.cache")
+ (file-exists? "/lib")
+ (file-exists? "/sbin")
+ (file-exists? "/usr/bin")
+ (file-exists? "/usr/include")
+ (file-exists? "/usr/lib")
+ (file-exists? "/usr/libexec")
+ (file-exists? "/usr/sbin")
+ (file-exists? "/usr/share")))'
+
+# Test that the ld cache was generated and can be successfully read.
+guix environment -C --emulate-fhs --ad-hoc --bootstrap guile-bootstrap \
+ -- guile -c '(exit (execlp "ldconfig" "-p"))'
--
2.37.3
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [bug#56677] [PATCH 0/2] environment: Add --emulate-fhs option.
2022-10-04 18:32 ` [bug#56677] [PATCH v2 1/2] " John Kehayias via Guix-patches via
@ 2022-10-13 7:37 ` Ludovic Courtès
0 siblings, 0 replies; 10+ messages in thread
From: Ludovic Courtès @ 2022-10-13 7:37 UTC (permalink / raw)
To: John Kehayias; +Cc: 56677
[-- Attachment #1: Type: text/plain, Size: 1666 bytes --]
Hi John,
John Kehayias <john.kehayias@protonmail.com> skribis:
> From de7ae9b203a65e1ec1b1429ca4076f3f91b1ae33 Mon Sep 17 00:00:00 2001
> From: John Kehayias <john.kehayias@protonmail.com>
> Date: Wed, 20 Jul 2022 23:46:45 -0400
> Subject: [PATCH] environment: Add '--emulate-fhs'.
>
> * guix/scripts/environment.scm (show-environment-options-help, %options): Add
> '--emulate-fhs'.
> (setup-fhs): New procedure. Setup for the Filesystem Hierarchy Standard (FHS)
> container. Defines and uses FHS-SYMLINKS and LINK-CONTENTS to create FHS
> expected directories and creates /etc/ld.so.conf.
> (launch-environment): Add 'emulate-fhs?' key and implement it to set $PATH and
> generate /etc/ld.so.cache before calling COMMAND.
> (launch-environment/container): Add 'emulate-fhs?' and 'setup-hook' keys and
> implement them. Define and use FHS-MAPPINGS, to set up additional bind mounts
> in the container to follow FHS expectations.
> (guix-environment*): Add glibc-for-fhs to the container packages when
> 'emulate-fhs?' key is in OPTS.
> * doc/guix.texi (Invoking guix shell): Document '--emulate-fhs'.
> (Invoking guix environment): Document '--emulate-fhs'.
> * tests/guix-environment-container.sh: Add tests for '--emulate-fhs'.
This new version of the patch looks great to me!
I applied it and took the liberty to make the changes below; let me know
if anything’s amiss.
Pushed together with a news entry:
2c11e857af news: Add entry for 'guix shell --emulate-fhs'.
c7ba5f38b8 environment: Add '--emulate-fhs'.
3d1d29e440 gnu: Add glibc-for-fhs.
Thanks a lot for your work and for your patience!
Ludo’.
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 7051 bytes --]
diff --git a/doc/guix.texi b/doc/guix.texi
index 43dfdd64f2..8803353f24 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -6196,21 +6196,26 @@ directory:
guix shell --container --expose=$HOME=/exchange guile -- guile
@end example
+@cindex file system hierarchy standard (FHS)
+@cindex FHS (file system hierarchy standard)
@item --emulate-fhs
-@item -F
-For containers, emulate a Filesystem Hierarchy Standard (FHS)
-configuration within the container, see
-@uref{https://refspecs.linuxfoundation.org/fhs.shtml, the official
-specification}. As Guix deviates from the FHS specification, this
+@itemx -F
+When used with @option{--container}, emulate a
+@uref{https://refspecs.linuxfoundation.org/fhs.shtml, Filesystem
+Hierarchy Standard (FHS)} configuration within the container, providing
+@file{/bin}, @file{/lib}, and other directories and files specified by
+the FHS.
+
+As Guix deviates from the FHS specification, this
option sets up the container to more closely mimic that of other
GNU/Linux distributions. This is useful for reproducing other
development environments, testing, and using programs which expect the
FHS specification to be followed. With this option, the container will
-include a version of @code{glibc} which will read
-@code{/etc/ld.so.cache} within the container for the shared library
-cache (contrary to @code{glibc} in regular Guix usage) and set up the
-expected FHS directories: @code{/bin}, @code{/etc}, @code{/lib}, and
-@code{/usr} from the container's profile.
+include a version of glibc that will read
+@file{/etc/ld.so.cache} within the container for the shared library
+cache (contrary to glibc in regular Guix usage) and set up the
+expected FHS directories: @file{/bin}, @file{/etc}, @file{/lib}, and
+@file{/usr} from the container's profile.
@item --rebuild-cache
@cindex caching, of profiles
diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index b566057b41..2a9f2f34fc 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -933,15 +933,15 @@ (define (linker-script? file)
;; in FHS containers.
(define-public glibc-for-fhs
(hidden-package
- (package
- (inherit glibc)
+ (package/inherit glibc
(name "glibc-for-fhs")
(source (origin (inherit (package-source glibc))
;; Remove Guix's patch to read ld.so.cache from /gnu/store
;; directories, re-enabling the default /etc/ld.so.cache
;; behavior.
- (patches (delete (car (search-patches "glibc-dl-cache.patch"))
- (origin-patches (package-source glibc)))))))))
+ (patches
+ (delete (search-patch "glibc-dl-cache.patch")
+ (origin-patches (package-source glibc)))))))))
;; Below are old libc versions, which we use mostly to build locale data in
;; the old format (which the new libc cannot cope with.)
diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scm
index f5b417457d..cf99760859 100644
--- a/guix/scripts/environment.scm
+++ b/guix/scripts/environment.scm
@@ -475,10 +475,10 @@ (define* (launch-environment command profile manifest
(catch 'system-error
(lambda ()
(when emulate-fhs?
- ;; When running in a container with EMULATE-FHS?, supplement $PATH
+ ;; When running in a container with EMULATE-FHS?, override $PATH
;; (optional, but to better match FHS expectations), and generate
;; /etc/ld.so.cache.
- (setenv "PATH" "/bin:/usr/bin:/sbin:/usr/sbin:$PATH")
+ (setenv "PATH" "/bin:/usr/bin:/sbin:/usr/sbin")
(invoke "ldconfig" "-X"))
(apply execlp program program args))
(lambda _
@@ -687,12 +687,15 @@ (define* (launch-environment/container #:key command bash user user-mappings
list of file system mappings, contains the user-specified host file systems to
mount inside the container. If USER is not #f, each target of USER-MAPPINGS
will be re-written relative to '/home/USER', and USER will be used for the
-passwd entry. When EMULATE-FHS?, set up the container to follow the
-Filesystem Hierarchy Standard and provide a glibc that reads the cache from
-/etc/ld.so.cache. SETUP-HOOK is an additional setup procedure to be called, as a
-list with the function name and arguments, currently only used with the
-EMULATE-FHS? option. LINK-PROFILE? creates a symbolic link from
-~/.guix-profile to the environment profile.
+passwd entry.
+
+When EMULATE-FHS?, set up the container to follow the Filesystem Hierarchy
+Standard and provide a glibc that reads the cache from /etc/ld.so.cache.
+SETUP-HOOK is an additional setup procedure to be called, currently only used
+with the EMULATE-FHS? option.
+
+LINK-PROFILE? creates a symbolic link from ~/.guix-profile to the
+environment profile.
Preserve environment variables whose name matches the one of the regexps in
WHILE-LIST."
@@ -801,11 +804,9 @@ (define fhs-mappings
(mkdir-p home-dir)
(setenv "HOME" home-dir)
- ;; Call an additional setup procedure, if provided. Currently
- ;; this is only used with the EMULATE-FHS? option, but could be
- ;; expanded to a general list of functions to be called.
- (if setup-hook
- (apply (car setup-hook) (cdr setup-hook)))
+ ;; Call an additional setup procedure, if provided.
+ (when setup-hook
+ (setup-hook profile))
;; If requested, link $GUIX_ENVIRONMENT to $HOME/.guix-profile;
;; this allows programs expecting that path to continue working as
@@ -1113,9 +1114,9 @@ (define manifest
#:network? network?
#:map-cwd? (not no-cwd?)
#:emulate-fhs? emulate-fhs?
- #:setup-hook (if emulate-fhs?
- (list setup-fhs profile)
- #f))))
+ #:setup-hook
+ (and emulate-fhs?
+ setup-fhs))))
(else
(return
diff --git a/tests/guix-environment-container.sh b/tests/guix-environment-container.sh
index 31e409420f..898c57f41b 100644
--- a/tests/guix-environment-container.sh
+++ b/tests/guix-environment-container.sh
@@ -230,4 +230,4 @@ guix environment -C --emulate-fhs --ad-hoc --bootstrap guile-bootstrap \
# Test that the ld cache was generated and can be successfully read.
guix environment -C --emulate-fhs --ad-hoc --bootstrap guile-bootstrap \
- -- guile -c '(exit (execlp "ldconfig" "-p"))'
+ -- guile -c '(execlp "ldconfig" "ldconfig" "-p")'
^ permalink raw reply related [flat|nested] 10+ messages in thread
end of thread, other threads:[~2022-10-13 8:12 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-07-21 4:14 [bug#56677] [PATCH 0/2] environment: Add --emulate-fhs option John Kehayias via Guix-patches via
2022-07-21 4:19 ` [bug#56677] [PATCH 1/2] gnu: Add glibc-for-fhs John Kehayias via Guix-patches via
2022-07-21 4:19 ` [bug#56677] [PATCH 2/2] environment: Add '--emulate-fhs' John Kehayias via Guix-patches via
2022-08-02 14:27 ` [bug#56677] [PATCH 2/2 v2] " John Kehayias via Guix-patches via
2022-08-04 10:36 ` [bug#56677] [PATCH 0/2] environment: Add --emulate-fhs option Ludovic Courtès
2022-08-17 21:43 ` John Kehayias via Guix-patches via
2022-09-08 20:58 ` Ludovic Courtès
2022-09-08 20:58 ` Ludovic Courtès
2022-10-04 18:32 ` [bug#56677] [PATCH v2 1/2] " John Kehayias via Guix-patches via
2022-10-13 7:37 ` [bug#56677] [PATCH 0/2] " Ludovic Courtès
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).