unofficial mirror of guix-patches@gnu.org 
 help / color / mirror / code / Atom feed
* [bug#41189] [PATCH 0/3] Add Fakechroot engine for 'guix pack -RR'
@ 2020-05-11 17:05 Ludovic Courtès
  2020-05-11 17:11 ` [bug#41189] [PATCH 1/3] pack: Wrapper honors 'GUIX_EXECUTION_ENGINE' environment variable Ludovic Courtès
  2020-05-11 21:18 ` [bug#41189] [PATCH 0/3] Add Fakechroot engine for 'guix pack -RR' Carlos O'Donell
  0 siblings, 2 replies; 14+ messages in thread
From: Ludovic Courtès @ 2020-05-11 17:05 UTC (permalink / raw)
  To: 41189; +Cc: Ludovic Courtès

Hello Guix!

‘guix pack -RR’ is wonderful, as we know ;-), because it produces
binaries that work everywhere.

However, the overhead of PRoot is sometimes inappropriate, in
particular for those who want to run packed software on
high-performance computers, the very kind of machine that lacks
Guix and unprivileged user namespaces.

This patch series adds an optional “execution engine” to wrappers
that uses ld.so and fakechroot LD_PRELOAD trickery.  Since it’s
just LD_PRELOAD, there’s very little overhead, unlike PRoot.
On the flip side, it doesn’t work as well as PRoot, because it’s
“just” LD_PRELOAD.

For example, some of the ‘open’ calls made in libc are not
intercepted; on such call is in ‘__gconv_load_cache’, which makes
it fail, and in turn makes Guile fail to start in its first
‘scm_to_locale_string’ call.  Things that work well include Bash
and Python 3.  Let me know how well it works for your favorite
application!

The execution engine can now be chosen at run time by setting the
‘GUIX_EXECUTION_ENGINE’.

For the record, tools like udocker support a similar range of
execution engines: <https://github.com/indigo-dc/udocker/>.

Feedback welcome!

Thanks,
Ludo’.

Ludovic Courtès (3):
  pack: Wrapper honors 'GUIX_EXECUTION_ENGINE' environment variable.
  gnu: Add fakechroot.
  pack: Add relocation via ld.so and fakechroot.

 doc/guix.texi                             |  43 +++-
 gnu/packages/aux-files/run-in-namespace.c | 250 ++++++++++++++++++++--
 gnu/packages/linux.scm                    |  30 +++
 guix/scripts/pack.scm                     |  65 +++++-
 tests/guix-pack-relocatable.sh            |  23 ++
 5 files changed, 376 insertions(+), 35 deletions(-)

-- 
2.26.2





^ permalink raw reply	[flat|nested] 14+ messages in thread

* [bug#41189] [PATCH 1/3] pack: Wrapper honors 'GUIX_EXECUTION_ENGINE' environment variable.
  2020-05-11 17:05 [bug#41189] [PATCH 0/3] Add Fakechroot engine for 'guix pack -RR' Ludovic Courtès
@ 2020-05-11 17:11 ` Ludovic Courtès
  2020-05-11 17:11   ` [bug#41189] [PATCH 2/3] gnu: Add fakechroot Ludovic Courtès
  2020-05-11 17:11   ` [bug#41189] [PATCH 3/3] pack: Add relocation via ld.so and fakechroot Ludovic Courtès
  2020-05-11 21:18 ` [bug#41189] [PATCH 0/3] Add Fakechroot engine for 'guix pack -RR' Carlos O'Donell
  1 sibling, 2 replies; 14+ messages in thread
From: Ludovic Courtès @ 2020-05-11 17:11 UTC (permalink / raw)
  To: 41189; +Cc: Ludovic Courtès

* gnu/packages/aux-files/run-in-namespace.c (struct engine): New type.
(exec_default): New function.
(engines): New variable.
(execution_engine): New function.
(main): Use it instead of calling 'exec_in_user_namespace' and
'exec_with_proot' directly.
* tests/guix-pack-relocatable.sh: Add test with 'GUIX_EXECUTION_ENGINE'.
* doc/guix.texi (Invoking guix pack): Document 'GUIX_EXECUTION_ENGINE'.
---
 doc/guix.texi                             | 30 +++++++--
 gnu/packages/aux-files/run-in-namespace.c | 78 ++++++++++++++++++++---
 tests/guix-pack-relocatable.sh            | 17 +++++
 3 files changed, 110 insertions(+), 15 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 0cba0ee1ec..958ed9ceec 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -5185,9 +5185,9 @@ When this option is passed once, the resulting binaries require support for
 @dfn{user namespaces} in the kernel Linux; when passed
 @emph{twice}@footnote{Here's a trick to memorize it: @code{-RR}, which adds
 PRoot support, can be thought of as the abbreviation of ``Really
-Relocatable''.  Neat, isn't it?}, relocatable binaries fall to back to PRoot
-if user namespaces are unavailable, and essentially work anywhere---see below
-for the implications.
+Relocatable''.  Neat, isn't it?}, relocatable binaries fall to back to
+other techniques if user namespaces are unavailable, and essentially
+work anywhere---see below for the implications.
 
 For example, if you create a pack containing Bash with:
 
@@ -5219,14 +5219,32 @@ turn it off.
 
 To produce relocatable binaries that work even in the absence of user
 namespaces, pass @option{--relocatable} or @option{-R} @emph{twice}.  In that
-case, binaries will try user namespace support and fall back to PRoot if user
-namespaces are not supported.
+case, binaries will try user namespace support and fall back to another
+@dfn{execution engine} if user namespaces are not supported.  The
+following execution engines are supported:
 
-The @uref{https://proot-me.github.io/, PRoot} program provides the necessary
+@table @code
+@item default
+Try user namespaces and fall back to PRoot if user namespaces are not
+supported (see below).
+
+@item userns
+Run the program through user namespaces and abort if they are not
+supported.
+
+@item proot
+Run through PRoot.  The @uref{https://proot-me.github.io/, PRoot} program
+provides the necessary
 support for file system virtualization.  It achieves that by using the
 @code{ptrace} system call on the running program.  This approach has the
 advantage to work without requiring special kernel support, but it incurs
 run-time overhead every time a system call is made.
+@end table
+
+@vindex GUIX_EXECUTION_ENGINE
+When running a wrapped program, you can explicitly request one of the
+execution engines listed above by setting the
+@code{GUIX_EXECUTION_ENGINE} environment variable accordingly.
 @end quotation
 
 @cindex entry point, for Docker images
diff --git a/gnu/packages/aux-files/run-in-namespace.c b/gnu/packages/aux-files/run-in-namespace.c
index 23e7875173..6beac7fd53 100644
--- a/gnu/packages/aux-files/run-in-namespace.c
+++ b/gnu/packages/aux-files/run-in-namespace.c
@@ -336,6 +336,71 @@ exec_with_proot (const char *store, int argc, char *argv[])
 
 #endif
 
+\f
+/* Execution engines.  */
+
+struct engine
+{
+  const char *name;
+  void (* exec) (const char *, int, char **);
+};
+
+static void
+buffer_stderr (void)
+{
+  static char stderr_buffer[4096];
+  setvbuf (stderr, stderr_buffer, _IOFBF, sizeof stderr_buffer);
+}
+
+/* The default engine.  */
+static void
+exec_default (const char *store, int argc, char *argv[])
+{
+  /* Buffer stderr so that nothing's displayed if 'exec_in_user_namespace'
+     fails but 'exec_with_proot' works.  */
+  buffer_stderr ();
+
+  exec_in_user_namespace (store, argc, argv);
+#ifdef PROOT_PROGRAM
+  exec_with_proot (store, argc, argv);
+#endif
+}
+
+/* List of supported engines.  */
+static const struct engine engines[] =
+  {
+   { "default", exec_default },
+   { "userns", exec_in_user_namespace },
+#ifdef PROOT_PROGRAM
+   { "proot", exec_with_proot },
+#endif
+   { NULL, NULL }
+  };
+
+/* Return the "execution engine" to use.  */
+static const struct engine *
+execution_engine (void)
+{
+  const char *str = getenv ("GUIX_EXECUTION_ENGINE");
+
+  if (str == NULL)
+    str = "default";
+
+ try:
+  for (const struct engine *engine = engines;
+       engine->name != NULL;
+       engine++)
+    {
+      if (strcmp (engine->name, str) == 0)
+	return engine;
+    }
+
+  fprintf (stderr, "%s: unsupported Guix execution engine; ignoring\n",
+	   str);
+  str = "default";
+  goto try;
+}
+
 \f
 int
 main (int argc, char *argv[])
@@ -362,22 +427,17 @@ main (int argc, char *argv[])
   if (strcmp (store, "@STORE_DIRECTORY@") != 0
       && lstat ("@WRAPPED_PROGRAM@", &statbuf) != 0)
     {
-      /* Buffer stderr so that nothing's displayed if 'exec_in_user_namespace'
-	 fails but 'exec_with_proot' works.  */
-      static char stderr_buffer[4096];
-      setvbuf (stderr, stderr_buffer, _IOFBF, sizeof stderr_buffer);
+      const struct engine *engine = execution_engine ();
+      engine->exec (store, argc, argv);
 
-      exec_in_user_namespace (store, argc, argv);
-#ifdef PROOT_PROGRAM
-      exec_with_proot (store, argc, argv);
-#else
+      /* If we reach this point, that's because ENGINE failed to do the
+	 job.  */
       fprintf (stderr, "\
 This may be because \"user namespaces\" are not supported on this system.\n\
 Consequently, we cannot run '@WRAPPED_PROGRAM@',\n\
 unless you move it to the '@STORE_DIRECTORY@' directory.\n\
 \n\
 Please refer to the 'guix pack' documentation for more information.\n");
-#endif
       return EXIT_FAILURE;
     }
 
diff --git a/tests/guix-pack-relocatable.sh b/tests/guix-pack-relocatable.sh
index a3fd45623c..cb56815fed 100644
--- a/tests/guix-pack-relocatable.sh
+++ b/tests/guix-pack-relocatable.sh
@@ -84,6 +84,23 @@ fi
 grep 'GNU sed' "$test_directory/output"
 chmod -Rf +w "$test_directory"; rm -rf "$test_directory"/*
 
+case "`uname -m`" in
+    x86_64|i?86)
+	# Try '-RR' and PRoot.
+	tarball="`guix pack -RR -S /Bin=bin sed`"
+	tar tvf "$tarball" | grep /bin/proot
+	(cd "$test_directory"; tar xvf "$tarball")
+	GUIX_EXECUTION_ENGINE="proot"
+	export GUIX_EXECUTION_ENGINE
+	"$test_directory/Bin/sed" --version > "$test_directory/output"
+	grep 'GNU sed' "$test_directory/output"
+	chmod -Rf +w "$test_directory"; rm -rf "$test_directory"/*
+	;;
+    *)
+	echo "skipping PRoot test" >&2
+	;;
+esac
+
 # Ensure '-R' works with outputs other than "out".
 tarball="`guix pack -R -S /share=share groff:doc`"
 (cd "$test_directory"; tar xvf "$tarball")
-- 
2.26.2





^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [bug#41189] [PATCH 2/3] gnu: Add fakechroot.
  2020-05-11 17:11 ` [bug#41189] [PATCH 1/3] pack: Wrapper honors 'GUIX_EXECUTION_ENGINE' environment variable Ludovic Courtès
@ 2020-05-11 17:11   ` Ludovic Courtès
  2020-05-11 17:11   ` [bug#41189] [PATCH 3/3] pack: Add relocation via ld.so and fakechroot Ludovic Courtès
  1 sibling, 0 replies; 14+ messages in thread
From: Ludovic Courtès @ 2020-05-11 17:11 UTC (permalink / raw)
  To: 41189; +Cc: Ludovic Courtès

* gnu/packages/linux.scm (fakechroot): New variable.
---
 gnu/packages/linux.scm | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm
index 7cf7521e24..35526b3513 100644
--- a/gnu/packages/linux.scm
+++ b/gnu/packages/linux.scm
@@ -6793,6 +6793,36 @@ have to construct the archives directly, without using the archiver.")
     (home-page "http://freshmeat.sourceforge.net/projects/fakeroot")
     (license license:gpl3+)))
 
+(define-public fakechroot
+  (package
+    (name "fakechroot")
+    (version "2.20.1")
+    (source (origin
+              (method url-fetch)
+              (uri (string-append
+                    "https://github.com/dex4er/fakechroot/releases/download/"
+                    version "/fakechroot-" version ".tar.gz"))
+              (sha256
+               (base32
+                "1aijkd0b45wav25v01qhw8zxwa3pl0nnp9fabmmy1nlx7hr09gas"))))
+    (build-system gnu-build-system)
+    (arguments
+     ;; XXX: The tests heavily assume they run on an FHS system so for now
+     ;; skip them.
+     '(#:tests? #f
+       #:configure-flags '("--disable-static")))
+    (synopsis "Emulate @code{chroot} by overriding file system calls")
+    (description
+     "@command{fakechroot} runs a command in an environment were is additional
+possibility to use @code{chroot} command without root privileges.  This is
+useful for allowing users to create own chrooted environment with possibility
+to install another packages without need for root privileges.
+
+It works by providing @file{libfakechroot.so}, a shared library meant to be
+set as @code{LD_PRELOAD} to override the C library file system functions.")
+    (home-page "https://github.com/dex4er/fakechroot/")
+    (license license:lgpl2.1+)))
+
 (define-public inputattach
   (package
     (name "inputattach")
-- 
2.26.2





^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [bug#41189] [PATCH 3/3] pack: Add relocation via ld.so and fakechroot.
  2020-05-11 17:11 ` [bug#41189] [PATCH 1/3] pack: Wrapper honors 'GUIX_EXECUTION_ENGINE' environment variable Ludovic Courtès
  2020-05-11 17:11   ` [bug#41189] [PATCH 2/3] gnu: Add fakechroot Ludovic Courtès
@ 2020-05-11 17:11   ` Ludovic Courtès
  1 sibling, 0 replies; 14+ messages in thread
From: Ludovic Courtès @ 2020-05-11 17:11 UTC (permalink / raw)
  To: 41189; +Cc: Ludovic Courtès

From: Ludovic Courtès <ludovic.courtes@inria.fr>

* gnu/packages/aux-files/run-in-namespace.c (HAVE_EXEC_WITH_LOADER): New
macro.
(bind_mount): Rename to...
(mirror_directory): ... this.  Add 'firmlink' argument and use it
instead of calling mkdir/open/close/mount directly.
(bind_mount, make_symlink): New functions.
(exec_in_user_namespace): Adjust accordingly.
(exec_with_loader) [HAVE_EXEC_WITH_LOADER]: New function.
(exec_performance): New function.
(engines): Add them.
* guix/scripts/pack.scm (wrapped-package)[fakechroot-library]: New
procedures.
[build](elf-interpreter, elf-loader-compile-flags): New procedures.
(build-wrapper): Use them.
* tests/guix-pack-relocatable.sh: Test with
'GUIX_EXECUTION_ENGINE=fakechroot'.
* doc/guix.texi (Invoking guix pack): Document the 'performance' and
'fakechroot' engines.
---
 doc/guix.texi                             |  13 ++
 gnu/packages/aux-files/run-in-namespace.c | 174 ++++++++++++++++++++--
 guix/scripts/pack.scm                     |  65 +++++++-
 tests/guix-pack-relocatable.sh            |   6 +
 4 files changed, 237 insertions(+), 21 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 958ed9ceec..a70a058afb 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -5228,6 +5228,10 @@ following execution engines are supported:
 Try user namespaces and fall back to PRoot if user namespaces are not
 supported (see below).
 
+@item performance
+Try user namespaces and fall back to Fakechroot if user namespaces are
+not supported (see below).
+
 @item userns
 Run the program through user namespaces and abort if they are not
 supported.
@@ -5239,6 +5243,15 @@ support for file system virtualization.  It achieves that by using the
 @code{ptrace} system call on the running program.  This approach has the
 advantage to work without requiring special kernel support, but it incurs
 run-time overhead every time a system call is made.
+
+@item fakechroot
+Run through Fakechroot.  @uref{https://github.com/dex4er/fakechroot/,
+Fakechroot} virtualizes file system accesses by intercepting calls to C
+library functions such as @code{open}, @code{stat}, @code{exec}, and so
+on.  Unlike PRoot, it incurs very little overhead.  However, it does not
+always work: for example, some file system accesses made from within the
+C library are not intercepted, and file system accesses made @i{via}
+direct syscalls are not intercepted either, leading to erratic behavior.
 @end table
 
 @vindex GUIX_EXECUTION_ENGINE
diff --git a/gnu/packages/aux-files/run-in-namespace.c b/gnu/packages/aux-files/run-in-namespace.c
index 6beac7fd53..ed72a169f2 100644
--- a/gnu/packages/aux-files/run-in-namespace.c
+++ b/gnu/packages/aux-files/run-in-namespace.c
@@ -42,6 +42,11 @@
 #include <dirent.h>
 #include <sys/syscall.h>
 
+/* Whether we're building the ld.so/libfakechroot wrapper.  */
+#define HAVE_EXEC_WITH_LOADER					\
+  (defined PROGRAM_INTERPRETER) && (defined PROGRAM_RUNPATH)
+
+
 /* Like 'malloc', but abort if 'malloc' returns NULL.  */
 static void *
 xmalloc (size_t size)
@@ -113,9 +118,42 @@ rm_rf (const char *directory)
     assert_perror (errno);
 }
 
-/* Bind mount all the top-level entries in SOURCE to TARGET.  */
+/* Make TARGET a bind-mount of SOURCE.  Take into account ENTRY's type, which
+   corresponds to SOURCE.  */
+static int
+bind_mount (const char *source, const struct dirent *entry,
+	    const char *target)
+{
+  if (entry->d_type == DT_DIR)
+    {
+      int err = mkdir (target, 0700);
+      if (err != 0)
+	return err;
+    }
+  else
+    close (open (target, O_WRONLY | O_CREAT));
+
+  return mount (source, target, "none",
+		MS_BIND | MS_REC | MS_RDONLY, NULL);
+}
+
+#if HAVE_EXEC_WITH_LOADER
+
+/* Make TARGET a symlink to SOURCE.  */
+static int
+make_symlink (const char *source, const struct dirent *entry,
+	      const char *target)
+{
+  return symlink (source, target);
+}
+
+#endif
+
+/* Mirror with FIRMLINK all the top-level entries in SOURCE to TARGET.  */
 static void
-bind_mount (const char *source, const char *target)
+mirror_directory (const char *source, const char *target,
+		  int (* firmlink) (const char *, const struct dirent *,
+				    const char *))
 {
   DIR *stream = opendir (source);
 
@@ -150,17 +188,7 @@ bind_mount (const char *source, const char *target)
       else
 	{
 	  /* Create the mount point.  */
-	  if (entry->d_type == DT_DIR)
-	    {
-	      int err = mkdir (new_entry, 0700);
-	      if (err != 0)
-		assert_perror (errno);
-	    }
-	  else
-	    close (open (new_entry, O_WRONLY | O_CREAT));
-
-	  int err = mount (abs_source, new_entry, "none",
-			   MS_BIND | MS_REC | MS_RDONLY, NULL);
+	  int err = firmlink (abs_source, entry, new_entry);
 
 	  /* It used to be that only directories could be bind-mounted.  Thus,
 	     keep going if we fail to bind-mount a non-directory entry.
@@ -244,7 +272,7 @@ exec_in_user_namespace (const char *store, int argc, char *argv[])
       /* Note: Due to <https://bugzilla.kernel.org/show_bug.cgi?id=183461>
 	 we cannot make NEW_ROOT a tmpfs (which would have saved the need
 	 for 'rm_rf'.)  */
-      bind_mount ("/", new_root);
+      mirror_directory ("/", new_root, bind_mount);
       mkdir_p (new_store);
       err = mount (store, new_store, "none", MS_BIND | MS_REC | MS_RDONLY,
 		   NULL);
@@ -336,6 +364,106 @@ exec_with_proot (const char *store, int argc, char *argv[])
 
 #endif
 
+\f
+#if HAVE_EXEC_WITH_LOADER
+
+static void
+exec_with_loader (const char *store, int argc, char *argv[])
+{
+  static const char *runpath[] = PROGRAM_RUNPATH;
+  char *library_path;
+  size_t size = 0;
+
+  for (size_t i = 0; runpath[i] != NULL; i++)
+    size += strlen (store) + strlen (runpath[i]) + 1;  /* upper bound */
+
+  library_path = xmalloc (size + 1);
+  library_path[0] = '\0';
+
+  for (size_t i = 0; runpath[i] != NULL; i++)
+    {
+      if (strncmp (runpath[i], "@STORE_DIRECTORY@",
+		   sizeof "@STORE_DIRECTORY@" - 1) == 0)
+	{
+	  strcat (library_path, store);
+	  strcat (library_path, runpath[i] + sizeof "@STORE_DIRECTORY@");
+	}
+      else
+	strcat (library_path, runpath[i]);	  /* possibly $ORIGIN */
+
+      strcat (library_path, ":");
+    }
+
+  library_path[strlen (library_path) - 1] = '\0'; /* Remove trailing colon.  */
+
+  char *loader = concat (store,
+			 PROGRAM_INTERPRETER + sizeof "@STORE_DIRECTORY@");
+  size_t loader_specific_argc = 6;
+  size_t loader_argc = argc + loader_specific_argc;
+  char *loader_argv[loader_argc + 1];
+  loader_argv[0] = argv[0];
+  loader_argv[1] = "--library-path";
+  loader_argv[2] = library_path;
+  loader_argv[3] = "--preload";
+  loader_argv[4] = concat (store,
+			   FAKECHROOT_LIBRARY + sizeof "@STORE_DIRECTORY@");
+  loader_argv[5] = concat (store,
+			   "@WRAPPED_PROGRAM@" + sizeof "@STORE_DIRECTORY@");
+
+  for (size_t i = 0; i < argc; i++)
+    loader_argv[i + loader_specific_argc] = argv[i + 1];
+
+  loader_argv[loader_argc] = NULL;
+
+  /* Set up the root directory.  */
+  int err;
+  char *new_root = mkdtemp (strdup ("/tmp/guix-exec-XXXXXX"));
+  mirror_directory ("/", new_root, make_symlink);
+
+  char *new_store = concat (new_root, "@STORE_DIRECTORY@");
+  char *new_store_parent = dirname (strdup (new_store));
+  mkdir_p (new_store_parent);
+  symlink (store, new_store);
+
+  setenv ("FAKECHROOT_BASE", new_root, 1);
+
+  pid_t child = fork ();
+  switch (child)
+    {
+    case 0:
+      err = execv (loader, loader_argv);
+      if (err < 0)
+	assert_perror (errno);
+      exit (EXIT_FAILURE);
+      break;
+
+    case -1:
+      assert_perror (errno);
+      exit (EXIT_FAILURE);
+      break;
+
+    default:
+      {
+  	int status;
+	waitpid (child, &status, 0);
+	chdir ("/");			  /* avoid EBUSY */
+	rm_rf (new_root);
+	free (new_root);
+
+	close (2);			/* flushing stderr should be silent */
+
+	if (WIFEXITED (status))
+	  exit (WEXITSTATUS (status));
+	else
+	  /* Abnormal termination cannot really be reproduced, so exit
+	     with 255.  */
+	  exit (255);
+      }
+    }
+}
+
+#endif
+
 \f
 /* Execution engines.  */
 
@@ -352,7 +480,7 @@ buffer_stderr (void)
   setvbuf (stderr, stderr_buffer, _IOFBF, sizeof stderr_buffer);
 }
 
-/* The default engine.  */
+/* The default engine: choose a robust method.  */
 static void
 exec_default (const char *store, int argc, char *argv[])
 {
@@ -366,13 +494,29 @@ exec_default (const char *store, int argc, char *argv[])
 #endif
 }
 
+/* The "performance" engine: choose performance over robustness.  */
+static void
+exec_performance (const char *store, int argc, char *argv[])
+{
+  buffer_stderr ();
+
+  exec_in_user_namespace (store, argc, argv);
+#if HAVE_EXEC_WITH_LOADER
+  exec_with_loader (store, argc, argv);
+#endif
+}
+
 /* List of supported engines.  */
 static const struct engine engines[] =
   {
    { "default", exec_default },
+   { "performance", exec_performance },
    { "userns", exec_in_user_namespace },
 #ifdef PROOT_PROGRAM
    { "proot", exec_with_proot },
+#endif
+#if HAVE_EXEC_WITH_LOADER
+   { "fakechroot", exec_with_loader },
 #endif
    { NULL, NULL }
   };
diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm
index 580f696b41..3b72496a34 100644
--- a/guix/scripts/pack.scm
+++ b/guix/scripts/pack.scm
@@ -684,15 +684,26 @@ last resort for relocation."
   (define (proot)
     (specification->package "proot-static"))
 
+  (define (fakechroot-library)
+    (file-append (specification->package "fakechroot")
+                 "/lib/fakechroot/libfakechroot.so"))
+
   (define build
     (with-imported-modules (source-module-closure
                             '((guix build utils)
-                              (guix build union)))
+                              (guix build union)
+                              (guix build gremlin)
+                              (guix elf)))
       #~(begin
           (use-modules (guix build utils)
                        ((guix build union) #:select (relative-file-name))
+                       (guix build gremlin)
+                       (guix elf)
+                       (ice-9 binary-ports)
                        (ice-9 ftw)
-                       (ice-9 match))
+                       (ice-9 match)
+                       (srfi srfi-1)
+                       (rnrs bytevectors))
 
           (define input
             ;; The OUTPUT* output of PACKAGE.
@@ -711,6 +722,47 @@ last resort for relocation."
                 (#f    base)
                 (index (string-drop base index)))))
 
+          (define (elf-interpreter elf)
+            ;; Return the interpreter of ELF as a string, or #f if ELF has no
+            ;; interpreter segment.
+            (match (find (lambda (segment)
+                           (= (elf-segment-type segment) PT_INTERP))
+                         (elf-segments elf))
+              (#f #f)                             ;maybe a .so
+              (segment
+               (let ((bv (make-bytevector (- (elf-segment-memsz segment) 1))))
+                 (bytevector-copy! (elf-bytes elf)
+                                   (elf-segment-offset segment)
+                                   bv 0 (bytevector-length bv))
+                 (utf8->string bv)))))
+
+          (define (elf-loader-compile-flags program)
+            ;; Return the cpp flags defining macros for the ld.so/fakechroot
+            ;; wrapper of PROGRAM.
+
+            ;; TODO: Handle scripts by wrapping their interpreter.
+            (if (elf-file? program)
+                (let* ((bv  (call-with-input-file program get-bytevector-all))
+                       (elf (parse-elf bv)))
+                  (match (elf-dynamic-info elf)
+                    (#f '())
+                    (dyninfo
+                     (let ((runpath (elf-dynamic-info-runpath dyninfo))
+                           (interp  (elf-interpreter elf)))
+                       (if interp
+                           (list (string-append "-DPROGRAM_INTERPRETER=\""
+                                                interp "\"")
+                                 (string-append "-DPROGRAM_RUNPATH={ "
+                                                (string-join
+                                                 (map object->string
+                                                      runpath)
+                                                 ", ")
+                                                ", NULL }")
+                                 (string-append "-DFAKECHROOT_LIBRARY=\""
+                                                #$(fakechroot-library) "\""))
+                           '())))))
+                '()))
+
           (define (build-wrapper program)
             ;; Build a user-namespace wrapper for PROGRAM.
             (format #t "building wrapper for '~a'...~%" program)
@@ -730,10 +782,11 @@ last resort for relocation."
               (mkdir-p (dirname result))
               (apply invoke #$compiler "-std=gnu99" "-static" "-Os" "-g0" "-Wall"
                      "run.c" "-o" result
-                     (if proot
-                         (list (string-append "-DPROOT_PROGRAM=\""
-                                              proot "\""))
-                         '()))
+                     (append (if proot
+                                 (list (string-append "-DPROOT_PROGRAM=\""
+                                                      proot "\""))
+                                 '())
+                             (elf-loader-compile-flags program)))
               (delete-file "run.c")))
 
           (setvbuf (current-output-port) 'line)
diff --git a/tests/guix-pack-relocatable.sh b/tests/guix-pack-relocatable.sh
index cb56815fed..358cac5b26 100644
--- a/tests/guix-pack-relocatable.sh
+++ b/tests/guix-pack-relocatable.sh
@@ -94,6 +94,12 @@ case "`uname -m`" in
 	export GUIX_EXECUTION_ENGINE
 	"$test_directory/Bin/sed" --version > "$test_directory/output"
 	grep 'GNU sed' "$test_directory/output"
+
+	# Now with fakechroot.
+	GUIX_EXECUTION_ENGINE="fakechroot"
+	"$test_directory/Bin/sed" --version > "$test_directory/output"
+	grep 'GNU sed' "$test_directory/output"
+
 	chmod -Rf +w "$test_directory"; rm -rf "$test_directory"/*
 	;;
     *)
-- 
2.26.2





^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [bug#41189] [PATCH 0/3] Add Fakechroot engine for 'guix pack -RR'
  2020-05-11 17:05 [bug#41189] [PATCH 0/3] Add Fakechroot engine for 'guix pack -RR' Ludovic Courtès
  2020-05-11 17:11 ` [bug#41189] [PATCH 1/3] pack: Wrapper honors 'GUIX_EXECUTION_ENGINE' environment variable Ludovic Courtès
@ 2020-05-11 21:18 ` Carlos O'Donell
  2020-05-12 10:03   ` Ludovic Courtès
  1 sibling, 1 reply; 14+ messages in thread
From: Carlos O'Donell @ 2020-05-11 21:18 UTC (permalink / raw)
  To: 41189

"For example, some of the ‘open’ calls made in libc are notintercepted; 
on such call is in ‘__gconv_load_cache’, which makesit fail, and in 
turn makes Guile fail to start in its first‘scm_to_locale_string’ call."
-- Ludovic Courtès wrote on Mon May 11 19:05:54+0200 2020

There are two issues at hand:
* Standard namespace issues (conformance)
* PLT avoidance issues (performance)

See:
https://sourceware.org/glibc/wiki/Style_and_Conventions#Double-underscore_names_for_public_API_functions

It is an internal implementation detail that open(2) is being called by
the library, and as such glibc bypasses the ELF interposable symbol
open, and instead calls open directly without this being visible to the
application.

There are many such cases where we bypass the ELF interposable symbol to
provide standard namespace cleanliness, performance, and so provide consistent
behaviour.

Yes, in your case this means you cannot override the behaviour of the
interface without using some kind of bind mount, or mount namespace
(to provide an alternate view of the filesystem).

We would have to argue upstream that some minimal subset of the filesystem
access should be interposable via open/close/read/write, but that's going
to get difficult quickly and have significant performance problems.

It would be simpler, IMO, to set LOCPATH and GCONV_PATH appropriately and
alter the runtime behaviour that way. If that doesn't work, perhaps because
of setuid, then we can discuss further.

-- 
Cheers,
Carlos.





^ permalink raw reply	[flat|nested] 14+ messages in thread

* [bug#41189] [PATCH 0/3] Add Fakechroot engine for 'guix pack -RR'
  2020-05-11 21:18 ` [bug#41189] [PATCH 0/3] Add Fakechroot engine for 'guix pack -RR' Carlos O'Donell
@ 2020-05-12 10:03   ` Ludovic Courtès
  2020-05-12 12:09     ` Carlos O'Donell
  0 siblings, 1 reply; 14+ messages in thread
From: Ludovic Courtès @ 2020-05-12 10:03 UTC (permalink / raw)
  To: Carlos O'Donell; +Cc: 41189

[-- Attachment #1: Type: text/plain, Size: 2316 bytes --]

Hi Carlos,

Carlos O'Donell <carlos@redhat.com> skribis:

> There are two issues at hand:
> * Standard namespace issues (conformance)
> * PLT avoidance issues (performance)
>
> See:
> https://sourceware.org/glibc/wiki/Style_and_Conventions#Double-underscore_names_for_public_API_functions
>
> It is an internal implementation detail that open(2) is being called by
> the library, and as such glibc bypasses the ELF interposable symbol
> open, and instead calls open directly without this being visible to the
> application.
>
> There are many such cases where we bypass the ELF interposable symbol to
> provide standard namespace cleanliness, performance, and so provide consistent
> behaviour.

It makes sense to me, thanks for explaining.

> Yes, in your case this means you cannot override the behaviour of the
> interface without using some kind of bind mount, or mount namespace
> (to provide an alternate view of the filesystem).

Agreed, unprivileged user namespaces with bind mounts are the preferred
solution; the LD_PRELOAD hack discussed here is for when they’re
unavailable and PRoot is too slow.

> We would have to argue upstream that some minimal subset of the filesystem
> access should be interposable via open/close/read/write, but that's going
> to get difficult quickly and have significant performance problems.

Yes, understood.  (I wasn’t going to suggest it.  :-))

> It would be simpler, IMO, to set LOCPATH and GCONV_PATH appropriately and
> alter the runtime behaviour that way. If that doesn't work, perhaps because
> of setuid, then we can discuss further.

Yes, setting ‘GCONV_PATH’ in particular seems like something the wrapper
could automatically do.  The attached patch does that and now Guile runs
fine with the ld.so/fakechroot “engine”.

One thing that won’t work is dlopen because our ‘--library-path’
argument is computed statically based on the RUNPATH of the wrapped
program.  So for instance if you try to load guile-readline.so from
Guile, it eventually fails because libreadline.so isn’t found
(libreadline.so is in the RUNPATH of guile-readline.so, but the loader
uses non-interposable calls here as well.)  Probably no simple solution
to that one.

Thanks for your feedback, Carlos!

Ludo’.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 2876 bytes --]

diff --git a/gnu/packages/aux-files/run-in-namespace.c b/gnu/packages/aux-files/run-in-namespace.c
index ed72a169f2..c56c35a510 100644
--- a/gnu/packages/aux-files/run-in-namespace.c
+++ b/gnu/packages/aux-files/run-in-namespace.c
@@ -425,6 +427,15 @@ exec_with_loader (const char *store, int argc, char *argv[])
   mkdir_p (new_store_parent);
   symlink (store, new_store);
 
+#ifdef GCONV_DIRECTORY
+  /* Tell libc where to find its gconv modules.  This is necessary because
+     gconv uses non-interposable 'open' calls.  */
+  char *gconv_path = concat (store,
+			     GCONV_DIRECTORY + sizeof "@STORE_DIRECTORY@");
+  setenv ("GCONV_PATH", gconv_path, 1);
+  free (gconv_path);
+#endif
+
   setenv ("FAKECHROOT_BASE", new_root, 1);
 
   pid_t child = fork ();
diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm
index 2d856066b2..2b37bf5027 100644
--- a/guix/scripts/pack.scm
+++ b/guix/scripts/pack.scm
@@ -739,6 +739,12 @@ last resort for relocation."
                                    bv 0 (bytevector-length bv))
                  (utf8->string bv)))))
 
+          (define (gconv-directory directory)
+            ;; Return DIRECTORY/gconv if it exists as a directory.
+            (let ((gconv (string-append directory "/gconv")))
+              (and (directory-exists? gconv)
+                   gconv)))
+
           (define (elf-loader-compile-flags program)
             ;; Return the cpp flags defining macros for the ld.so/fakechroot
             ;; wrapper of PROGRAM.
@@ -750,8 +756,9 @@ last resort for relocation."
                   (match (elf-dynamic-info elf)
                     (#f '())
                     (dyninfo
-                     (let ((runpath (elf-dynamic-info-runpath dyninfo))
-                           (interp  (elf-interpreter elf)))
+                     (let* ((runpath (elf-dynamic-info-runpath dyninfo))
+                            (gconv   (any gconv-directory runpath))
+                            (interp  (elf-interpreter elf)))
                        (if interp
                            (list (string-append "-DPROGRAM_INTERPRETER=\""
                                                 interp "\"")
@@ -762,7 +769,12 @@ last resort for relocation."
                                                  ", ")
                                                 ", NULL }")
                                  (string-append "-DFAKECHROOT_LIBRARY=\""
-                                                #$(fakechroot-library) "\""))
+                                                #$(fakechroot-library) "\"")
+
+                                 (if gconv
+                                     (string-append "-DGCONV_DIRECTORY=\""
+                                                    gconv "\"")
+                                     "-UGCONV_DIRECTORY"))
                            '())))))
                 '()))
 

^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [bug#41189] [PATCH 0/3] Add Fakechroot engine for 'guix pack -RR'
  2020-05-12 10:03   ` Ludovic Courtès
@ 2020-05-12 12:09     ` Carlos O'Donell
  2020-05-12 15:32       ` Ludovic Courtès
  0 siblings, 1 reply; 14+ messages in thread
From: Carlos O'Donell @ 2020-05-12 12:09 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 41189

On 5/12/20 6:03 AM, Ludovic Courtès wrote:
> One thing that won’t work is dlopen because our ‘--library-path’
> argument is computed statically based on the RUNPATH of the wrapped
> program.  So for instance if you try to load guile-readline.so from
> Guile, it eventually fails because libreadline.so isn’t found
> (libreadline.so is in the RUNPATH of guile-readline.so, but the loader
> uses non-interposable calls here as well.)  Probably no simple solution
> to that one.

There is a simple solution. You need to write a dynamic loader audit module
that handles la_objsearch() and inject your lookup path. See man 7 rtld-audit.
The dynamic loader audit modules allow you to alter the loader's core behaviour
with a plugin.

-- 
Cheers,
Carlos.





^ permalink raw reply	[flat|nested] 14+ messages in thread

* [bug#41189] [PATCH 0/3] Add Fakechroot engine for 'guix pack -RR'
  2020-05-12 12:09     ` Carlos O'Donell
@ 2020-05-12 15:32       ` Ludovic Courtès
  2020-05-13 12:52         ` [bug#41189] [PATCH v2 0/4] " Ludovic Courtès
  0 siblings, 1 reply; 14+ messages in thread
From: Ludovic Courtès @ 2020-05-12 15:32 UTC (permalink / raw)
  To: Carlos O'Donell; +Cc: 41189

[-- Attachment #1: Type: text/plain, Size: 1179 bytes --]

Hi Carlos,

Carlos O'Donell <carlos@redhat.com> skribis:

> On 5/12/20 6:03 AM, Ludovic Courtès wrote:
>> One thing that won’t work is dlopen because our ‘--library-path’
>> argument is computed statically based on the RUNPATH of the wrapped
>> program.  So for instance if you try to load guile-readline.so from
>> Guile, it eventually fails because libreadline.so isn’t found
>> (libreadline.so is in the RUNPATH of guile-readline.so, but the loader
>> uses non-interposable calls here as well.)  Probably no simple solution
>> to that one.
>
> There is a simple solution. You need to write a dynamic loader audit module
> that handles la_objsearch() and inject your lookup path. See man 7 rtld-audit.
> The dynamic loader audit modules allow you to alter the loader's core behaviour
> with a plugin.

That’s a great idea, and it works like a charm.  Thank you!

Concretely, I can do:

  guix pack -RR -S /bin=bin -S /etc=etc guile guile-readline

and then, on the target machine:

  tar xf pack.tar.gz
  . ./etc/profile
  export GUIX_EXECUTION_ENGINE=fakechroot
  ./bin/guile -c '(use-modules (ice-9 readline))'

Neat!

Ludo’.


[-- Attachment #2: Type: text/x-patch, Size: 6417 bytes --]

diff --git a/Makefile.am b/Makefile.am
index 6cd6e79cab..f65bf5f900 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -338,6 +338,7 @@ AUX_FILES =						\
   gnu/packages/aux-files/linux-libre/4.9-x86_64.conf	\
   gnu/packages/aux-files/linux-libre/4.4-i686.conf	\
   gnu/packages/aux-files/linux-libre/4.4-x86_64.conf	\
+  gnu/packages/aux-files/pack-audit.c			\
   gnu/packages/aux-files/run-in-namespace.c
 
 # Templates, examples.
diff --git a/gnu/packages/aux-files/pack-audit.c b/gnu/packages/aux-files/pack-audit.c
new file mode 100644
index 0000000000..374787e8b9
--- /dev/null
+++ b/gnu/packages/aux-files/pack-audit.c
@@ -0,0 +1,85 @@
+/* GNU Guix --- Functional package management for GNU
+   Copyright (C) 2020 Ludovic Courtès <ludo@gnu.org>
+
+   This file is part of GNU Guix.
+
+   GNU Guix is free software; you can redistribute it and/or modify it
+   under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or (at
+   your option) any later version.
+
+   GNU Guix is distributed in the hope that it will be useful, but
+   WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.  */
+
+/* This file implements part of the GNU ld.so audit interface.  It is used by
+   the "fakechroot" engine of the 'guix pack -RR' wrappers to make sure the
+   loader looks for shared objects under the "fake" root directory.  */
+
+#define _GNU_SOURCE 1
+
+#include <link.h>
+
+#include <error.h>
+#include <stdlib.h>
+#include <string.h>
+#include <assert.h>
+
+/* The pseudo root directory and store that we are relocating to.  */
+static const char *root_directory;
+static char *store;
+
+/* The original store, "/gnu/store" by default.  */
+static const char original_store[] = "@STORE_DIRECTORY@";
+
+/* Like 'malloc', but abort if 'malloc' returns NULL.  */
+static void *
+xmalloc (size_t size)
+{
+  void *result = malloc (size);
+  assert (result != NULL);
+  return result;
+}
+
+unsigned int
+la_version (unsigned int v)
+{
+  if (v != LAV_CURRENT)
+    error (1, 0, "cannot handle interface version %u", v);
+
+  root_directory = getenv ("FAKECHROOT_BASE");
+  if (root_directory == NULL)
+    error (1, 0, "'FAKECHROOT_BASE' is not set");
+
+  store = xmalloc (strlen (root_directory) + sizeof original_store);
+  strcpy (store, root_directory);
+  strcat (store, original_store);
+
+  return v;
+}
+
+/* Return NAME, a shared object file name, relocated under STORE.  This
+   function is called by the loader whenever it looks for a shared object.  */
+char *
+la_objsearch (const char *name, uintptr_t *cookie, unsigned int flag)
+{
+  char *result;
+
+  if (strncmp (name, original_store,
+	       sizeof original_store - 1) == 0)
+    {
+      size_t len = strlen (name) - sizeof original_store
+	+ strlen (store) + 1;
+      result = xmalloc (len);
+      strcpy (result, store);
+      strcat (result, name + sizeof original_store - 1);
+    }
+  else
+    result = strdup (name);
+
+  return result;
+}
diff --git a/gnu/packages/aux-files/run-in-namespace.c b/gnu/packages/aux-files/run-in-namespace.c
index c56c35a510..5ed1c6a1bc 100644
--- a/gnu/packages/aux-files/run-in-namespace.c
+++ b/gnu/packages/aux-files/run-in-namespace.c
@@ -400,7 +400,7 @@ exec_with_loader (const char *store, int argc, char *argv[])
 
   char *loader = concat (store,
 			 PROGRAM_INTERPRETER + sizeof "@STORE_DIRECTORY@");
-  size_t loader_specific_argc = 6;
+  size_t loader_specific_argc = 8;
   size_t loader_argc = argc + loader_specific_argc;
   char *loader_argv[loader_argc + 1];
   loader_argv[0] = argv[0];
@@ -409,7 +409,10 @@ exec_with_loader (const char *store, int argc, char *argv[])
   loader_argv[3] = "--preload";
   loader_argv[4] = concat (store,
 			   FAKECHROOT_LIBRARY + sizeof "@STORE_DIRECTORY@");
-  loader_argv[5] = concat (store,
+  loader_argv[5] = "--audit";
+  loader_argv[6] = concat (store,
+			   LOADER_AUDIT_MODULE + sizeof "@STORE_DIRECTORY@");
+  loader_argv[7] = concat (store,
 			   "@WRAPPED_PROGRAM@" + sizeof "@STORE_DIRECTORY@");
 
   for (size_t i = 0; i < argc; i++)
diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm
index 2b37bf5027..13ede581e9 100644
--- a/guix/scripts/pack.scm
+++ b/guix/scripts/pack.scm
@@ -681,6 +681,9 @@ last resort for relocation."
   (define runner
     (local-file (search-auxiliary-file "run-in-namespace.c")))
 
+  (define audit-source
+    (local-file (search-auxiliary-file "pack-audit.c")))
+
   (define (proot)
     (specification->package "proot-static"))
 
@@ -691,6 +694,21 @@ last resort for relocation."
                                    "/lib/fakechroot/libfakechroot.so")
                                 #$output)))
 
+  (define (audit-module)
+    (computed-file "pack-audit.so"
+                   (with-imported-modules '((guix build utils))
+                     #~(begin
+                         (use-modules (guix build utils))
+
+                         (copy-file #$audit-source "audit.c")
+                         (substitute* "audit.c"
+                           (("@STORE_DIRECTORY@")
+                            (%store-directory)))
+
+                         (invoke #$compiler "-std=gnu99"
+                                 "-shared" "-fPIC" "-Os" "-g0"
+                                 "-Wall" "audit.c" "-o" #$output)))))
+
   (define build
     (with-imported-modules (source-module-closure
                             '((guix build utils)
@@ -771,6 +789,8 @@ last resort for relocation."
                                  (string-append "-DFAKECHROOT_LIBRARY=\""
                                                 #$(fakechroot-library) "\"")
 
+                                 (string-append "-DLOADER_AUDIT_MODULE=\""
+                                                #$(audit-module) "\"")
                                  (if gconv
                                      (string-append "-DGCONV_DIRECTORY=\""
                                                     gconv "\"")

^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [bug#41189] [PATCH v2 0/4] Add Fakechroot engine for 'guix pack -RR'
  2020-05-12 15:32       ` Ludovic Courtès
@ 2020-05-13 12:52         ` Ludovic Courtès
  2020-05-13 12:52           ` [bug#41189] [PATCH v2 1/4] pack: Wrapper honors 'GUIX_EXECUTION_ENGINE' environment variable Ludovic Courtès
                             ` (4 more replies)
  0 siblings, 5 replies; 14+ messages in thread
From: Ludovic Courtès @ 2020-05-13 12:52 UTC (permalink / raw)
  To: 41189; +Cc: Ludovic Courtès

Hello!

This version incorporates changes discussed in this thread along with
subsequent cleanups:

  • Set ‘GCONV_PATH’.

  • Build an ld.so audit module to rewrite store file names read
    from RUNPATH entries and the likes.

  • Get rid of the ‘--library-path’ argument to ld.so, which is now
    useless since the audit module takes care of rewriting .so file
    names.  This simplifies both ‘exec_with_loader’ in the wrapper
    and ‘wrapped-package’ in ‘guix pack’.

Ludo’.

Ludovic Courtès (4):
  pack: Wrapper honors 'GUIX_EXECUTION_ENGINE' environment variable.
  pack: Factorize store references in wrapper.
  gnu: Add fakechroot.
  pack: Add relocation via ld.so and fakechroot.

 Makefile.am                               |   1 +
 doc/guix.texi                             |  43 +++-
 gnu/packages/aux-files/pack-audit.c       |  85 ++++++++
 gnu/packages/aux-files/run-in-namespace.c | 251 +++++++++++++++++++---
 gnu/packages/linux.scm                    |  30 +++
 guix/scripts/pack.scm                     |  87 +++++++-
 tests/guix-pack-relocatable.sh            |  23 ++
 7 files changed, 479 insertions(+), 41 deletions(-)
 create mode 100644 gnu/packages/aux-files/pack-audit.c

-- 
2.26.2





^ permalink raw reply	[flat|nested] 14+ messages in thread

* [bug#41189] [PATCH v2 1/4] pack: Wrapper honors 'GUIX_EXECUTION_ENGINE' environment variable.
  2020-05-13 12:52         ` [bug#41189] [PATCH v2 0/4] " Ludovic Courtès
@ 2020-05-13 12:52           ` Ludovic Courtès
  2020-05-13 12:52           ` [bug#41189] [PATCH v2 2/4] pack: Factorize store references in wrapper Ludovic Courtès
                             ` (3 subsequent siblings)
  4 siblings, 0 replies; 14+ messages in thread
From: Ludovic Courtès @ 2020-05-13 12:52 UTC (permalink / raw)
  To: 41189; +Cc: Ludovic Courtès

* gnu/packages/aux-files/run-in-namespace.c (struct engine): New type.
(exec_default): New function.
(engines): New variable.
(execution_engine): New function.
(main): Use it instead of calling 'exec_in_user_namespace' and
'exec_with_proot' directly.
* tests/guix-pack-relocatable.sh: Add test with 'GUIX_EXECUTION_ENGINE'.
* doc/guix.texi (Invoking guix pack): Document 'GUIX_EXECUTION_ENGINE'.
---
 doc/guix.texi                             | 30 +++++++--
 gnu/packages/aux-files/run-in-namespace.c | 78 ++++++++++++++++++++---
 tests/guix-pack-relocatable.sh            | 17 +++++
 3 files changed, 110 insertions(+), 15 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 0cba0ee1ec..958ed9ceec 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -5185,9 +5185,9 @@ When this option is passed once, the resulting binaries require support for
 @dfn{user namespaces} in the kernel Linux; when passed
 @emph{twice}@footnote{Here's a trick to memorize it: @code{-RR}, which adds
 PRoot support, can be thought of as the abbreviation of ``Really
-Relocatable''.  Neat, isn't it?}, relocatable binaries fall to back to PRoot
-if user namespaces are unavailable, and essentially work anywhere---see below
-for the implications.
+Relocatable''.  Neat, isn't it?}, relocatable binaries fall to back to
+other techniques if user namespaces are unavailable, and essentially
+work anywhere---see below for the implications.
 
 For example, if you create a pack containing Bash with:
 
@@ -5219,14 +5219,32 @@ turn it off.
 
 To produce relocatable binaries that work even in the absence of user
 namespaces, pass @option{--relocatable} or @option{-R} @emph{twice}.  In that
-case, binaries will try user namespace support and fall back to PRoot if user
-namespaces are not supported.
+case, binaries will try user namespace support and fall back to another
+@dfn{execution engine} if user namespaces are not supported.  The
+following execution engines are supported:
 
-The @uref{https://proot-me.github.io/, PRoot} program provides the necessary
+@table @code
+@item default
+Try user namespaces and fall back to PRoot if user namespaces are not
+supported (see below).
+
+@item userns
+Run the program through user namespaces and abort if they are not
+supported.
+
+@item proot
+Run through PRoot.  The @uref{https://proot-me.github.io/, PRoot} program
+provides the necessary
 support for file system virtualization.  It achieves that by using the
 @code{ptrace} system call on the running program.  This approach has the
 advantage to work without requiring special kernel support, but it incurs
 run-time overhead every time a system call is made.
+@end table
+
+@vindex GUIX_EXECUTION_ENGINE
+When running a wrapped program, you can explicitly request one of the
+execution engines listed above by setting the
+@code{GUIX_EXECUTION_ENGINE} environment variable accordingly.
 @end quotation
 
 @cindex entry point, for Docker images
diff --git a/gnu/packages/aux-files/run-in-namespace.c b/gnu/packages/aux-files/run-in-namespace.c
index 23e7875173..6beac7fd53 100644
--- a/gnu/packages/aux-files/run-in-namespace.c
+++ b/gnu/packages/aux-files/run-in-namespace.c
@@ -336,6 +336,71 @@ exec_with_proot (const char *store, int argc, char *argv[])
 
 #endif
 
+\f
+/* Execution engines.  */
+
+struct engine
+{
+  const char *name;
+  void (* exec) (const char *, int, char **);
+};
+
+static void
+buffer_stderr (void)
+{
+  static char stderr_buffer[4096];
+  setvbuf (stderr, stderr_buffer, _IOFBF, sizeof stderr_buffer);
+}
+
+/* The default engine.  */
+static void
+exec_default (const char *store, int argc, char *argv[])
+{
+  /* Buffer stderr so that nothing's displayed if 'exec_in_user_namespace'
+     fails but 'exec_with_proot' works.  */
+  buffer_stderr ();
+
+  exec_in_user_namespace (store, argc, argv);
+#ifdef PROOT_PROGRAM
+  exec_with_proot (store, argc, argv);
+#endif
+}
+
+/* List of supported engines.  */
+static const struct engine engines[] =
+  {
+   { "default", exec_default },
+   { "userns", exec_in_user_namespace },
+#ifdef PROOT_PROGRAM
+   { "proot", exec_with_proot },
+#endif
+   { NULL, NULL }
+  };
+
+/* Return the "execution engine" to use.  */
+static const struct engine *
+execution_engine (void)
+{
+  const char *str = getenv ("GUIX_EXECUTION_ENGINE");
+
+  if (str == NULL)
+    str = "default";
+
+ try:
+  for (const struct engine *engine = engines;
+       engine->name != NULL;
+       engine++)
+    {
+      if (strcmp (engine->name, str) == 0)
+	return engine;
+    }
+
+  fprintf (stderr, "%s: unsupported Guix execution engine; ignoring\n",
+	   str);
+  str = "default";
+  goto try;
+}
+
 \f
 int
 main (int argc, char *argv[])
@@ -362,22 +427,17 @@ main (int argc, char *argv[])
   if (strcmp (store, "@STORE_DIRECTORY@") != 0
       && lstat ("@WRAPPED_PROGRAM@", &statbuf) != 0)
     {
-      /* Buffer stderr so that nothing's displayed if 'exec_in_user_namespace'
-	 fails but 'exec_with_proot' works.  */
-      static char stderr_buffer[4096];
-      setvbuf (stderr, stderr_buffer, _IOFBF, sizeof stderr_buffer);
+      const struct engine *engine = execution_engine ();
+      engine->exec (store, argc, argv);
 
-      exec_in_user_namespace (store, argc, argv);
-#ifdef PROOT_PROGRAM
-      exec_with_proot (store, argc, argv);
-#else
+      /* If we reach this point, that's because ENGINE failed to do the
+	 job.  */
       fprintf (stderr, "\
 This may be because \"user namespaces\" are not supported on this system.\n\
 Consequently, we cannot run '@WRAPPED_PROGRAM@',\n\
 unless you move it to the '@STORE_DIRECTORY@' directory.\n\
 \n\
 Please refer to the 'guix pack' documentation for more information.\n");
-#endif
       return EXIT_FAILURE;
     }
 
diff --git a/tests/guix-pack-relocatable.sh b/tests/guix-pack-relocatable.sh
index a3fd45623c..cb56815fed 100644
--- a/tests/guix-pack-relocatable.sh
+++ b/tests/guix-pack-relocatable.sh
@@ -84,6 +84,23 @@ fi
 grep 'GNU sed' "$test_directory/output"
 chmod -Rf +w "$test_directory"; rm -rf "$test_directory"/*
 
+case "`uname -m`" in
+    x86_64|i?86)
+	# Try '-RR' and PRoot.
+	tarball="`guix pack -RR -S /Bin=bin sed`"
+	tar tvf "$tarball" | grep /bin/proot
+	(cd "$test_directory"; tar xvf "$tarball")
+	GUIX_EXECUTION_ENGINE="proot"
+	export GUIX_EXECUTION_ENGINE
+	"$test_directory/Bin/sed" --version > "$test_directory/output"
+	grep 'GNU sed' "$test_directory/output"
+	chmod -Rf +w "$test_directory"; rm -rf "$test_directory"/*
+	;;
+    *)
+	echo "skipping PRoot test" >&2
+	;;
+esac
+
 # Ensure '-R' works with outputs other than "out".
 tarball="`guix pack -R -S /share=share groff:doc`"
 (cd "$test_directory"; tar xvf "$tarball")
-- 
2.26.2





^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [bug#41189] [PATCH v2 2/4] pack: Factorize store references in wrapper.
  2020-05-13 12:52         ` [bug#41189] [PATCH v2 0/4] " Ludovic Courtès
  2020-05-13 12:52           ` [bug#41189] [PATCH v2 1/4] pack: Wrapper honors 'GUIX_EXECUTION_ENGINE' environment variable Ludovic Courtès
@ 2020-05-13 12:52           ` Ludovic Courtès
  2020-05-13 12:52           ` [bug#41189] [PATCH v2 3/4] gnu: Add fakechroot Ludovic Courtès
                             ` (2 subsequent siblings)
  4 siblings, 0 replies; 14+ messages in thread
From: Ludovic Courtès @ 2020-05-13 12:52 UTC (permalink / raw)
  To: 41189; +Cc: Ludovic Courtès

* gnu/packages/aux-files/run-in-namespace.c (original_store): New variable.
(exec_in_user_namespace, exec_with_proot, main): Use it instead of the
literal "@STORE_DIRECTORY@".
---
 gnu/packages/aux-files/run-in-namespace.c | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/gnu/packages/aux-files/run-in-namespace.c b/gnu/packages/aux-files/run-in-namespace.c
index 6beac7fd53..6e97359078 100644
--- a/gnu/packages/aux-files/run-in-namespace.c
+++ b/gnu/packages/aux-files/run-in-namespace.c
@@ -42,6 +42,10 @@
 #include <dirent.h>
 #include <sys/syscall.h>
 
+/* The original store, "/gnu/store" by default.  */
+static const char original_store[] = "@STORE_DIRECTORY@";
+
+
 /* Like 'malloc', but abort if 'malloc' returns NULL.  */
 static void *
 xmalloc (size_t size)
@@ -228,7 +232,7 @@ exec_in_user_namespace (const char *store, int argc, char *argv[])
      bind-mounted in the right place.  */
   int err;
   char *new_root = mkdtemp (strdup ("/tmp/guix-exec-XXXXXX"));
-  char *new_store = concat (new_root, "@STORE_DIRECTORY@");
+  char *new_store = concat (new_root, original_store);
   char *cwd = get_current_dir_name ();
 
   /* Create a child with separate namespaces and set up bind-mounts from
@@ -307,11 +311,11 @@ exec_with_proot (const char *store, int argc, char *argv[])
   int proot_specific_argc = 4;
   int proot_argc = argc + proot_specific_argc;
   char *proot_argv[proot_argc + 1], *proot;
-  char bind_spec[strlen (store) + 1 + sizeof "@STORE_DIRECTORY@"];
+  char bind_spec[strlen (store) + 1 + sizeof original_store];
 
   strcpy (bind_spec, store);
   strcat (bind_spec, ":");
-  strcat (bind_spec, "@STORE_DIRECTORY@");
+  strcat (bind_spec, original_store);
 
   proot = concat (store, PROOT_PROGRAM);
 
@@ -413,8 +417,7 @@ main (int argc, char *argv[])
   /* SELF is something like "/home/ludo/.local/gnu/store/…-foo/bin/ls" and we
      want to extract "/home/ludo/.local/gnu/store".  */
   size_t index = strlen (self)
-    - strlen ("@WRAPPED_PROGRAM@")
-    + strlen ("@STORE_DIRECTORY@");
+    - strlen ("@WRAPPED_PROGRAM@") + strlen (original_store);
   char *store = strdup (self);
   store[index] = '\0';
 
@@ -424,7 +427,7 @@ main (int argc, char *argv[])
      @WRAPPED_PROGRAM@ right away.  This is not just an optimization: it's
      needed when running one of these wrappers from within an unshare'd
      namespace, because 'unshare' fails with EPERM in that context.  */
-  if (strcmp (store, "@STORE_DIRECTORY@") != 0
+  if (strcmp (store, original_store) != 0
       && lstat ("@WRAPPED_PROGRAM@", &statbuf) != 0)
     {
       const struct engine *engine = execution_engine ();
-- 
2.26.2





^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [bug#41189] [PATCH v2 3/4] gnu: Add fakechroot.
  2020-05-13 12:52         ` [bug#41189] [PATCH v2 0/4] " Ludovic Courtès
  2020-05-13 12:52           ` [bug#41189] [PATCH v2 1/4] pack: Wrapper honors 'GUIX_EXECUTION_ENGINE' environment variable Ludovic Courtès
  2020-05-13 12:52           ` [bug#41189] [PATCH v2 2/4] pack: Factorize store references in wrapper Ludovic Courtès
@ 2020-05-13 12:52           ` Ludovic Courtès
  2020-05-13 12:52           ` [bug#41189] [PATCH v2 4/4] pack: Add relocation via ld.so and fakechroot Ludovic Courtès
  2020-05-14 15:24           ` bug#41189: [PATCH v2 0/4] Add Fakechroot engine for 'guix pack -RR' Ludovic Courtès
  4 siblings, 0 replies; 14+ messages in thread
From: Ludovic Courtès @ 2020-05-13 12:52 UTC (permalink / raw)
  To: 41189; +Cc: Ludovic Courtès

* gnu/packages/linux.scm (fakechroot): New variable.
---
 gnu/packages/linux.scm | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm
index 7cf7521e24..35526b3513 100644
--- a/gnu/packages/linux.scm
+++ b/gnu/packages/linux.scm
@@ -6793,6 +6793,36 @@ have to construct the archives directly, without using the archiver.")
     (home-page "http://freshmeat.sourceforge.net/projects/fakeroot")
     (license license:gpl3+)))
 
+(define-public fakechroot
+  (package
+    (name "fakechroot")
+    (version "2.20.1")
+    (source (origin
+              (method url-fetch)
+              (uri (string-append
+                    "https://github.com/dex4er/fakechroot/releases/download/"
+                    version "/fakechroot-" version ".tar.gz"))
+              (sha256
+               (base32
+                "1aijkd0b45wav25v01qhw8zxwa3pl0nnp9fabmmy1nlx7hr09gas"))))
+    (build-system gnu-build-system)
+    (arguments
+     ;; XXX: The tests heavily assume they run on an FHS system so for now
+     ;; skip them.
+     '(#:tests? #f
+       #:configure-flags '("--disable-static")))
+    (synopsis "Emulate @code{chroot} by overriding file system calls")
+    (description
+     "@command{fakechroot} runs a command in an environment were is additional
+possibility to use @code{chroot} command without root privileges.  This is
+useful for allowing users to create own chrooted environment with possibility
+to install another packages without need for root privileges.
+
+It works by providing @file{libfakechroot.so}, a shared library meant to be
+set as @code{LD_PRELOAD} to override the C library file system functions.")
+    (home-page "https://github.com/dex4er/fakechroot/")
+    (license license:lgpl2.1+)))
+
 (define-public inputattach
   (package
     (name "inputattach")
-- 
2.26.2





^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [bug#41189] [PATCH v2 4/4] pack: Add relocation via ld.so and fakechroot.
  2020-05-13 12:52         ` [bug#41189] [PATCH v2 0/4] " Ludovic Courtès
                             ` (2 preceding siblings ...)
  2020-05-13 12:52           ` [bug#41189] [PATCH v2 3/4] gnu: Add fakechroot Ludovic Courtès
@ 2020-05-13 12:52           ` Ludovic Courtès
  2020-05-14 15:24           ` bug#41189: [PATCH v2 0/4] Add Fakechroot engine for 'guix pack -RR' Ludovic Courtès
  4 siblings, 0 replies; 14+ messages in thread
From: Ludovic Courtès @ 2020-05-13 12:52 UTC (permalink / raw)
  To: 41189; +Cc: Ludovic Courtès

From: Ludovic Courtès <ludovic.courtes@inria.fr>

* gnu/packages/aux-files/run-in-namespace.c (HAVE_EXEC_WITH_LOADER): New
macro.
(bind_mount): Rename to...
(mirror_directory): ... this.  Add 'firmlink' argument and use it
instead of calling mkdir/open/close/mount directly.
(bind_mount, make_symlink): New functions.
(exec_in_user_namespace): Adjust accordingly.
(exec_with_loader) [HAVE_EXEC_WITH_LOADER]: New function.
(exec_performance): New function.
(engines): Add them.
* guix/scripts/pack.scm (wrapped-package)[fakechroot-library]
[audit-module]: New procedures.
[audit-source]: New variable.
[build](elf-interpreter, elf-loader-compile-flags): New procedures.
(build-wrapper): Use them.
* tests/guix-pack-relocatable.sh: Test with
'GUIX_EXECUTION_ENGINE=fakechroot'.
* doc/guix.texi (Invoking guix pack): Document the 'performance' and
'fakechroot' engines.
* gnu/packages/aux-files/pack-audit.c: New file.
* Makefile.am (AUX_FILES): Add it.
---
 Makefile.am                               |   1 +
 doc/guix.texi                             |  13 ++
 gnu/packages/aux-files/pack-audit.c       |  85 ++++++++++++
 gnu/packages/aux-files/run-in-namespace.c | 160 ++++++++++++++++++++--
 guix/scripts/pack.scm                     |  87 +++++++++++-
 tests/guix-pack-relocatable.sh            |   6 +
 6 files changed, 331 insertions(+), 21 deletions(-)
 create mode 100644 gnu/packages/aux-files/pack-audit.c

diff --git a/Makefile.am b/Makefile.am
index 6cd6e79cab..f65bf5f900 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -338,6 +338,7 @@ AUX_FILES =						\
   gnu/packages/aux-files/linux-libre/4.9-x86_64.conf	\
   gnu/packages/aux-files/linux-libre/4.4-i686.conf	\
   gnu/packages/aux-files/linux-libre/4.4-x86_64.conf	\
+  gnu/packages/aux-files/pack-audit.c			\
   gnu/packages/aux-files/run-in-namespace.c
 
 # Templates, examples.
diff --git a/doc/guix.texi b/doc/guix.texi
index 958ed9ceec..a70a058afb 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -5228,6 +5228,10 @@ following execution engines are supported:
 Try user namespaces and fall back to PRoot if user namespaces are not
 supported (see below).
 
+@item performance
+Try user namespaces and fall back to Fakechroot if user namespaces are
+not supported (see below).
+
 @item userns
 Run the program through user namespaces and abort if they are not
 supported.
@@ -5239,6 +5243,15 @@ support for file system virtualization.  It achieves that by using the
 @code{ptrace} system call on the running program.  This approach has the
 advantage to work without requiring special kernel support, but it incurs
 run-time overhead every time a system call is made.
+
+@item fakechroot
+Run through Fakechroot.  @uref{https://github.com/dex4er/fakechroot/,
+Fakechroot} virtualizes file system accesses by intercepting calls to C
+library functions such as @code{open}, @code{stat}, @code{exec}, and so
+on.  Unlike PRoot, it incurs very little overhead.  However, it does not
+always work: for example, some file system accesses made from within the
+C library are not intercepted, and file system accesses made @i{via}
+direct syscalls are not intercepted either, leading to erratic behavior.
 @end table
 
 @vindex GUIX_EXECUTION_ENGINE
diff --git a/gnu/packages/aux-files/pack-audit.c b/gnu/packages/aux-files/pack-audit.c
new file mode 100644
index 0000000000..374787e8b9
--- /dev/null
+++ b/gnu/packages/aux-files/pack-audit.c
@@ -0,0 +1,85 @@
+/* GNU Guix --- Functional package management for GNU
+   Copyright (C) 2020 Ludovic Courtès <ludo@gnu.org>
+
+   This file is part of GNU Guix.
+
+   GNU Guix is free software; you can redistribute it and/or modify it
+   under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or (at
+   your option) any later version.
+
+   GNU Guix is distributed in the hope that it will be useful, but
+   WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.  */
+
+/* This file implements part of the GNU ld.so audit interface.  It is used by
+   the "fakechroot" engine of the 'guix pack -RR' wrappers to make sure the
+   loader looks for shared objects under the "fake" root directory.  */
+
+#define _GNU_SOURCE 1
+
+#include <link.h>
+
+#include <error.h>
+#include <stdlib.h>
+#include <string.h>
+#include <assert.h>
+
+/* The pseudo root directory and store that we are relocating to.  */
+static const char *root_directory;
+static char *store;
+
+/* The original store, "/gnu/store" by default.  */
+static const char original_store[] = "@STORE_DIRECTORY@";
+
+/* Like 'malloc', but abort if 'malloc' returns NULL.  */
+static void *
+xmalloc (size_t size)
+{
+  void *result = malloc (size);
+  assert (result != NULL);
+  return result;
+}
+
+unsigned int
+la_version (unsigned int v)
+{
+  if (v != LAV_CURRENT)
+    error (1, 0, "cannot handle interface version %u", v);
+
+  root_directory = getenv ("FAKECHROOT_BASE");
+  if (root_directory == NULL)
+    error (1, 0, "'FAKECHROOT_BASE' is not set");
+
+  store = xmalloc (strlen (root_directory) + sizeof original_store);
+  strcpy (store, root_directory);
+  strcat (store, original_store);
+
+  return v;
+}
+
+/* Return NAME, a shared object file name, relocated under STORE.  This
+   function is called by the loader whenever it looks for a shared object.  */
+char *
+la_objsearch (const char *name, uintptr_t *cookie, unsigned int flag)
+{
+  char *result;
+
+  if (strncmp (name, original_store,
+	       sizeof original_store - 1) == 0)
+    {
+      size_t len = strlen (name) - sizeof original_store
+	+ strlen (store) + 1;
+      result = xmalloc (len);
+      strcpy (result, store);
+      strcat (result, name + sizeof original_store - 1);
+    }
+  else
+    result = strdup (name);
+
+  return result;
+}
diff --git a/gnu/packages/aux-files/run-in-namespace.c b/gnu/packages/aux-files/run-in-namespace.c
index 6e97359078..5a6b932b87 100644
--- a/gnu/packages/aux-files/run-in-namespace.c
+++ b/gnu/packages/aux-files/run-in-namespace.c
@@ -42,6 +42,11 @@
 #include <dirent.h>
 #include <sys/syscall.h>
 
+/* Whether we're building the ld.so/libfakechroot wrapper.  */
+#define HAVE_EXEC_WITH_LOADER						\
+  (defined PROGRAM_INTERPRETER) && (defined LOADER_AUDIT_MODULE)	\
+  && (defined FAKECHROOT_LIBRARY)
+
 /* The original store, "/gnu/store" by default.  */
 static const char original_store[] = "@STORE_DIRECTORY@";
 
@@ -117,9 +122,42 @@ rm_rf (const char *directory)
     assert_perror (errno);
 }
 
-/* Bind mount all the top-level entries in SOURCE to TARGET.  */
+/* Make TARGET a bind-mount of SOURCE.  Take into account ENTRY's type, which
+   corresponds to SOURCE.  */
+static int
+bind_mount (const char *source, const struct dirent *entry,
+	    const char *target)
+{
+  if (entry->d_type == DT_DIR)
+    {
+      int err = mkdir (target, 0700);
+      if (err != 0)
+	return err;
+    }
+  else
+    close (open (target, O_WRONLY | O_CREAT));
+
+  return mount (source, target, "none",
+		MS_BIND | MS_REC | MS_RDONLY, NULL);
+}
+
+#if HAVE_EXEC_WITH_LOADER
+
+/* Make TARGET a symlink to SOURCE.  */
+static int
+make_symlink (const char *source, const struct dirent *entry,
+	      const char *target)
+{
+  return symlink (source, target);
+}
+
+#endif
+
+/* Mirror with FIRMLINK all the top-level entries in SOURCE to TARGET.  */
 static void
-bind_mount (const char *source, const char *target)
+mirror_directory (const char *source, const char *target,
+		  int (* firmlink) (const char *, const struct dirent *,
+				    const char *))
 {
   DIR *stream = opendir (source);
 
@@ -154,17 +192,7 @@ bind_mount (const char *source, const char *target)
       else
 	{
 	  /* Create the mount point.  */
-	  if (entry->d_type == DT_DIR)
-	    {
-	      int err = mkdir (new_entry, 0700);
-	      if (err != 0)
-		assert_perror (errno);
-	    }
-	  else
-	    close (open (new_entry, O_WRONLY | O_CREAT));
-
-	  int err = mount (abs_source, new_entry, "none",
-			   MS_BIND | MS_REC | MS_RDONLY, NULL);
+	  int err = firmlink (abs_source, entry, new_entry);
 
 	  /* It used to be that only directories could be bind-mounted.  Thus,
 	     keep going if we fail to bind-mount a non-directory entry.
@@ -248,7 +276,7 @@ exec_in_user_namespace (const char *store, int argc, char *argv[])
       /* Note: Due to <https://bugzilla.kernel.org/show_bug.cgi?id=183461>
 	 we cannot make NEW_ROOT a tmpfs (which would have saved the need
 	 for 'rm_rf'.)  */
-      bind_mount ("/", new_root);
+      mirror_directory ("/", new_root, bind_mount);
       mkdir_p (new_store);
       err = mount (store, new_store, "none", MS_BIND | MS_REC | MS_RDONLY,
 		   NULL);
@@ -340,6 +368,92 @@ exec_with_proot (const char *store, int argc, char *argv[])
 
 #endif
 
+\f
+#if HAVE_EXEC_WITH_LOADER
+
+/* Execute the wrapped program by invoking the loader (ld.so) directly,
+   passing it the audit module and preloading libfakechroot.so.  */
+static void
+exec_with_loader (const char *store, int argc, char *argv[])
+{
+  char *loader = concat (store,
+			 PROGRAM_INTERPRETER + sizeof original_store);
+  size_t loader_specific_argc = 6;
+  size_t loader_argc = argc + loader_specific_argc;
+  char *loader_argv[loader_argc + 1];
+  loader_argv[0] = argv[0];
+  loader_argv[1] = "--audit";
+  loader_argv[2] = concat (store,
+			   LOADER_AUDIT_MODULE + sizeof original_store);
+  loader_argv[3] = "--preload";
+  loader_argv[4] = concat (store,
+			   FAKECHROOT_LIBRARY + sizeof original_store);
+  loader_argv[5] = concat (store,
+			   "@WRAPPED_PROGRAM@" + sizeof original_store);
+
+  for (size_t i = 0; i < argc; i++)
+    loader_argv[i + loader_specific_argc] = argv[i + 1];
+
+  loader_argv[loader_argc] = NULL;
+
+  /* Set up the root directory.  */
+  int err;
+  char *new_root = mkdtemp (strdup ("/tmp/guix-exec-XXXXXX"));
+  mirror_directory ("/", new_root, make_symlink);
+
+  char *new_store = concat (new_root, original_store);
+  char *new_store_parent = dirname (strdup (new_store));
+  mkdir_p (new_store_parent);
+  symlink (store, new_store);
+
+#ifdef GCONV_DIRECTORY
+  /* Tell libc where to find its gconv modules.  This is necessary because
+     gconv uses non-interposable 'open' calls.  */
+  char *gconv_path = concat (store,
+			     GCONV_DIRECTORY + sizeof original_store);
+  setenv ("GCONV_PATH", gconv_path, 1);
+  free (gconv_path);
+#endif
+
+  setenv ("FAKECHROOT_BASE", new_root, 1);
+
+  pid_t child = fork ();
+  switch (child)
+    {
+    case 0:
+      err = execv (loader, loader_argv);
+      if (err < 0)
+	assert_perror (errno);
+      exit (EXIT_FAILURE);
+      break;
+
+    case -1:
+      assert_perror (errno);
+      exit (EXIT_FAILURE);
+      break;
+
+    default:
+      {
+  	int status;
+	waitpid (child, &status, 0);
+	chdir ("/");			  /* avoid EBUSY */
+	rm_rf (new_root);
+	free (new_root);
+
+	close (2);			/* flushing stderr should be silent */
+
+	if (WIFEXITED (status))
+	  exit (WEXITSTATUS (status));
+	else
+	  /* Abnormal termination cannot really be reproduced, so exit
+	     with 255.  */
+	  exit (255);
+      }
+    }
+}
+
+#endif
+
 \f
 /* Execution engines.  */
 
@@ -356,7 +470,7 @@ buffer_stderr (void)
   setvbuf (stderr, stderr_buffer, _IOFBF, sizeof stderr_buffer);
 }
 
-/* The default engine.  */
+/* The default engine: choose a robust method.  */
 static void
 exec_default (const char *store, int argc, char *argv[])
 {
@@ -370,13 +484,29 @@ exec_default (const char *store, int argc, char *argv[])
 #endif
 }
 
+/* The "performance" engine: choose performance over robustness.  */
+static void
+exec_performance (const char *store, int argc, char *argv[])
+{
+  buffer_stderr ();
+
+  exec_in_user_namespace (store, argc, argv);
+#if HAVE_EXEC_WITH_LOADER
+  exec_with_loader (store, argc, argv);
+#endif
+}
+
 /* List of supported engines.  */
 static const struct engine engines[] =
   {
    { "default", exec_default },
+   { "performance", exec_performance },
    { "userns", exec_in_user_namespace },
 #ifdef PROOT_PROGRAM
    { "proot", exec_with_proot },
+#endif
+#if HAVE_EXEC_WITH_LOADER
+   { "fakechroot", exec_with_loader },
 #endif
    { NULL, NULL }
   };
diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm
index 580f696b41..23aab01701 100644
--- a/guix/scripts/pack.scm
+++ b/guix/scripts/pack.scm
@@ -681,18 +681,50 @@ last resort for relocation."
   (define runner
     (local-file (search-auxiliary-file "run-in-namespace.c")))
 
+  (define audit-source
+    (local-file (search-auxiliary-file "pack-audit.c")))
+
   (define (proot)
     (specification->package "proot-static"))
 
+  (define (fakechroot-library)
+    (computed-file "libfakechroot.so"
+                   #~(copy-file #$(file-append
+                                   (specification->package "fakechroot")
+                                   "/lib/fakechroot/libfakechroot.so")
+                                #$output)))
+
+  (define (audit-module)
+    ;; Return an ld.so audit module for use by the 'fakechroot' execution
+    ;; engine that translates file names of all the files ld.so loads.
+    (computed-file "pack-audit.so"
+                   (with-imported-modules '((guix build utils))
+                     #~(begin
+                         (use-modules (guix build utils))
+
+                         (copy-file #$audit-source "audit.c")
+                         (substitute* "audit.c"
+                           (("@STORE_DIRECTORY@")
+                            (%store-directory)))
+
+                         (invoke #$compiler "-std=gnu99"
+                                 "-shared" "-fPIC" "-Os" "-g0"
+                                 "-Wall" "audit.c" "-o" #$output)))))
+
   (define build
     (with-imported-modules (source-module-closure
                             '((guix build utils)
-                              (guix build union)))
+                              (guix build union)
+                              (guix elf)))
       #~(begin
           (use-modules (guix build utils)
                        ((guix build union) #:select (relative-file-name))
+                       (guix elf)
+                       (ice-9 binary-ports)
                        (ice-9 ftw)
-                       (ice-9 match))
+                       (ice-9 match)
+                       (srfi srfi-1)
+                       (rnrs bytevectors))
 
           (define input
             ;; The OUTPUT* output of PACKAGE.
@@ -711,6 +743,48 @@ last resort for relocation."
                 (#f    base)
                 (index (string-drop base index)))))
 
+          (define (elf-interpreter elf)
+            ;; Return the interpreter of ELF as a string, or #f if ELF has no
+            ;; interpreter segment.
+            (match (find (lambda (segment)
+                           (= (elf-segment-type segment) PT_INTERP))
+                         (elf-segments elf))
+              (#f #f)                             ;maybe a .so
+              (segment
+               (let ((bv (make-bytevector (- (elf-segment-memsz segment) 1))))
+                 (bytevector-copy! (elf-bytes elf)
+                                   (elf-segment-offset segment)
+                                   bv 0 (bytevector-length bv))
+                 (utf8->string bv)))))
+
+          (define (elf-loader-compile-flags program)
+            ;; Return the cpp flags defining macros for the ld.so/fakechroot
+            ;; wrapper of PROGRAM.
+
+            ;; TODO: Handle scripts by wrapping their interpreter.
+            (if (elf-file? program)
+                (let* ((bv      (call-with-input-file program
+                                  get-bytevector-all))
+                       (elf     (parse-elf bv))
+                       (interp  (elf-interpreter elf))
+                       (gconv   (and interp
+                                     (string-append (dirname interp)
+                                                    "/gconv"))))
+                  (if interp
+                      (list (string-append "-DPROGRAM_INTERPRETER=\""
+                                           interp "\"")
+                            (string-append "-DFAKECHROOT_LIBRARY=\""
+                                           #$(fakechroot-library) "\"")
+
+                            (string-append "-DLOADER_AUDIT_MODULE=\""
+                                           #$(audit-module) "\"")
+                            (if gconv
+                                (string-append "-DGCONV_DIRECTORY=\""
+                                               gconv "\"")
+                                "-UGCONV_DIRECTORY"))
+                      '()))
+                '()))
+
           (define (build-wrapper program)
             ;; Build a user-namespace wrapper for PROGRAM.
             (format #t "building wrapper for '~a'...~%" program)
@@ -730,10 +804,11 @@ last resort for relocation."
               (mkdir-p (dirname result))
               (apply invoke #$compiler "-std=gnu99" "-static" "-Os" "-g0" "-Wall"
                      "run.c" "-o" result
-                     (if proot
-                         (list (string-append "-DPROOT_PROGRAM=\""
-                                              proot "\""))
-                         '()))
+                     (append (if proot
+                                 (list (string-append "-DPROOT_PROGRAM=\""
+                                                      proot "\""))
+                                 '())
+                             (elf-loader-compile-flags program)))
               (delete-file "run.c")))
 
           (setvbuf (current-output-port) 'line)
diff --git a/tests/guix-pack-relocatable.sh b/tests/guix-pack-relocatable.sh
index cb56815fed..358cac5b26 100644
--- a/tests/guix-pack-relocatable.sh
+++ b/tests/guix-pack-relocatable.sh
@@ -94,6 +94,12 @@ case "`uname -m`" in
 	export GUIX_EXECUTION_ENGINE
 	"$test_directory/Bin/sed" --version > "$test_directory/output"
 	grep 'GNU sed' "$test_directory/output"
+
+	# Now with fakechroot.
+	GUIX_EXECUTION_ENGINE="fakechroot"
+	"$test_directory/Bin/sed" --version > "$test_directory/output"
+	grep 'GNU sed' "$test_directory/output"
+
 	chmod -Rf +w "$test_directory"; rm -rf "$test_directory"/*
 	;;
     *)
-- 
2.26.2





^ permalink raw reply related	[flat|nested] 14+ messages in thread

* bug#41189: [PATCH v2 0/4] Add Fakechroot engine for 'guix pack -RR'
  2020-05-13 12:52         ` [bug#41189] [PATCH v2 0/4] " Ludovic Courtès
                             ` (3 preceding siblings ...)
  2020-05-13 12:52           ` [bug#41189] [PATCH v2 4/4] pack: Add relocation via ld.so and fakechroot Ludovic Courtès
@ 2020-05-14 15:24           ` Ludovic Courtès
  4 siblings, 0 replies; 14+ messages in thread
From: Ludovic Courtès @ 2020-05-14 15:24 UTC (permalink / raw)
  To: 41189-done

Ludovic Courtès <ludo@gnu.org> skribis:

>   pack: Wrapper honors 'GUIX_EXECUTION_ENGINE' environment variable.
>   pack: Factorize store references in wrapper.
>   gnu: Add fakechroot.
>   pack: Add relocation via ld.so and fakechroot.

Pushed as 6456232164890dbf5aa20394ee24637feb4b7b9e!

Ludo’.




^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2020-05-14 15:43 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-05-11 17:05 [bug#41189] [PATCH 0/3] Add Fakechroot engine for 'guix pack -RR' Ludovic Courtès
2020-05-11 17:11 ` [bug#41189] [PATCH 1/3] pack: Wrapper honors 'GUIX_EXECUTION_ENGINE' environment variable Ludovic Courtès
2020-05-11 17:11   ` [bug#41189] [PATCH 2/3] gnu: Add fakechroot Ludovic Courtès
2020-05-11 17:11   ` [bug#41189] [PATCH 3/3] pack: Add relocation via ld.so and fakechroot Ludovic Courtès
2020-05-11 21:18 ` [bug#41189] [PATCH 0/3] Add Fakechroot engine for 'guix pack -RR' Carlos O'Donell
2020-05-12 10:03   ` Ludovic Courtès
2020-05-12 12:09     ` Carlos O'Donell
2020-05-12 15:32       ` Ludovic Courtès
2020-05-13 12:52         ` [bug#41189] [PATCH v2 0/4] " Ludovic Courtès
2020-05-13 12:52           ` [bug#41189] [PATCH v2 1/4] pack: Wrapper honors 'GUIX_EXECUTION_ENGINE' environment variable Ludovic Courtès
2020-05-13 12:52           ` [bug#41189] [PATCH v2 2/4] pack: Factorize store references in wrapper Ludovic Courtès
2020-05-13 12:52           ` [bug#41189] [PATCH v2 3/4] gnu: Add fakechroot Ludovic Courtès
2020-05-13 12:52           ` [bug#41189] [PATCH v2 4/4] pack: Add relocation via ld.so and fakechroot Ludovic Courtès
2020-05-14 15:24           ` bug#41189: [PATCH v2 0/4] Add Fakechroot engine for 'guix pack -RR' Ludovic Courtès

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).