From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id eA6xJ7OHuV4sbgAA0tVLHw (envelope-from ) for ; Mon, 11 May 2020 17:13:23 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id yBdNKMGHuV4XVwAAbx9fmQ (envelope-from ) for ; Mon, 11 May 2020 17:13:37 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 24864941188 for ; Mon, 11 May 2020 17:13:35 +0000 (UTC) Received: from localhost ([::1]:33770 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jYBzj-0006y1-LF for larch@yhetil.org; Mon, 11 May 2020 13:13:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60616) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jYBzC-0006JT-8M for guix-patches@gnu.org; Mon, 11 May 2020 13:13:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:41590) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jYBzB-0001mr-QY for guix-patches@gnu.org; Mon, 11 May 2020 13:13:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jYBzB-0001eh-MQ for guix-patches@gnu.org; Mon, 11 May 2020 13:13:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#41189] [PATCH 3/3] pack: Add relocation via ld.so and fakechroot. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 11 May 2020 17:13:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 41189 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 41189@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 41189-submit@debbugs.gnu.org id=B41189.15892171346308 (code B ref 41189); Mon, 11 May 2020 17:13:01 +0000 Received: (at 41189) by debbugs.gnu.org; 11 May 2020 17:12:14 +0000 Received: from localhost ([127.0.0.1]:53134 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYByB-0001cv-6f for submit@debbugs.gnu.org; Mon, 11 May 2020 13:12:14 -0400 Received: from eggs.gnu.org ([209.51.188.92]:57352) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jYBy4-0001cF-Rj for 41189@debbugs.gnu.org; Mon, 11 May 2020 13:11:57 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:41569) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jYBxz-0001Tt-3c; Mon, 11 May 2020 13:11:47 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=38172 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jYBxy-0005Cj-4S; Mon, 11 May 2020 13:11:46 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Mon, 11 May 2020 19:11:35 +0200 Message-Id: <20200511171135.23157-3-ludo@gnu.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200511171135.23157-1-ludo@gnu.org> References: <20200511171135.23157-1-ludo@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -1.0 (-) X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Scanner: scn0 X-Spam-Score: -0.01 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Scan-Result: default: False [-0.01 / 13.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; GENERIC_REPUTATION(0.00)[-0.54012258284346]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.51.188.0/24:c]; IP_REPUTATION_HAM(0.00)[asn: 22989(0.07), country: US(-0.00), ip: 209.51.188.17(-0.54)]; DWL_DNSWL_FAIL(0.00)[209.51.188.17:server fail]; MX_GOOD(-0.50)[cached: eggs.gnu.org]; RCPT_COUNT_TWO(0.00)[2]; MAILLIST(-0.20)[mailman]; FORGED_RECIPIENTS_MAILLIST(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:22989, ipnet:209.51.188.0/24, country:US]; MIME_TRACE(0.00)[0:+]; TAGGED_FROM(0.00)[larch=yhetil.org]; ARC_NA(0.00)[]; FROM_NEQ_ENVFROM(0.00)[ludo@gnu.org,guix-patches-bounces@gnu.org]; FROM_HAS_DN(0.00)[]; URIBL_BLOCKED(0.00)[inria.fr:email,ld.so:url]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[gnu.org]; HAS_LIST_UNSUB(-0.01)[]; DNSWL_BLOCKED(0.00)[209.51.188.17:from]; MID_CONTAINS_FROM(1.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.51.188.17:from]; RCVD_COUNT_SEVEN(0.00)[9]; FORGED_SENDER_MAILLIST(0.00)[] X-TUID: 6v4ub9AvU+vT From: Ludovic Courtès * gnu/packages/aux-files/run-in-namespace.c (HAVE_EXEC_WITH_LOADER): New macro. (bind_mount): Rename to... (mirror_directory): ... this. Add 'firmlink' argument and use it instead of calling mkdir/open/close/mount directly. (bind_mount, make_symlink): New functions. (exec_in_user_namespace): Adjust accordingly. (exec_with_loader) [HAVE_EXEC_WITH_LOADER]: New function. (exec_performance): New function. (engines): Add them. * guix/scripts/pack.scm (wrapped-package)[fakechroot-library]: New procedures. [build](elf-interpreter, elf-loader-compile-flags): New procedures. (build-wrapper): Use them. * tests/guix-pack-relocatable.sh: Test with 'GUIX_EXECUTION_ENGINE=fakechroot'. * doc/guix.texi (Invoking guix pack): Document the 'performance' and 'fakechroot' engines. --- doc/guix.texi | 13 ++ gnu/packages/aux-files/run-in-namespace.c | 174 ++++++++++++++++++++-- guix/scripts/pack.scm | 65 +++++++- tests/guix-pack-relocatable.sh | 6 + 4 files changed, 237 insertions(+), 21 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 958ed9ceec..a70a058afb 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -5228,6 +5228,10 @@ following execution engines are supported: Try user namespaces and fall back to PRoot if user namespaces are not supported (see below). +@item performance +Try user namespaces and fall back to Fakechroot if user namespaces are +not supported (see below). + @item userns Run the program through user namespaces and abort if they are not supported. @@ -5239,6 +5243,15 @@ support for file system virtualization. It achieves that by using the @code{ptrace} system call on the running program. This approach has the advantage to work without requiring special kernel support, but it incurs run-time overhead every time a system call is made. + +@item fakechroot +Run through Fakechroot. @uref{https://github.com/dex4er/fakechroot/, +Fakechroot} virtualizes file system accesses by intercepting calls to C +library functions such as @code{open}, @code{stat}, @code{exec}, and so +on. Unlike PRoot, it incurs very little overhead. However, it does not +always work: for example, some file system accesses made from within the +C library are not intercepted, and file system accesses made @i{via} +direct syscalls are not intercepted either, leading to erratic behavior. @end table @vindex GUIX_EXECUTION_ENGINE diff --git a/gnu/packages/aux-files/run-in-namespace.c b/gnu/packages/aux-files/run-in-namespace.c index 6beac7fd53..ed72a169f2 100644 --- a/gnu/packages/aux-files/run-in-namespace.c +++ b/gnu/packages/aux-files/run-in-namespace.c @@ -42,6 +42,11 @@ #include #include +/* Whether we're building the ld.so/libfakechroot wrapper. */ +#define HAVE_EXEC_WITH_LOADER \ + (defined PROGRAM_INTERPRETER) && (defined PROGRAM_RUNPATH) + + /* Like 'malloc', but abort if 'malloc' returns NULL. */ static void * xmalloc (size_t size) @@ -113,9 +118,42 @@ rm_rf (const char *directory) assert_perror (errno); } -/* Bind mount all the top-level entries in SOURCE to TARGET. */ +/* Make TARGET a bind-mount of SOURCE. Take into account ENTRY's type, which + corresponds to SOURCE. */ +static int +bind_mount (const char *source, const struct dirent *entry, + const char *target) +{ + if (entry->d_type == DT_DIR) + { + int err = mkdir (target, 0700); + if (err != 0) + return err; + } + else + close (open (target, O_WRONLY | O_CREAT)); + + return mount (source, target, "none", + MS_BIND | MS_REC | MS_RDONLY, NULL); +} + +#if HAVE_EXEC_WITH_LOADER + +/* Make TARGET a symlink to SOURCE. */ +static int +make_symlink (const char *source, const struct dirent *entry, + const char *target) +{ + return symlink (source, target); +} + +#endif + +/* Mirror with FIRMLINK all the top-level entries in SOURCE to TARGET. */ static void -bind_mount (const char *source, const char *target) +mirror_directory (const char *source, const char *target, + int (* firmlink) (const char *, const struct dirent *, + const char *)) { DIR *stream = opendir (source); @@ -150,17 +188,7 @@ bind_mount (const char *source, const char *target) else { /* Create the mount point. */ - if (entry->d_type == DT_DIR) - { - int err = mkdir (new_entry, 0700); - if (err != 0) - assert_perror (errno); - } - else - close (open (new_entry, O_WRONLY | O_CREAT)); - - int err = mount (abs_source, new_entry, "none", - MS_BIND | MS_REC | MS_RDONLY, NULL); + int err = firmlink (abs_source, entry, new_entry); /* It used to be that only directories could be bind-mounted. Thus, keep going if we fail to bind-mount a non-directory entry. @@ -244,7 +272,7 @@ exec_in_user_namespace (const char *store, int argc, char *argv[]) /* Note: Due to we cannot make NEW_ROOT a tmpfs (which would have saved the need for 'rm_rf'.) */ - bind_mount ("/", new_root); + mirror_directory ("/", new_root, bind_mount); mkdir_p (new_store); err = mount (store, new_store, "none", MS_BIND | MS_REC | MS_RDONLY, NULL); @@ -336,6 +364,106 @@ exec_with_proot (const char *store, int argc, char *argv[]) #endif + +#if HAVE_EXEC_WITH_LOADER + +static void +exec_with_loader (const char *store, int argc, char *argv[]) +{ + static const char *runpath[] = PROGRAM_RUNPATH; + char *library_path; + size_t size = 0; + + for (size_t i = 0; runpath[i] != NULL; i++) + size += strlen (store) + strlen (runpath[i]) + 1; /* upper bound */ + + library_path = xmalloc (size + 1); + library_path[0] = '\0'; + + for (size_t i = 0; runpath[i] != NULL; i++) + { + if (strncmp (runpath[i], "@STORE_DIRECTORY@", + sizeof "@STORE_DIRECTORY@" - 1) == 0) + { + strcat (library_path, store); + strcat (library_path, runpath[i] + sizeof "@STORE_DIRECTORY@"); + } + else + strcat (library_path, runpath[i]); /* possibly $ORIGIN */ + + strcat (library_path, ":"); + } + + library_path[strlen (library_path) - 1] = '\0'; /* Remove trailing colon. */ + + char *loader = concat (store, + PROGRAM_INTERPRETER + sizeof "@STORE_DIRECTORY@"); + size_t loader_specific_argc = 6; + size_t loader_argc = argc + loader_specific_argc; + char *loader_argv[loader_argc + 1]; + loader_argv[0] = argv[0]; + loader_argv[1] = "--library-path"; + loader_argv[2] = library_path; + loader_argv[3] = "--preload"; + loader_argv[4] = concat (store, + FAKECHROOT_LIBRARY + sizeof "@STORE_DIRECTORY@"); + loader_argv[5] = concat (store, + "@WRAPPED_PROGRAM@" + sizeof "@STORE_DIRECTORY@"); + + for (size_t i = 0; i < argc; i++) + loader_argv[i + loader_specific_argc] = argv[i + 1]; + + loader_argv[loader_argc] = NULL; + + /* Set up the root directory. */ + int err; + char *new_root = mkdtemp (strdup ("/tmp/guix-exec-XXXXXX")); + mirror_directory ("/", new_root, make_symlink); + + char *new_store = concat (new_root, "@STORE_DIRECTORY@"); + char *new_store_parent = dirname (strdup (new_store)); + mkdir_p (new_store_parent); + symlink (store, new_store); + + setenv ("FAKECHROOT_BASE", new_root, 1); + + pid_t child = fork (); + switch (child) + { + case 0: + err = execv (loader, loader_argv); + if (err < 0) + assert_perror (errno); + exit (EXIT_FAILURE); + break; + + case -1: + assert_perror (errno); + exit (EXIT_FAILURE); + break; + + default: + { + int status; + waitpid (child, &status, 0); + chdir ("/"); /* avoid EBUSY */ + rm_rf (new_root); + free (new_root); + + close (2); /* flushing stderr should be silent */ + + if (WIFEXITED (status)) + exit (WEXITSTATUS (status)); + else + /* Abnormal termination cannot really be reproduced, so exit + with 255. */ + exit (255); + } + } +} + +#endif + /* Execution engines. */ @@ -352,7 +480,7 @@ buffer_stderr (void) setvbuf (stderr, stderr_buffer, _IOFBF, sizeof stderr_buffer); } -/* The default engine. */ +/* The default engine: choose a robust method. */ static void exec_default (const char *store, int argc, char *argv[]) { @@ -366,13 +494,29 @@ exec_default (const char *store, int argc, char *argv[]) #endif } +/* The "performance" engine: choose performance over robustness. */ +static void +exec_performance (const char *store, int argc, char *argv[]) +{ + buffer_stderr (); + + exec_in_user_namespace (store, argc, argv); +#if HAVE_EXEC_WITH_LOADER + exec_with_loader (store, argc, argv); +#endif +} + /* List of supported engines. */ static const struct engine engines[] = { { "default", exec_default }, + { "performance", exec_performance }, { "userns", exec_in_user_namespace }, #ifdef PROOT_PROGRAM { "proot", exec_with_proot }, +#endif +#if HAVE_EXEC_WITH_LOADER + { "fakechroot", exec_with_loader }, #endif { NULL, NULL } }; diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm index 580f696b41..3b72496a34 100644 --- a/guix/scripts/pack.scm +++ b/guix/scripts/pack.scm @@ -684,15 +684,26 @@ last resort for relocation." (define (proot) (specification->package "proot-static")) + (define (fakechroot-library) + (file-append (specification->package "fakechroot") + "/lib/fakechroot/libfakechroot.so")) + (define build (with-imported-modules (source-module-closure '((guix build utils) - (guix build union))) + (guix build union) + (guix build gremlin) + (guix elf))) #~(begin (use-modules (guix build utils) ((guix build union) #:select (relative-file-name)) + (guix build gremlin) + (guix elf) + (ice-9 binary-ports) (ice-9 ftw) - (ice-9 match)) + (ice-9 match) + (srfi srfi-1) + (rnrs bytevectors)) (define input ;; The OUTPUT* output of PACKAGE. @@ -711,6 +722,47 @@ last resort for relocation." (#f base) (index (string-drop base index))))) + (define (elf-interpreter elf) + ;; Return the interpreter of ELF as a string, or #f if ELF has no + ;; interpreter segment. + (match (find (lambda (segment) + (= (elf-segment-type segment) PT_INTERP)) + (elf-segments elf)) + (#f #f) ;maybe a .so + (segment + (let ((bv (make-bytevector (- (elf-segment-memsz segment) 1)))) + (bytevector-copy! (elf-bytes elf) + (elf-segment-offset segment) + bv 0 (bytevector-length bv)) + (utf8->string bv))))) + + (define (elf-loader-compile-flags program) + ;; Return the cpp flags defining macros for the ld.so/fakechroot + ;; wrapper of PROGRAM. + + ;; TODO: Handle scripts by wrapping their interpreter. + (if (elf-file? program) + (let* ((bv (call-with-input-file program get-bytevector-all)) + (elf (parse-elf bv))) + (match (elf-dynamic-info elf) + (#f '()) + (dyninfo + (let ((runpath (elf-dynamic-info-runpath dyninfo)) + (interp (elf-interpreter elf))) + (if interp + (list (string-append "-DPROGRAM_INTERPRETER=\"" + interp "\"") + (string-append "-DPROGRAM_RUNPATH={ " + (string-join + (map object->string + runpath) + ", ") + ", NULL }") + (string-append "-DFAKECHROOT_LIBRARY=\"" + #$(fakechroot-library) "\"")) + '()))))) + '())) + (define (build-wrapper program) ;; Build a user-namespace wrapper for PROGRAM. (format #t "building wrapper for '~a'...~%" program) @@ -730,10 +782,11 @@ last resort for relocation." (mkdir-p (dirname result)) (apply invoke #$compiler "-std=gnu99" "-static" "-Os" "-g0" "-Wall" "run.c" "-o" result - (if proot - (list (string-append "-DPROOT_PROGRAM=\"" - proot "\"")) - '())) + (append (if proot + (list (string-append "-DPROOT_PROGRAM=\"" + proot "\"")) + '()) + (elf-loader-compile-flags program))) (delete-file "run.c"))) (setvbuf (current-output-port) 'line) diff --git a/tests/guix-pack-relocatable.sh b/tests/guix-pack-relocatable.sh index cb56815fed..358cac5b26 100644 --- a/tests/guix-pack-relocatable.sh +++ b/tests/guix-pack-relocatable.sh @@ -94,6 +94,12 @@ case "`uname -m`" in export GUIX_EXECUTION_ENGINE "$test_directory/Bin/sed" --version > "$test_directory/output" grep 'GNU sed' "$test_directory/output" + + # Now with fakechroot. + GUIX_EXECUTION_ENGINE="fakechroot" + "$test_directory/Bin/sed" --version > "$test_directory/output" + grep 'GNU sed' "$test_directory/output" + chmod -Rf +w "$test_directory"; rm -rf "$test_directory"/* ;; *) -- 2.26.2