[bug#48434] [PATCH] guile: allow pre-inst-env inject local paths

[Sergei Trofimovich <slyich@gmail.com>]

[-- Attachment #1.1: Type: text/plain, Size: 4207 bytes --]

On Mon, 16 Aug 2021 20:52:48 +0200
Maxime Devos <maximedevos@telenet.be> wrote:

> Sergei Trofimovich schreef op ma 16-08-2021 om 18:28 [+0100]:
> > On Sat, 15 May 2021 10:52:27 +0100
> > Sergei Trofimovich <slyfox@gentoo.org> wrote:
> >   
> > > I observed the problem when tried to run 'guix refresh' from local git checkout:
> > > 
> > >     $ strace -f ./pre-inst-env guix refresh -u re2c |& fgrep re2c.scm
> > >     ...
> > >     [pid 3014757] openat(AT_FDCWD, "/usr/share/guile/site/3.0/gnu/packages/re2c.scm.qilB0R",
> > >         O_RDWR|O_CREAT|O_EXCL, 0600) = -1 EACCES (Permission denied)
> > > 
> > > Attempt to /usr/share happens because local directory override is ignored:

> The local directory isn't ignored, it's at the end (the effect is about the same though).

Reworded to: "because local directory override has too low priority".

> > > The change keeps GUILE_LOAD_PATH / GUILE_LOAD_COMPILED_PATH for ./pre-inst-env.
> > > 
> > > * gnu/packages/aux-files/guile-launcher.c (main): don't ignore
> > > GUILE_LOAD_PATH / GUILE_LOAD_COMPILED_PATH in GUIX_UNINSTALLED=1 mode.  
> 
> Could you do something like
> 
> #define GUIX_UNINSTALLED 1
> #if GUIX_UNINSTALLED
> new behaviour
> #else
> OLD BEHAVIOUR
> #endif
> 
> and change "Makefile.am" to compile two variants of "guile",
> one with "DGUIX_UNINSTALLED=1" which goes into libexec, and another
> with "DGUIX_UNINSTALLED=0" named "$CHECKOUT/guile" which isn't installed
> anywhere but will be added to PATH by "pre-inst-env", or something like that?

Attached v2 patch that should solve all the above.

Added two 'guile' flavours:
    inplace/guile (to be used inplace)
    store/guile (to be installed to libexec)
While at it moved 'guix-daemon' to 'inplace/' as well to make it clear that
tests use inplace variant sometimes. Installation location did not change.

Also moved scripts/guix to inplace/guix as ./pre-inst-env relies on it to
be injected to PATH.

> Also, "guile-launcher.c" is used by 'quiet-quile" in (guix self).  It should probably be
> verified that "guix pull" still works well.

It's a bit hard to test right now as guix-master is slightly broken due to
missing installed files when installed as a primary package manager, but at
least fetch part works fine:

$ guix pull
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
Authenticating channel 'guix', commits 9edb3f6 to f7094f5 (29 new commits)...
Building from this channel:
  guix      https://git.savannah.gnu.org/git/guix.git   f7094f5
Backtrace:
In ice-9/boot-9.scm:
   222:29 19 (map1 (((guix store)) ((guix records)) ((guix #)) (#) …))
   222:29 18 (map1 (((guix records)) ((guix profiles)) ((guix #)) # …))
   222:29 17 (map1 (((guix profiles)) ((guix discovery)) ((guix …)) …))
   222:29 16 (map1 (((guix discovery)) ((guix combinators)) ((# …)) …))
   222:29 15 (map1 (((guix combinators)) ((guix channels)) ((# #)) …))
   222:29 14 (map1 (((guix channels)) ((guix describe)) ((guix #)) …))
   222:29 13 (map1 (((guix describe)) ((guix sets)) ((guix ui)) (#) …))
   222:29 12 (map1 (((guix sets)) ((guix ui)) ((guix diagnostics)) …))
   222:29 11 (map1 (((guix ui)) ((guix diagnostics)) ((guix #)) (#) …))
   222:29 10 (map1 (((guix diagnostics)) ((guix modules)) ((# #)) # …))
   222:29  9 (map1 (((guix modules)) ((guix packages)) ((guix #)) # …))
   222:29  8 (map1 (((guix packages)) ((guix utils)) ((gnu # #)) # …))
   222:29  7 (map1 (((guix utils)) ((gnu packages base)) ((gnu …)) …))
   222:29  6 (map1 (((gnu packages base)) ((gnu packages bash)) (#) …))
   222:29  5 (map1 (((gnu packages bash)) ((gnu packages hurd)) (#) …))
   222:29  4 (map1 (((gnu packages hurd)) ((gnu system setuid)) (#) …))
   222:17  3 (map1 (((gnu system setuid)) ((srfi srfi-1)) ((# #)) # …))
   3329:6  2 (resolve-interface (gnu system setuid) #:select _ #:hide …)
  1685:16  1 (raise-exception _ #:continuable? _)
  1685:16  0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1685:16: In procedure raise-exception:
no code for module (gnu system setuid)

Thanks!

-- 

  Sergei

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: v2-0001-guile-allow-pre-inst-env-inject-local-paths.patch --]
[-- Type: text/x-patch, Size: 11517 bytes --]

From 20fea816b93f31f6b079aa752c0a2ec681649cd0 Mon Sep 17 00:00:00 2001
From: Sergei Trofimovich <slyich@gmail.com>
Date: Sat, 15 May 2021 10:40:12 +0100
Subject: [PATCH v2] guile: allow pre-inst-env inject local paths

I observed the problem when tried to run 'guix refresh' from local git checkout:

    $ strace -f ./pre-inst-env guix refresh -u re2c |& fgrep re2c.scm
    ...
    [pid 3014757] openat(AT_FDCWD, "/usr/share/guile/site/3.0/gnu/packages/re2c.scm.qilB0R",
        O_RDWR|O_CREAT|O_EXCL, 0600) = -1 EACCES (Permission denied)

Attempt to /usr/share happens because local directory override has too low priority:

    $ ./pre-inst-env guile -c '(display (search-path %load-path "gnu/packages/re2c.scm")) (newline) (display %load-path) (newline)'
    /usr/share/guile/site/3.0/gnu/packages/re2c.scm
    (/usr/share/guile/3.0 \
     /usr/share/guile/site/3.0 \
     /usr/share/guile/site \
     /usr/share/guile \
     /home/slyfox/dev/git/guix \
     /home/slyfox/dev/git/guix)

It happens because ./guile ignores GUILE_LOAD_PATH / GUILE_LOAD_COMPILED_PATH
unconditionally.

The change drops 'GUILE_LOAD_PATH' environment variable handling. Instead
we bake the constant with into 'guile' binary by distinguishing 'store/guile'
and 'inplace/guile'. And we bake '%guix-is-inplace' in 'inplace/guix' and
in '/usr/bin/guix'. While at it also move 'guix-daemon' to
'inplace/guix-daemon' to make it clear it is used locally by tests.

	* .gitignore: Ignore new 'store' and 'inplace' directories, ignore
	two new 'guile-launcher' object files.
	* Makefile.am: Move 'scripts/guix' to 'inplace/guix'. Move './guile'
	to 'inplace/guile'. Add 'store/guile' that differes from 'inplace/guile'
	by absence of '-DGUIX_INPLACE_GUILE=1' define.
	* build-aux/pre-inst-env.in: Don't add '$abs_top_builddir' to the
	'PATH'. Add only '$abs_top_builddir/inplace' to the 'PATH'. Drop
	'GUIX_UNINSTALLED' environment variable.
	* build-aux/test-env.in: Update to new 'inplace/guix-daemon' location.
	* doc/local.mk: Update to new 'inplace/guix-daemon' location.
	* gnu/packages/aux-files/guile-launcher.c (main): Use compile-time
	* nix/local.mk: Move 'guix-daemon' to 'inplace/guix-daemon'.
	'GUIX_INPLACE_GUILE' define to distinguish 'guile' binary used inplace
	and used from store.
	* scripts/guix.in: Drop 'GUIX_UNINSTALLED' environment variable
	handling in favour of new '%guix-is-inplace' constant.
---
 .gitignore                              |  5 ++++-
 Makefile.am                             | 29 +++++++++++++++++--------
 build-aux/pre-inst-env.in               | 12 ++++------
 build-aux/test-env.in                   |  4 ++--
 doc/local.mk                            |  2 +-
 gnu/packages/aux-files/guile-launcher.c | 26 ++++++++++++++++------
 nix/local.mk                            | 14 ++++++------
 scripts/guix.in                         |  6 +++--
 8 files changed, 61 insertions(+), 37 deletions(-)

diff --git a/.gitignore b/.gitignore
index 88fe24586d..5a7dfa1f31 100644
--- a/.gitignore
+++ b/.gitignore
@@ -78,6 +78,7 @@
 /etc/openrc/guix-daemon
 /guix-*
 /guix/config.scm
+/inplace
 /libformat.a
 /libstore.a
 /libutil.a
@@ -132,6 +133,7 @@
 /pre-inst-env
 /release-*
 /scripts/guix
+/store
 /test-env
 /test-tmp
 /tests/*.trs
@@ -152,5 +154,6 @@ tmp
 /.version
 /doc/stamp-*
 /gnu/packages/bootstrap
-/gnu/packages/aux-files/guile-guile-launcher.o
+/gnu/packages/aux-files/inplace_guile-guile-launcher.o
+/gnu/packages/aux-files/store_guile-guile-launcher.o
 /guile
diff --git a/Makefile.am b/Makefile.am
index 5542aa1c56..0666c7b7c2 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -33,33 +33,44 @@
 
 MSGMERGE_UPDATE = @MSGMERGE@ --update
 
-bin_SCRIPTS = scripts/guix
+bin_SCRIPTS = inplace/guix
 
 # Handle substitution of fully-expanded Autoconf variables.
 do_subst = $(SED)					\
   -e 's,[@]GUILE[@],$(GUILE),g'				\
   -e 's,[@]guilemoduledir[@],$(guilemoduledir),g'	\
   -e 's,[@]guileobjectdir[@],$(guileobjectdir),g'	\
+  -e 's,[@]guix_is_inplace[@],t,g'			\
   -e 's,[@]abs_top_builddir[@],$(abs_top_builddir),g'	\
   -e 's,[@]localedir[@],$(localedir),g'
 
-scripts/guix: scripts/guix.in Makefile
+inplace/guix: scripts/guix.in Makefile
 	$(AM_V_at)rm -f $@ $@-t
 	$(AM_V_at)$(MKDIR_P) "$(@D)"
-	$(AM_V_GEN)$(do_subst) < "$(srcdir)/$@.in" > "$@-t"
+	$(AM_V_GEN)$(do_subst) < "$(srcdir)/$<" > "$@-t"
 	$(AM_V_at)chmod a+x,a-w "$@-t" && mv -f "$@-t" "$@"
 
 # This is our variant of the 'guile' executable, one that doesn't complain
-# about locales.
-pkglibexec_PROGRAMS = guile
-guile_SOURCES = gnu/packages/aux-files/guile-launcher.c
-guile_LDADD   = $(GUILE_LIBS)
-guile_CFLAGS  = $(GUILE_CFLAGS)
+# about locales. Intended to be executed from store.
+pkglibexec_PROGRAMS = store/guile
+store_guile_SOURCES = gnu/packages/aux-files/guile-launcher.c
+store_guile_LDADD   = $(GUILE_LIBS)
+store_guile_CFLAGS  = $(GUILE_CFLAGS)
+
+# This is similar to pkglibexec_PROGRAMS with an exception of honoring local
+# modules:
+# - GUILE_LOAD_PATH / GUILE_LOAD_COMPILED_PATH to allow './pre-inst-env guix refresh'
+#   and friends access to inplace sources: http://issues.guix.gnu.org/48434.
+noinst_PROGRAMS = inplace/guile
+inplace_guile_SOURCES = gnu/packages/aux-files/guile-launcher.c
+inplace_guile_LDADD   = $(GUILE_LIBS)
+inplace_guile_CFLAGS  = $(GUILE_CFLAGS) -DGUIX_INPLACE_GUILE=1
 
 # Have the 'guix' command refer to our 'guile'.
 install-exec-hook:
 	$(SED) -i "$(DESTDIR)$(bindir)/guix"				\
-	       -e 's,^#![[:graph:]]\+,#!$(pkglibexecdir)/guile,g'
+	       -e 's,^#![[:graph:]]\+,#!$(pkglibexecdir)/guile,g'	\
+	       -e 's,define %guix-is-inplace #t,define %guix-is-inplace #f,g'
 
 nodist_noinst_SCRIPTS =				\
   pre-inst-env					\
diff --git a/build-aux/pre-inst-env.in b/build-aux/pre-inst-env.in
index cd90a06cbc..d4e03a92fb 100644
--- a/build-aux/pre-inst-env.in
+++ b/build-aux/pre-inst-env.in
@@ -39,17 +39,13 @@ export GUILE_LOAD_COMPILED_PATH GUILE_LOAD_PATH
 
 # Define $PATH so that `guix' and friends are easily found.
 
-PATH="$abs_top_builddir/scripts:$abs_top_builddir:$PATH"
+PATH="$abs_top_builddir/inplace:$PATH"
 export PATH
 
 # The daemon invokes 'guix'; tell it which one to use.
-GUIX="$abs_top_builddir/scripts/guix"
+# Use inpalce `guix' to honor local GUILE_LOAD_PATH / GUILE_LOAD_COMPILED_PATH:
+# http://issues.guix.gnu.org/48434
+GUIX="$abs_top_builddir/inplace/guix"
 export GUIX
 
-# Define $GUIX_UNINSTALLED to prevent `guix' from
-# prepending @guilemoduledir@ to the Guile load paths.
-
-GUIX_UNINSTALLED=1
-export GUIX_UNINSTALLED
-
 exec "$@"
diff --git a/build-aux/test-env.in b/build-aux/test-env.in
index 7efc43206c..b41662393d 100644
--- a/build-aux/test-env.in
+++ b/build-aux/test-env.in
@@ -39,7 +39,7 @@ case "$1" in
 	;;
 esac
 
-if [ -x "@abs_top_builddir@/guix-daemon" ]
+if [ -x "@abs_top_builddir@/inplace/guix-daemon" ]
 then
     NIX_STORE_DIR="@GUIX_TEST_ROOT@/store"
 
@@ -100,7 +100,7 @@ then
     # Launch the daemon without chroot support because is may be
     # unavailable, for instance if we're not running as root.
     "@abs_top_builddir@/pre-inst-env"				\
-	"@abs_top_builddir@/guix-daemon" --disable-chroot	\
+	"@abs_top_builddir@/inplace/guix-daemon" --disable-chroot	\
 	--substitute-urls="$GUIX_BINARY_SUBSTITUTE_URL" &
 
     daemon_pid=$!
diff --git a/doc/local.mk b/doc/local.mk
index 8340b75a87..6068a2cd2b 100644
--- a/doc/local.mk
+++ b/doc/local.mk
@@ -251,7 +251,7 @@ if !CROSS_COMPILING
 
 dist_man1_MANS += $(srcdir)/%D%/guix-daemon.1
 
-$(srcdir)/%D%/guix-daemon.1: guix-daemon$(EXEEXT)
+$(srcdir)/%D%/guix-daemon.1: inplace/guix-daemon$(EXEEXT)
 	-$(AM_V_HELP2MAN)$(gen_man) --output="$@" `basename "$@" .1`
 
 endif
diff --git a/gnu/packages/aux-files/guile-launcher.c b/gnu/packages/aux-files/guile-launcher.c
index 47ba069de1..3dbd738573 100644
--- a/gnu/packages/aux-files/guile-launcher.c
+++ b/gnu/packages/aux-files/guile-launcher.c
@@ -66,6 +66,12 @@ inner_main (void *unused, int argc, char **argv)
 int
 main (int argc, char **argv)
 {
+#if defined(GUIX_INPLACE_GUILE)
+  int run_guile_from_store = 0;
+#else
+  int run_guile_from_store = 1;
+#endif
+
   /* Try to install the current locale; remain silent if it fails.  */
   if (setlocale (LC_ALL, "") == NULL)
     /* The 'guix pull'-provided 'guix' includes at least en_US.utf8 so use
@@ -73,14 +79,20 @@ main (int argc, char **argv)
        which is always preferable over the C locale.  */
     setlocale (LC_ALL, "en_US.utf8");
 
-  const char *str;
-  str = getenv ("GUILE_LOAD_PATH");
-  load_path = str != NULL ? strdup (str) : NULL;
-  str = getenv ("GUILE_LOAD_COMPILED_PATH");
-  load_compiled_path = str ? strdup (str) : NULL;
 
-  unsetenv ("GUILE_LOAD_PATH");
-  unsetenv ("GUILE_LOAD_COMPILED_PATH");
+  /* Allow ./pre-inst-env to inject local paths. That way local sources
+     are preferred for most operations.  */
+  if (run_guile_from_store)
+    {
+      const char *str;
+      str = getenv ("GUILE_LOAD_PATH");
+      load_path = str != NULL ? strdup (str) : NULL;
+      str = getenv ("GUILE_LOAD_COMPILED_PATH");
+      load_compiled_path = str ? strdup (str) : NULL;
+
+      unsetenv ("GUILE_LOAD_PATH");
+      unsetenv ("GUILE_LOAD_COMPILED_PATH");
+    }
 
   /* XXX: Do not let GMP allocate via libgc as this can lead to memory
      corruption in GnuTLS/Nettle since Nettle also uses GMP:
diff --git a/nix/local.mk b/nix/local.mk
index 7c438ea78c..1d7705b1e5 100644
--- a/nix/local.mk
+++ b/nix/local.mk
@@ -111,33 +111,33 @@ libstore_a_CPPFLAGS =				\
 libstore_a_CXXFLAGS = $(AM_CXXFLAGS)		\
   $(SQLITE3_CFLAGS) $(LIBGCRYPT_CFLAGS)
 
-bin_PROGRAMS = guix-daemon
+bin_PROGRAMS = inplace/guix-daemon
 
-guix_daemon_SOURCES =				\
+inplace_guix_daemon_SOURCES =			\
   %D%/nix-daemon/nix-daemon.cc			\
   %D%/nix-daemon/guix-daemon.cc
 
-guix_daemon_CPPFLAGS =				\
+inplace_guix_daemon_CPPFLAGS =			\
   -DLOCALEDIR=\"$(localedir)\"			\
   $(libutil_a_CPPFLAGS)				\
   -I$(top_srcdir)/%D%/libstore
 
-guix_daemon_LDADD =				\
+inplace_guix_daemon_LDADD =			\
   libstore.a libutil.a libformat.a -lz		\
   $(SQLITE3_LIBS) $(LIBGCRYPT_LIBS)
 
-guix_daemon_headers =				\
+inplace_guix_daemon_headers =			\
   %D%/nix-daemon/shared.hh
 
 if HAVE_LIBBZ2
 
-guix_daemon_LDADD += -lbz2
+inplace_guix_daemon_LDADD += -lbz2
 
 endif HAVE_LIBBZ2
 
 noinst_HEADERS =						\
   $(libformat_headers) $(libutil_headers) $(libstore_headers)	\
-  $(guix_daemon_headers)
+  $(inplace_guix_daemon_headers)
 
 %D%/libstore/schema.sql.hh: guix/store/schema.sql
 	$(AM_V_GEN)$(GUILE) --no-auto-compile -c		\
diff --git a/scripts/guix.in b/scripts/guix.in
index e0194d6ea2..a9fc29baa4 100644
--- a/scripts/guix.in
+++ b/scripts/guix.in
@@ -1,4 +1,4 @@
-#!@abs_top_builddir@/guile \
+#!@abs_top_builddir@/inplace/guile \
 --no-auto-compile -e main -s
 !#
 ;;; GNU Guix --- Functional package management for GNU
@@ -31,8 +31,10 @@
   (push! "@guilemoduledir@" %load-path)
   (push! "@guileobjectdir@" %load-compiled-path))
 
+(define %guix-is-inplace #@guix_is_inplace@)
+
 (define* (main #:optional (args (command-line)))
-  (unless (getenv "GUIX_UNINSTALLED")
+  (unless %guix-is-inplace
     (augment-load-paths!))
 
   (let ((guix-main (module-ref (resolve-interface '(guix ui))
-- 
2.32.0


[-- Attachment #2: Цифровая подпись OpenPGP --]
[-- Type: application/pgp-signature, Size: 981 bytes --]