[bug#48434] [PATCH] guile: allow pre-inst-env inject local paths

[bug#48434] [PATCH] guile: allow pre-inst-env inject local paths

[Sergei Trofimovich]

I observed the problem when tried to run 'guix refresh' from local git checkout:

    $ strace -f ./pre-inst-env guix refresh -u re2c |& fgrep re2c.scm
    ...
    [pid 3014757] openat(AT_FDCWD, "/usr/share/guile/site/3.0/gnu/packages/re2c.scm.qilB0R",
        O_RDWR|O_CREAT|O_EXCL, 0600) = -1 EACCES (Permission denied)

Attempt to /usr/share happens because local directory override is ignored:

    $ ./pre-inst-env guile -c '(display (search-path %load-path "gnu/packages/re2c.scm")) (newline) (display %load-path) (newline)'
    /usr/share/guile/site/3.0/gnu/packages/re2c.scm
    (/usr/share/guile/3.0 \
     /usr/share/guile/site/3.0 \
     /usr/share/guile/site \
     /usr/share/guile \
     /home/slyfox/dev/git/guix \
     /home/slyfox/dev/git/guix)

It happens because ./guile ignores GUILE_LOAD_PATH / GUILE_LOAD_COMPILED_PATH
unconditionally.

The change keeps GUILE_LOAD_PATH / GUILE_LOAD_COMPILED_PATH for ./pre-inst-env.

* gnu/packages/aux-files/guile-launcher.c (main): don't ignore
GUILE_LOAD_PATH / GUILE_LOAD_COMPILED_PATH in GUIX_UNINSTALLED=1 mode.

Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>
---
 gnu/packages/aux-files/guile-launcher.c | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)

diff --git a/gnu/packages/aux-files/guile-launcher.c b/gnu/packages/aux-files/guile-launcher.c
index 47ba069de1..bed63353a9 100644
--- a/gnu/packages/aux-files/guile-launcher.c
+++ b/gnu/packages/aux-files/guile-launcher.c
@@ -73,14 +73,19 @@ main (int argc, char **argv)
        which is always preferable over the C locale.  */
     setlocale (LC_ALL, "en_US.utf8");
 
-  const char *str;
-  str = getenv ("GUILE_LOAD_PATH");
-  load_path = str != NULL ? strdup (str) : NULL;
-  str = getenv ("GUILE_LOAD_COMPILED_PATH");
-  load_compiled_path = str ? strdup (str) : NULL;
+  /* Allow ./pre-inst-env to inject local paths. That way local sources
+     are preferred for most operations.  */
+  if (getenv ("GUIX_UNINSTALLED") == NULL)
+    {
+      const char *str;
+      str = getenv ("GUILE_LOAD_PATH");
+      load_path = str != NULL ? strdup (str) : NULL;
+      str = getenv ("GUILE_LOAD_COMPILED_PATH");
+      load_compiled_path = str ? strdup (str) : NULL;
 
-  unsetenv ("GUILE_LOAD_PATH");
-  unsetenv ("GUILE_LOAD_COMPILED_PATH");
+      unsetenv ("GUILE_LOAD_PATH");
+      unsetenv ("GUILE_LOAD_COMPILED_PATH");
+    }
 
   /* XXX: Do not let GMP allocate via libgc as this can lead to memory
      corruption in GnuTLS/Nettle since Nettle also uses GMP:
-- 
2.31.1

[bug#48434] [PATCH] guile: allow pre-inst-env inject local paths

[Sergei Trofimovich]

On Sat, 15 May 2021 10:52:27 +0100
Sergei Trofimovich <slyfox@gentoo.org> wrote:

> I observed the problem when tried to run 'guix refresh' from local git checkout:
> 
>     $ strace -f ./pre-inst-env guix refresh -u re2c |& fgrep re2c.scm
>     ...
>     [pid 3014757] openat(AT_FDCWD, "/usr/share/guile/site/3.0/gnu/packages/re2c.scm.qilB0R",
>         O_RDWR|O_CREAT|O_EXCL, 0600) = -1 EACCES (Permission denied)
> 
> Attempt to /usr/share happens because local directory override is ignored:
> 
>     $ ./pre-inst-env guile -c '(display (search-path %load-path "gnu/packages/re2c.scm")) (newline) (display %load-path) (newline)'
>     /usr/share/guile/site/3.0/gnu/packages/re2c.scm
>     (/usr/share/guile/3.0 \
>      /usr/share/guile/site/3.0 \
>      /usr/share/guile/site \
>      /usr/share/guile \
>      /home/slyfox/dev/git/guix \
>      /home/slyfox/dev/git/guix)
> 
> It happens because ./guile ignores GUILE_LOAD_PATH / GUILE_LOAD_COMPILED_PATH
> unconditionally.
> 
> The change keeps GUILE_LOAD_PATH / GUILE_LOAD_COMPILED_PATH for ./pre-inst-env.
> 
> * gnu/packages/aux-files/guile-launcher.c (main): don't ignore
> GUILE_LOAD_PATH / GUILE_LOAD_COMPILED_PATH in GUIX_UNINSTALLED=1 mode.
> 
> Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>
> ---
>  gnu/packages/aux-files/guile-launcher.c | 19 ++++++++++++-------
>  1 file changed, 12 insertions(+), 7 deletions(-)
> 
> diff --git a/gnu/packages/aux-files/guile-launcher.c b/gnu/packages/aux-files/guile-launcher.c
> index 47ba069de1..bed63353a9 100644
> --- a/gnu/packages/aux-files/guile-launcher.c
> +++ b/gnu/packages/aux-files/guile-launcher.c
> @@ -73,14 +73,19 @@ main (int argc, char **argv)
>         which is always preferable over the C locale.  */
>      setlocale (LC_ALL, "en_US.utf8");
>  
> -  const char *str;
> -  str = getenv ("GUILE_LOAD_PATH");
> -  load_path = str != NULL ? strdup (str) : NULL;
> -  str = getenv ("GUILE_LOAD_COMPILED_PATH");
> -  load_compiled_path = str ? strdup (str) : NULL;
> +  /* Allow ./pre-inst-env to inject local paths. That way local sources
> +     are preferred for most operations.  */
> +  if (getenv ("GUIX_UNINSTALLED") == NULL)
> +    {
> +      const char *str;
> +      str = getenv ("GUILE_LOAD_PATH");
> +      load_path = str != NULL ? strdup (str) : NULL;
> +      str = getenv ("GUILE_LOAD_COMPILED_PATH");
> +      load_compiled_path = str ? strdup (str) : NULL;
>  
> -  unsetenv ("GUILE_LOAD_PATH");
> -  unsetenv ("GUILE_LOAD_COMPILED_PATH");
> +      unsetenv ("GUILE_LOAD_PATH");
> +      unsetenv ("GUILE_LOAD_COMPILED_PATH");
> +    }
>  
>    /* XXX: Do not let GMP allocate via libgc as this can lead to memory
>       corruption in GnuTLS/Nettle since Nettle also uses GMP:

Stumbled on it again today when cloned fresh guix repo and forgot to
apply this patch. Is it a reasonable approach? Or something else is
at fault here?

Thanks!

-- 

  Sergei

[bug#48434] [PATCH] guile: allow pre-inst-env inject local paths

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 2968 bytes --]

Sergei Trofimovich schreef op ma 16-08-2021 om 18:28 [+0100]:
> On Sat, 15 May 2021 10:52:27 +0100
> Sergei Trofimovich <slyfox@gentoo.org> wrote:
> 
> > I observed the problem when tried to run 'guix refresh' from local git checkout:
> > 
> >     $ strace -f ./pre-inst-env guix refresh -u re2c |& fgrep re2c.scm
> >     ...
> >     [pid 3014757] openat(AT_FDCWD, "/usr/share/guile/site/3.0/gnu/packages/re2c.scm.qilB0R",
> >         O_RDWR|O_CREAT|O_EXCL, 0600) = -1 EACCES (Permission denied)
> > 
> > Attempt to /usr/share happens because local directory override is ignored:
> > 
> >     $ ./pre-inst-env guile -c '(display (search-path %load-path "gnu/packages/re2c.scm")) (newline) (display %load-path) (newline)'
> >     /usr/share/guile/site/3.0/gnu/packages/re2c.scm
> >     (/usr/share/guile/3.0 \
> >      /usr/share/guile/site/3.0 \
> >      /usr/share/guile/site \
> >      /usr/share/guile \
> >      /home/slyfox/dev/git/guix \
> >      /home/slyfox/dev/git/guix)

         ^ here's the checkout in the load path

> > 
> > It happens because ./guile ignores GUILE_LOAD_PATH / GUILE_LOAD_COMPILED_PATH
> > unconditionally.

The local directory isn't ignored, it's at the end (the effect is about the same though).

> > The change keeps GUILE_LOAD_PATH / GUILE_LOAD_COMPILED_PATH for ./pre-inst-env.
> > 
> > * gnu/packages/aux-files/guile-launcher.c (main): don't ignore
> > GUILE_LOAD_PATH / GUILE_LOAD_COMPILED_PATH in GUIX_UNINSTALLED=1 mode.

Could you do something like

#define GUIX_UNINSTALLED 1
#if GUIX_UNINSTALLED
new behaviour
#else
OLD BEHAVIOUR
#endif

and change "Makefile.am" to compile two variants of "guile",
one with "DGUIX_UNINSTALLED=1" which goes into libexec, and another
with "DGUIX_UNINSTALLED=0" named "$CHECKOUT/guile" which isn't installed
anywhere but will be added to PATH by "pre-inst-env", or something like that?

I've a bit of an aversion towards using $..._UNINSTALLED environment variables,
as it leads to difficult situations like ‘what should happen if I /gnu/store/.../bin/guix
is run with GUIX_UNINSTALLED set to 1’:

  (a): Reset "GUILE_LOAD{,_COMPILED}_PATH" because we're running directly from the store
  (b): Don't reset "GUILE_LOAD_{,_COMPILED}_PATH" because GUIX_UNINSTALLED is set to 1.

As a comparison, GUIX_UNINSTALLED, the preprocessor variable, is like a ‘lexical variable’,
and GUIX_UNINSTALLED, the environment variable, is like a
‘parameter object’/‘dynamically bound variable’ (see, e.g., 6.11.12 Parameters in the Guile
manual).  I tend to prefer the former, except for dynamic things like LC_ALL, $[...]_PATH
(when used by "guile", "gcc" -- applications typically shouldn't depend on $GUILE_LOAD_PATH,
if they do, they need 'wrap-program' or the like’).

Also, "guile-launcher.c" is used by 'quiet-quile" in (guix self).  It should probably be
verified that "guix pull" still works well.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

[bug#48434] [PATCH] guile: allow pre-inst-env inject local paths

[Sergei Trofimovich]

[-- Attachment #1.1: Type: text/plain, Size: 4207 bytes --]

On Mon, 16 Aug 2021 20:52:48 +0200
Maxime Devos <maximedevos@telenet.be> wrote:

> Sergei Trofimovich schreef op ma 16-08-2021 om 18:28 [+0100]:
> > On Sat, 15 May 2021 10:52:27 +0100
> > Sergei Trofimovich <slyfox@gentoo.org> wrote:
> >   
> > > I observed the problem when tried to run 'guix refresh' from local git checkout:
> > > 
> > >     $ strace -f ./pre-inst-env guix refresh -u re2c |& fgrep re2c.scm
> > >     ...
> > >     [pid 3014757] openat(AT_FDCWD, "/usr/share/guile/site/3.0/gnu/packages/re2c.scm.qilB0R",
> > >         O_RDWR|O_CREAT|O_EXCL, 0600) = -1 EACCES (Permission denied)
> > > 
> > > Attempt to /usr/share happens because local directory override is ignored:

> The local directory isn't ignored, it's at the end (the effect is about the same though).

Reworded to: "because local directory override has too low priority".

> > > The change keeps GUILE_LOAD_PATH / GUILE_LOAD_COMPILED_PATH for ./pre-inst-env.
> > > 
> > > * gnu/packages/aux-files/guile-launcher.c (main): don't ignore
> > > GUILE_LOAD_PATH / GUILE_LOAD_COMPILED_PATH in GUIX_UNINSTALLED=1 mode.  
> 
> Could you do something like
> 
> #define GUIX_UNINSTALLED 1
> #if GUIX_UNINSTALLED
> new behaviour
> #else
> OLD BEHAVIOUR
> #endif
> 
> and change "Makefile.am" to compile two variants of "guile",
> one with "DGUIX_UNINSTALLED=1" which goes into libexec, and another
> with "DGUIX_UNINSTALLED=0" named "$CHECKOUT/guile" which isn't installed
> anywhere but will be added to PATH by "pre-inst-env", or something like that?

Attached v2 patch that should solve all the above.

Added two 'guile' flavours:
    inplace/guile (to be used inplace)
    store/guile (to be installed to libexec)
While at it moved 'guix-daemon' to 'inplace/' as well to make it clear that
tests use inplace variant sometimes. Installation location did not change.

Also moved scripts/guix to inplace/guix as ./pre-inst-env relies on it to
be injected to PATH.

> Also, "guile-launcher.c" is used by 'quiet-quile" in (guix self).  It should probably be
> verified that "guix pull" still works well.

It's a bit hard to test right now as guix-master is slightly broken due to
missing installed files when installed as a primary package manager, but at
least fetch part works fine:

$ guix pull
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
Authenticating channel 'guix', commits 9edb3f6 to f7094f5 (29 new commits)...
Building from this channel:
  guix      https://git.savannah.gnu.org/git/guix.git   f7094f5
Backtrace:
In ice-9/boot-9.scm:
   222:29 19 (map1 (((guix store)) ((guix records)) ((guix #)) (#) …))
   222:29 18 (map1 (((guix records)) ((guix profiles)) ((guix #)) # …))
   222:29 17 (map1 (((guix profiles)) ((guix discovery)) ((guix …)) …))
   222:29 16 (map1 (((guix discovery)) ((guix combinators)) ((# …)) …))
   222:29 15 (map1 (((guix combinators)) ((guix channels)) ((# #)) …))
   222:29 14 (map1 (((guix channels)) ((guix describe)) ((guix #)) …))
   222:29 13 (map1 (((guix describe)) ((guix sets)) ((guix ui)) (#) …))
   222:29 12 (map1 (((guix sets)) ((guix ui)) ((guix diagnostics)) …))
   222:29 11 (map1 (((guix ui)) ((guix diagnostics)) ((guix #)) (#) …))
   222:29 10 (map1 (((guix diagnostics)) ((guix modules)) ((# #)) # …))
   222:29  9 (map1 (((guix modules)) ((guix packages)) ((guix #)) # …))
   222:29  8 (map1 (((guix packages)) ((guix utils)) ((gnu # #)) # …))
   222:29  7 (map1 (((guix utils)) ((gnu packages base)) ((gnu …)) …))
   222:29  6 (map1 (((gnu packages base)) ((gnu packages bash)) (#) …))
   222:29  5 (map1 (((gnu packages bash)) ((gnu packages hurd)) (#) …))
   222:29  4 (map1 (((gnu packages hurd)) ((gnu system setuid)) (#) …))
   222:17  3 (map1 (((gnu system setuid)) ((srfi srfi-1)) ((# #)) # …))
   3329:6  2 (resolve-interface (gnu system setuid) #:select _ #:hide …)
  1685:16  1 (raise-exception _ #:continuable? _)
  1685:16  0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1685:16: In procedure raise-exception:
no code for module (gnu system setuid)

Thanks!

-- 

  Sergei

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: v2-0001-guile-allow-pre-inst-env-inject-local-paths.patch --]
[-- Type: text/x-patch, Size: 11517 bytes --]

From 20fea816b93f31f6b079aa752c0a2ec681649cd0 Mon Sep 17 00:00:00 2001
From: Sergei Trofimovich <slyich@gmail.com>
Date: Sat, 15 May 2021 10:40:12 +0100
Subject: [PATCH v2] guile: allow pre-inst-env inject local paths

I observed the problem when tried to run 'guix refresh' from local git checkout:

    $ strace -f ./pre-inst-env guix refresh -u re2c |& fgrep re2c.scm
    ...
    [pid 3014757] openat(AT_FDCWD, "/usr/share/guile/site/3.0/gnu/packages/re2c.scm.qilB0R",
        O_RDWR|O_CREAT|O_EXCL, 0600) = -1 EACCES (Permission denied)

Attempt to /usr/share happens because local directory override has too low priority:

    $ ./pre-inst-env guile -c '(display (search-path %load-path "gnu/packages/re2c.scm")) (newline) (display %load-path) (newline)'
    /usr/share/guile/site/3.0/gnu/packages/re2c.scm
    (/usr/share/guile/3.0 \
     /usr/share/guile/site/3.0 \
     /usr/share/guile/site \
     /usr/share/guile \
     /home/slyfox/dev/git/guix \
     /home/slyfox/dev/git/guix)

It happens because ./guile ignores GUILE_LOAD_PATH / GUILE_LOAD_COMPILED_PATH
unconditionally.

The change drops 'GUILE_LOAD_PATH' environment variable handling. Instead
we bake the constant with into 'guile' binary by distinguishing 'store/guile'
and 'inplace/guile'. And we bake '%guix-is-inplace' in 'inplace/guix' and
in '/usr/bin/guix'. While at it also move 'guix-daemon' to
'inplace/guix-daemon' to make it clear it is used locally by tests.

	* .gitignore: Ignore new 'store' and 'inplace' directories, ignore
	two new 'guile-launcher' object files.
	* Makefile.am: Move 'scripts/guix' to 'inplace/guix'. Move './guile'
	to 'inplace/guile'. Add 'store/guile' that differes from 'inplace/guile'
	by absence of '-DGUIX_INPLACE_GUILE=1' define.
	* build-aux/pre-inst-env.in: Don't add '$abs_top_builddir' to the
	'PATH'. Add only '$abs_top_builddir/inplace' to the 'PATH'. Drop
	'GUIX_UNINSTALLED' environment variable.
	* build-aux/test-env.in: Update to new 'inplace/guix-daemon' location.
	* doc/local.mk: Update to new 'inplace/guix-daemon' location.
	* gnu/packages/aux-files/guile-launcher.c (main): Use compile-time
	* nix/local.mk: Move 'guix-daemon' to 'inplace/guix-daemon'.
	'GUIX_INPLACE_GUILE' define to distinguish 'guile' binary used inplace
	and used from store.
	* scripts/guix.in: Drop 'GUIX_UNINSTALLED' environment variable
	handling in favour of new '%guix-is-inplace' constant.
---
 .gitignore                              |  5 ++++-
 Makefile.am                             | 29 +++++++++++++++++--------
 build-aux/pre-inst-env.in               | 12 ++++------
 build-aux/test-env.in                   |  4 ++--
 doc/local.mk                            |  2 +-
 gnu/packages/aux-files/guile-launcher.c | 26 ++++++++++++++++------
 nix/local.mk                            | 14 ++++++------
 scripts/guix.in                         |  6 +++--
 8 files changed, 61 insertions(+), 37 deletions(-)

diff --git a/.gitignore b/.gitignore
index 88fe24586d..5a7dfa1f31 100644
--- a/.gitignore
+++ b/.gitignore
@@ -78,6 +78,7 @@
 /etc/openrc/guix-daemon
 /guix-*
 /guix/config.scm
+/inplace
 /libformat.a
 /libstore.a
 /libutil.a
@@ -132,6 +133,7 @@
 /pre-inst-env
 /release-*
 /scripts/guix
+/store
 /test-env
 /test-tmp
 /tests/*.trs
@@ -152,5 +154,6 @@ tmp
 /.version
 /doc/stamp-*
 /gnu/packages/bootstrap
-/gnu/packages/aux-files/guile-guile-launcher.o
+/gnu/packages/aux-files/inplace_guile-guile-launcher.o
+/gnu/packages/aux-files/store_guile-guile-launcher.o
 /guile
diff --git a/Makefile.am b/Makefile.am
index 5542aa1c56..0666c7b7c2 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -33,33 +33,44 @@
 
 MSGMERGE_UPDATE = @MSGMERGE@ --update
 
-bin_SCRIPTS = scripts/guix
+bin_SCRIPTS = inplace/guix
 
 # Handle substitution of fully-expanded Autoconf variables.
 do_subst = $(SED)					\
   -e 's,[@]GUILE[@],$(GUILE),g'				\
   -e 's,[@]guilemoduledir[@],$(guilemoduledir),g'	\
   -e 's,[@]guileobjectdir[@],$(guileobjectdir),g'	\
+  -e 's,[@]guix_is_inplace[@],t,g'			\
   -e 's,[@]abs_top_builddir[@],$(abs_top_builddir),g'	\
   -e 's,[@]localedir[@],$(localedir),g'
 
-scripts/guix: scripts/guix.in Makefile
+inplace/guix: scripts/guix.in Makefile
 	$(AM_V_at)rm -f $@ $@-t
 	$(AM_V_at)$(MKDIR_P) "$(@D)"
-	$(AM_V_GEN)$(do_subst) < "$(srcdir)/$@.in" > "$@-t"
+	$(AM_V_GEN)$(do_subst) < "$(srcdir)/$<" > "$@-t"
 	$(AM_V_at)chmod a+x,a-w "$@-t" && mv -f "$@-t" "$@"
 
 # This is our variant of the 'guile' executable, one that doesn't complain
-# about locales.
-pkglibexec_PROGRAMS = guile
-guile_SOURCES = gnu/packages/aux-files/guile-launcher.c
-guile_LDADD   = $(GUILE_LIBS)
-guile_CFLAGS  = $(GUILE_CFLAGS)
+# about locales. Intended to be executed from store.
+pkglibexec_PROGRAMS = store/guile
+store_guile_SOURCES = gnu/packages/aux-files/guile-launcher.c
+store_guile_LDADD   = $(GUILE_LIBS)
+store_guile_CFLAGS  = $(GUILE_CFLAGS)
+
+# This is similar to pkglibexec_PROGRAMS with an exception of honoring local
+# modules:
+# - GUILE_LOAD_PATH / GUILE_LOAD_COMPILED_PATH to allow './pre-inst-env guix refresh'
+#   and friends access to inplace sources: http://issues.guix.gnu.org/48434.
+noinst_PROGRAMS = inplace/guile
+inplace_guile_SOURCES = gnu/packages/aux-files/guile-launcher.c
+inplace_guile_LDADD   = $(GUILE_LIBS)
+inplace_guile_CFLAGS  = $(GUILE_CFLAGS) -DGUIX_INPLACE_GUILE=1
 
 # Have the 'guix' command refer to our 'guile'.
 install-exec-hook:
 	$(SED) -i "$(DESTDIR)$(bindir)/guix"				\
-	       -e 's,^#![[:graph:]]\+,#!$(pkglibexecdir)/guile,g'
+	       -e 's,^#![[:graph:]]\+,#!$(pkglibexecdir)/guile,g'	\
+	       -e 's,define %guix-is-inplace #t,define %guix-is-inplace #f,g'
 
 nodist_noinst_SCRIPTS =				\
   pre-inst-env					\
diff --git a/build-aux/pre-inst-env.in b/build-aux/pre-inst-env.in
index cd90a06cbc..d4e03a92fb 100644
--- a/build-aux/pre-inst-env.in
+++ b/build-aux/pre-inst-env.in
@@ -39,17 +39,13 @@ export GUILE_LOAD_COMPILED_PATH GUILE_LOAD_PATH
 
 # Define $PATH so that `guix' and friends are easily found.
 
-PATH="$abs_top_builddir/scripts:$abs_top_builddir:$PATH"
+PATH="$abs_top_builddir/inplace:$PATH"
 export PATH
 
 # The daemon invokes 'guix'; tell it which one to use.
-GUIX="$abs_top_builddir/scripts/guix"
+# Use inpalce `guix' to honor local GUILE_LOAD_PATH / GUILE_LOAD_COMPILED_PATH:
+# http://issues.guix.gnu.org/48434
+GUIX="$abs_top_builddir/inplace/guix"
 export GUIX
 
-# Define $GUIX_UNINSTALLED to prevent `guix' from
-# prepending @guilemoduledir@ to the Guile load paths.
-
-GUIX_UNINSTALLED=1
-export GUIX_UNINSTALLED
-
 exec "$@"
diff --git a/build-aux/test-env.in b/build-aux/test-env.in
index 7efc43206c..b41662393d 100644
--- a/build-aux/test-env.in
+++ b/build-aux/test-env.in
@@ -39,7 +39,7 @@ case "$1" in
 	;;
 esac
 
-if [ -x "@abs_top_builddir@/guix-daemon" ]
+if [ -x "@abs_top_builddir@/inplace/guix-daemon" ]
 then
     NIX_STORE_DIR="@GUIX_TEST_ROOT@/store"
 
@@ -100,7 +100,7 @@ then
     # Launch the daemon without chroot support because is may be
     # unavailable, for instance if we're not running as root.
     "@abs_top_builddir@/pre-inst-env"				\
-	"@abs_top_builddir@/guix-daemon" --disable-chroot	\
+	"@abs_top_builddir@/inplace/guix-daemon" --disable-chroot	\
 	--substitute-urls="$GUIX_BINARY_SUBSTITUTE_URL" &
 
     daemon_pid=$!
diff --git a/doc/local.mk b/doc/local.mk
index 8340b75a87..6068a2cd2b 100644
--- a/doc/local.mk
+++ b/doc/local.mk
@@ -251,7 +251,7 @@ if !CROSS_COMPILING
 
 dist_man1_MANS += $(srcdir)/%D%/guix-daemon.1
 
-$(srcdir)/%D%/guix-daemon.1: guix-daemon$(EXEEXT)
+$(srcdir)/%D%/guix-daemon.1: inplace/guix-daemon$(EXEEXT)
 	-$(AM_V_HELP2MAN)$(gen_man) --output="$@" `basename "$@" .1`
 
 endif
diff --git a/gnu/packages/aux-files/guile-launcher.c b/gnu/packages/aux-files/guile-launcher.c
index 47ba069de1..3dbd738573 100644
--- a/gnu/packages/aux-files/guile-launcher.c
+++ b/gnu/packages/aux-files/guile-launcher.c
@@ -66,6 +66,12 @@ inner_main (void *unused, int argc, char **argv)
 int
 main (int argc, char **argv)
 {
+#if defined(GUIX_INPLACE_GUILE)
+  int run_guile_from_store = 0;
+#else
+  int run_guile_from_store = 1;
+#endif
+
   /* Try to install the current locale; remain silent if it fails.  */
   if (setlocale (LC_ALL, "") == NULL)
     /* The 'guix pull'-provided 'guix' includes at least en_US.utf8 so use
@@ -73,14 +79,20 @@ main (int argc, char **argv)
        which is always preferable over the C locale.  */
     setlocale (LC_ALL, "en_US.utf8");
 
-  const char *str;
-  str = getenv ("GUILE_LOAD_PATH");
-  load_path = str != NULL ? strdup (str) : NULL;
-  str = getenv ("GUILE_LOAD_COMPILED_PATH");
-  load_compiled_path = str ? strdup (str) : NULL;
 
-  unsetenv ("GUILE_LOAD_PATH");
-  unsetenv ("GUILE_LOAD_COMPILED_PATH");
+  /* Allow ./pre-inst-env to inject local paths. That way local sources
+     are preferred for most operations.  */
+  if (run_guile_from_store)
+    {
+      const char *str;
+      str = getenv ("GUILE_LOAD_PATH");
+      load_path = str != NULL ? strdup (str) : NULL;
+      str = getenv ("GUILE_LOAD_COMPILED_PATH");
+      load_compiled_path = str ? strdup (str) : NULL;
+
+      unsetenv ("GUILE_LOAD_PATH");
+      unsetenv ("GUILE_LOAD_COMPILED_PATH");
+    }
 
   /* XXX: Do not let GMP allocate via libgc as this can lead to memory
      corruption in GnuTLS/Nettle since Nettle also uses GMP:
diff --git a/nix/local.mk b/nix/local.mk
index 7c438ea78c..1d7705b1e5 100644
--- a/nix/local.mk
+++ b/nix/local.mk
@@ -111,33 +111,33 @@ libstore_a_CPPFLAGS =				\
 libstore_a_CXXFLAGS = $(AM_CXXFLAGS)		\
   $(SQLITE3_CFLAGS) $(LIBGCRYPT_CFLAGS)
 
-bin_PROGRAMS = guix-daemon
+bin_PROGRAMS = inplace/guix-daemon
 
-guix_daemon_SOURCES =				\
+inplace_guix_daemon_SOURCES =			\
   %D%/nix-daemon/nix-daemon.cc			\
   %D%/nix-daemon/guix-daemon.cc
 
-guix_daemon_CPPFLAGS =				\
+inplace_guix_daemon_CPPFLAGS =			\
   -DLOCALEDIR=\"$(localedir)\"			\
   $(libutil_a_CPPFLAGS)				\
   -I$(top_srcdir)/%D%/libstore
 
-guix_daemon_LDADD =				\
+inplace_guix_daemon_LDADD =			\
   libstore.a libutil.a libformat.a -lz		\
   $(SQLITE3_LIBS) $(LIBGCRYPT_LIBS)
 
-guix_daemon_headers =				\
+inplace_guix_daemon_headers =			\
   %D%/nix-daemon/shared.hh
 
 if HAVE_LIBBZ2
 
-guix_daemon_LDADD += -lbz2
+inplace_guix_daemon_LDADD += -lbz2
 
 endif HAVE_LIBBZ2
 
 noinst_HEADERS =						\
   $(libformat_headers) $(libutil_headers) $(libstore_headers)	\
-  $(guix_daemon_headers)
+  $(inplace_guix_daemon_headers)
 
 %D%/libstore/schema.sql.hh: guix/store/schema.sql
 	$(AM_V_GEN)$(GUILE) --no-auto-compile -c		\
diff --git a/scripts/guix.in b/scripts/guix.in
index e0194d6ea2..a9fc29baa4 100644
--- a/scripts/guix.in
+++ b/scripts/guix.in
@@ -1,4 +1,4 @@
-#!@abs_top_builddir@/guile \
+#!@abs_top_builddir@/inplace/guile \
 --no-auto-compile -e main -s
 !#
 ;;; GNU Guix --- Functional package management for GNU
@@ -31,8 +31,10 @@
   (push! "@guilemoduledir@" %load-path)
   (push! "@guileobjectdir@" %load-compiled-path))
 
+(define %guix-is-inplace #@guix_is_inplace@)
+
 (define* (main #:optional (args (command-line)))
-  (unless (getenv "GUIX_UNINSTALLED")
+  (unless %guix-is-inplace
     (augment-load-paths!))
 
   (let ((guix-main (module-ref (resolve-interface '(guix ui))
-- 
2.32.0


[-- Attachment #2: Цифровая подпись OpenPGP --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

[bug#48434] [PATCH] guile: allow pre-inst-env inject local paths

[Sergei Trofimovich]

[-- Attachment #1: Type: text/plain, Size: 4606 bytes --]

On Tue, 17 Aug 2021 09:28:30 +0100
Sergei Trofimovich <slyich@gmail.com> wrote:

> On Mon, 16 Aug 2021 20:52:48 +0200
> Maxime Devos <maximedevos@telenet.be> wrote:
> 
> > Sergei Trofimovich schreef op ma 16-08-2021 om 18:28 [+0100]:  
> > > On Sat, 15 May 2021 10:52:27 +0100
> > > Sergei Trofimovich <slyfox@gentoo.org> wrote:
> > >     
> > > > I observed the problem when tried to run 'guix refresh' from local git checkout:
> > > > 
> > > >     $ strace -f ./pre-inst-env guix refresh -u re2c |& fgrep re2c.scm
> > > >     ...
> > > >     [pid 3014757] openat(AT_FDCWD, "/usr/share/guile/site/3.0/gnu/packages/re2c.scm.qilB0R",
> > > >         O_RDWR|O_CREAT|O_EXCL, 0600) = -1 EACCES (Permission denied)
> > > > 
> > > > Attempt to /usr/share happens because local directory override is ignored:  
> 
> > The local directory isn't ignored, it's at the end (the effect is about the same though).  
> 
> Reworded to: "because local directory override has too low priority".
> 
> > > > The change keeps GUILE_LOAD_PATH / GUILE_LOAD_COMPILED_PATH for ./pre-inst-env.
> > > > 
> > > > * gnu/packages/aux-files/guile-launcher.c (main): don't ignore
> > > > GUILE_LOAD_PATH / GUILE_LOAD_COMPILED_PATH in GUIX_UNINSTALLED=1 mode.    
> > 
> > Could you do something like
> > 
> > #define GUIX_UNINSTALLED 1
> > #if GUIX_UNINSTALLED
> > new behaviour
> > #else
> > OLD BEHAVIOUR
> > #endif
> > 
> > and change "Makefile.am" to compile two variants of "guile",
> > one with "DGUIX_UNINSTALLED=1" which goes into libexec, and another
> > with "DGUIX_UNINSTALLED=0" named "$CHECKOUT/guile" which isn't installed
> > anywhere but will be added to PATH by "pre-inst-env", or something like that?  
> 
> Attached v2 patch that should solve all the above.
> 
> Added two 'guile' flavours:
>     inplace/guile (to be used inplace)
>     store/guile (to be installed to libexec)
> While at it moved 'guix-daemon' to 'inplace/' as well to make it clear that
> tests use inplace variant sometimes. Installation location did not change.
> 
> Also moved scripts/guix to inplace/guix as ./pre-inst-env relies on it to
> be injected to PATH.
> 
> > Also, "guile-launcher.c" is used by 'quiet-quile" in (guix self).  It should probably be
> > verified that "guix pull" still works well.  
> 
> It's a bit hard to test right now as guix-master is slightly broken due to
> missing installed files when installed as a primary package manager, but at
> least fetch part works fine:
> 
> $ guix pull
> Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
> Authenticating channel 'guix', commits 9edb3f6 to f7094f5 (29 new commits)...
> Building from this channel:
>   guix      https://git.savannah.gnu.org/git/guix.git   f7094f5
> Backtrace:
> In ice-9/boot-9.scm:
>    222:29 19 (map1 (((guix store)) ((guix records)) ((guix #)) (#) …))
>    222:29 18 (map1 (((guix records)) ((guix profiles)) ((guix #)) # …))
>    222:29 17 (map1 (((guix profiles)) ((guix discovery)) ((guix …)) …))
>    222:29 16 (map1 (((guix discovery)) ((guix combinators)) ((# …)) …))
>    222:29 15 (map1 (((guix combinators)) ((guix channels)) ((# #)) …))
>    222:29 14 (map1 (((guix channels)) ((guix describe)) ((guix #)) …))
>    222:29 13 (map1 (((guix describe)) ((guix sets)) ((guix ui)) (#) …))
>    222:29 12 (map1 (((guix sets)) ((guix ui)) ((guix diagnostics)) …))
>    222:29 11 (map1 (((guix ui)) ((guix diagnostics)) ((guix #)) (#) …))
>    222:29 10 (map1 (((guix diagnostics)) ((guix modules)) ((# #)) # …))
>    222:29  9 (map1 (((guix modules)) ((guix packages)) ((guix #)) # …))
>    222:29  8 (map1 (((guix packages)) ((guix utils)) ((gnu # #)) # …))
>    222:29  7 (map1 (((guix utils)) ((gnu packages base)) ((gnu …)) …))
>    222:29  6 (map1 (((gnu packages base)) ((gnu packages bash)) (#) …))
>    222:29  5 (map1 (((gnu packages bash)) ((gnu packages hurd)) (#) …))
>    222:29  4 (map1 (((gnu packages hurd)) ((gnu system setuid)) (#) …))
>    222:17  3 (map1 (((gnu system setuid)) ((srfi srfi-1)) ((# #)) # …))
>    3329:6  2 (resolve-interface (gnu system setuid) #:select _ #:hide …)
>   1685:16  1 (raise-exception _ #:continuable? _)
>   1685:16  0 (raise-exception _ #:continuable? _)
> 
> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
> no code for module (gnu system setuid)

With https://debbugs.gnu.org/cgi/bugreport.cgi?bug=50090 applied 'guix pull'
works successfully on a multi-user foreign distribution.

-- 

  Sergei

[-- Attachment #2: Цифровая подпись OpenPGP --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

[bug#48434] [PATCH] guile: allow pre-inst-env inject local paths

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 976 bytes --]

Sergei Trofimovich schreef op di 17-08-2021 om 09:28 [+0100]:
> It's a bit hard to test right now as guix-master is slightly broken due to
> missing installed files when installed as a primary package manager, but at
> least fetch part works fine:
> 
> $ guix pull
> Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
> Authenticating channel 'guix', commits 9edb3f6 to f7094f5 (29 new commits)...
> Building from this channel:
>   guix      https://git.savannah.gnu.org/git/guix.git   f7094f5

I meant something like "$ guix pull --url=. --commit=... --root=guix0"
or "$ ./pre-inst-env guix pull --url=. --commit=... --root=guix1" or
"$ ./pre-inst-env guix pull --root=guix2".  Simply running "$
guix pull",
outside "./pre-inst-env", would simply test the (unpatched) guix you have
installed (with "make install" or "apt-get install guix" or "guix pull"),
and doesn't test the patched guix.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

[bug#48434] [PATCH] guile: allow pre-inst-env inject local paths

[Sergei Trofimovich]

[-- Attachment #1: Type: text/plain, Size: 1558 bytes --]

On Tue, 17 Aug 2021 12:24:50 +0200
Maxime Devos <maximedevos@telenet.be> wrote:

> Sergei Trofimovich schreef op di 17-08-2021 om 09:28 [+0100]:
> > It's a bit hard to test right now as guix-master is slightly broken due to
> > missing installed files when installed as a primary package manager, but at
> > least fetch part works fine:
> > 
> > $ guix pull
> > Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
> > Authenticating channel 'guix', commits 9edb3f6 to f7094f5 (29 new commits)...
> > Building from this channel:
> >   guix      https://git.savannah.gnu.org/git/guix.git   f7094f5  
> 
> I meant something like "$ guix pull --url=. --commit=... --root=guix0"
> or "$ ./pre-inst-env guix pull --url=. --commit=... --root=guix1" or
> "$ ./pre-inst-env guix pull --root=guix2".  Simply running "$
> guix pull",
> outside "./pre-inst-env", would simply test the (unpatched) guix you have
> installed (with "make install" or "apt-get install guix" or "guix pull"),
> and doesn't test the patched guix.

Ah. I did install patched guix into the system (and that's how
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=50090).

I still don't quite understand if channel we pull from also should be patched.
Let's assume it should:

  $ LANG=C ./pre-inst-env guix pull --url=. --commit=e9029ed5eae45ee8f53f055ee66f9bce353cac84 --root=guix0
  guix pull: error: root=guix0: unrecognized option

I'm not sure what --root= does. Is it a user name? Or a profile name?

-- 

  Sergei

[-- Attachment #2: Цифровая подпись OpenPGP --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

[bug#48434] [PATCH] guile: allow pre-inst-env inject local paths

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 778 bytes --]

> Ah. I did install patched guix into the system (and that's how
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=50090).
> 
> I still don't quite understand if channel we pull from also should be patched.
> Let's assume it should:
> 
>   $ LANG=C ./pre-inst-env guix pull --url=. --commit=e9029ed5eae45ee8f53f055ee66f9bce353cac84 --root=guix0
>   guix pull: error: root=guix0: unrecognized option
> 
> I'm not sure what --root= does. Is it a user name? Or a profile name?

I forgot the exact option, it should have been "--profile=guix0".
From (guix)Submitting patches:

  15. Make sure your changes do not break Guix and simulate a ‘guix
     pull’ with:
          guix pull --url=/path/to/your/checkout --profile=/tmp/guix.master

Greetings,
Maxime.


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

[bug#48434] [PATCH] guile: allow pre-inst-env inject local paths

[Sergei Trofimovich]

[-- Attachment #1: Type: text/plain, Size: 1137 bytes --]

On Tue, 17 Aug 2021 15:59:05 +0200
Maxime Devos <maximedevos@telenet.be> wrote:

> > Ah. I did install patched guix into the system (and that's how
> > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=50090).
> > 
> > I still don't quite understand if channel we pull from also should be patched.
> > Let's assume it should:
> > 
> >   $ LANG=C ./pre-inst-env guix pull --url=. --commit=e9029ed5eae45ee8f53f055ee66f9bce353cac84 --root=guix0
> >   guix pull: error: root=guix0: unrecognized option
> > 
> > I'm not sure what --root= does. Is it a user name? Or a profile name?  
> 
> I forgot the exact option, it should have been "--profile=guix0".
> From (guix)Submitting patches:
> 
>   15. Make sure your changes do not break Guix and simulate a ‘guix
>      pull’ with:
>           guix pull --url=/path/to/your/checkout --profile=/tmp/guix.master
> 

Aha, thank you! That works:

$                guix pull --commit=... --url=${PWD} --profile=/tmp/g1.master --disable-authentication
$ ./pre-inst-env guix pull --commit=... --url=${PWD} --profile=/tmp/g2.master --disable-authentication

-- 

  Sergei

[-- Attachment #2: Цифровая подпись OpenPGP --]
[-- Type: application/pgp-signature, Size: 981 bytes --]