Document /homeless-shelter?

Document /homeless-shelter?

[Matt]

In the IRC, someone asked about an error message:

PermissionError: [Errno 13] Permission denied: '/homeless-shelter'

This error happened when trying to build a Python package, python-libpysal.

The solution was to set HOME to /tmp:

    (arguments
     `(#:phases
       (modify-phases %standard-phases
         (add-before 'check 'fix-home-directory
           (lambda _
             ;; Tests fail with "Permission denied: '/homeless-shelter'".
             (setenv "HOME" "/tmp"))))

Basically, some projects expect a $HOME to be defined and will fail because Guix sets $HOME to /homeless-shelter, which doesn't exist in the build environment.

As I understand it, $HOME is set to /homeless-shelter during build because that's how Nix does/did it and "there’s no home
directory in build environments, and perhaps Eelco Dolstra and others back then found that setting ‘HOME’ to a non-existing directory broke
fewer builds that leaving it unset."

Is my understanding correct?

Is this something that should be documented, and if so, where?

IRC log:
https://logs.guix.gnu.org/guix/2022-01-16.log#003312

Previous Guix discussion: 
https://lists.gnu.org/archive/html/guix-devel/2019-11/msg00311.html

Re: Document /homeless-shelter?

[Leo Famulari]

On Sat, Jan 15, 2022 at 10:19:54PM -0500, Matt wrote:
> As I understand it, $HOME is set to /homeless-shelter during build because that's how Nix does/did it and "there’s no home
> directory in build environments, and perhaps Eelco Dolstra and others back then found that setting ‘HOME’ to a non-existing directory broke
> fewer builds that leaving it unset."

A further rationale is that by using an obviously strange and bogus
value for $HOME, all uses of $HOME in the build container are
highlighted.

Otherwise, if $HOME was unset, build scripts might fall back to
something like $HOME/$USER.

To be clear, the situation is not simply that "there's no home directory
in the build environment". Rather, there must not be a home directory
there. By design, that's a small part of how we ensure that builds are
deterministic.

In all cases, package builds should not be able to depend on the
pathname of a home directory.

> Is this something that should be documented, and if so, where?

It could be documented briefly in the manual section Build Environment
Setup.

Re: Document /homeless-shelter?

[Matt]

[-- Attachment #1: Type: text/plain, Size: 1451 bytes --]


 ---- On Sun, 16 Jan 2022 00:20:49 -0500 Leo Famulari <leo@famulari.name> wrote ----

 > It could be documented briefly in the manual section Build Environment
 > Setup.

How does this look?

From 34fcd075efad1577aa139012c5d2fccf44e10058 Mon Sep 17 00:00:00 2001
From: Matt <matt@excalamus.com>
Date: Sun, 16 Jan 2022 12:30:37 -0500
Subject: [PATCH] Document /homeless-shelter

---
 doc/guix.texi | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/doc/guix.texi b/doc/guix.texi
index 42691570ff..fd48a0dcd2 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -1193,6 +1193,19 @@ environment variables for HTTP and HTTPS downloads it performs, be it
 for fixed-output derivations (@pxref{Derivations}) or for substitutes
 (@pxref{Substitutes}).
 
+The @env{HOME} environment variable is set to @file{/homeless-shelter}
+during the build process.  This ensures builds are determistic and
+highlights all uses of @env{HOME}.  Packages should not depend on the
+pathname of a home directory.  Instead, modify the build phase to set
+@env{HOME} to @file{/tmp}:
+
+@lisp
+(modify-phases %standard-phases
+  (add-before 'check 'fix-home-directory
+    (lambda _
+      (setenv "HOME" "/tmp"))))
+@end lisp
+
 If you are installing Guix as an unprivileged user, it is still possible
 to run @command{guix-daemon} provided you pass @option{--disable-chroot}.
 However, build processes will not be isolated from one another, and not
-- 
2.34.0



[-- Attachment #2: 0001-Document-homeless-shelter.patch --]
[-- Type: application/octet-stream, Size: 1257 bytes --]

From 34fcd075efad1577aa139012c5d2fccf44e10058 Mon Sep 17 00:00:00 2001
From: Matt <matt@excalamus.com>
Date: Sun, 16 Jan 2022 12:30:37 -0500
Subject: [PATCH] Document /homeless-shelter

---
 doc/guix.texi | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/doc/guix.texi b/doc/guix.texi
index 42691570ff..fd48a0dcd2 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -1193,6 +1193,19 @@ environment variables for HTTP and HTTPS downloads it performs, be it
 for fixed-output derivations (@pxref{Derivations}) or for substitutes
 (@pxref{Substitutes}).
 
+The @env{HOME} environment variable is set to @file{/homeless-shelter}
+during the build process.  This ensures builds are determistic and
+highlights all uses of @env{HOME}.  Packages should not depend on the
+pathname of a home directory.  Instead, modify the build phase to set
+@env{HOME} to @file{/tmp}:
+
+@lisp
+(modify-phases %standard-phases
+  (add-before 'check 'fix-home-directory
+    (lambda _
+      (setenv "HOME" "/tmp"))))
+@end lisp
+
 If you are installing Guix as an unprivileged user, it is still possible
 to run @command{guix-daemon} provided you pass @option{--disable-chroot}.
 However, build processes will not be isolated from one another, and not
-- 
2.34.0

Re: Document /homeless-shelter?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 795 bytes --]

Hi,

Matt schreef op zo 16-01-2022 om 12:38 [-0500]:
> +The @env{HOME} environment variable is set to @file{/homeless-shelter}
> +during the build process.  This ensures builds are determistic and
> +highlights all uses of @env{HOME}.  Packages should not depend on the
> +pathname of a home directory.  Instead, modify the build phase to set
> +@env{HOME} to @file{/tmp}:
> +
> +@lisp
> +(modify-phases %standard-phases
> +  (add-before 'check 'fix-home-directory
> +    (lambda _
> +      (setenv "HOME" "/tmp"))))
> +@end lisp

Sometimes ‘hand-written, automake/cmake/... is bloat!’-style Makefiles
install things to $HOME in the 'install' target, maybe you could give
something like that as an example for when setting HOME to /tmp won't
work?

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Document /homeless-shelter?

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 693 bytes --]

On Sun, Jan 16, 2022 at 07:16:11PM +0100, Maxime Devos wrote:
> > +(modify-phases %standard-phases
> > +  (add-before 'check 'fix-home-directory
> > +    (lambda _
> > +      (setenv "HOME" "/tmp"))))
> > +@end lisp
> 
> Sometimes ‘hand-written, automake/cmake/... is bloat!’-style Makefiles
> install things to $HOME in the 'install' target, maybe you could give
> something like that as an example for when setting HOME to /tmp won't
> work?

As I suggested in my reply (concurrent to yours), this use of /tmp is
not something we need to document authoritatively. Rather, packagers
have to make a judgement about how to handle package build scripts that
try using $HOME.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Document /homeless-shelter?

[Leo Famulari]

On Sun, Jan 16, 2022 at 12:38:52PM -0500, Matt wrote:
> Subject: [PATCH] Document /homeless-shelter

I pushed a simpler addition to the manual:

https://git.savannah.gnu.org/cgit/guix.git/tree/doc/guix.texi#n1181

Of course I made a silly typo and so this change takes place over two
commits.

> +The @env{HOME} environment variable is set to @file{/homeless-shelter}
> +during the build process.  This ensures builds are determistic and
> +highlights all uses of @env{HOME}.  Packages should not depend on the
> +pathname of a home directory.  Instead, modify the build phase to set
> +@env{HOME} to @file{/tmp}:
> +
> +@lisp
> +(modify-phases %standard-phases
> +  (add-before 'check 'fix-home-directory
> +    (lambda _
> +      (setenv "HOME" "/tmp"))))
> +@end lisp

This text is basically correct but we have to balance verbosity with
readability.

The important thing was to document /homeless-shelter, so that packagers
understand it comes from Guix, and to explain its rationale.

It's not 100% true that setting HOME=homeless-shelter ensures that
builds are deterministic and highlights all uses of $HOME, although it
does help with those goals.

I don't think we want to document the use of /tmp, or recommend it as an
authoritative workaround.

Rather, it's a common solution, but packagers must seek to understand
how the package build scripts are trying to use $HOME and make a
judgement.

Additionally, I don't think that Build Environment Setup is the right
place to document workarounds.

Re: Document /homeless-shelter?

[Matt]

 ---- On Sun, 16 Jan 2022 13:25:57 -0500 Leo Famulari <leo@famulari.name> wrote ----
 > On Sun, Jan 16, 2022 at 12:38:52PM -0500, Matt wrote:
 > > Subject: [PATCH] Document /homeless-shelter
 > 
 > I pushed a simpler addition to the manual:
 > 
 > https://git.savannah.gnu.org/cgit/guix.git/tree/doc/guix.texi#n1181

That looks great. 

 > > +The @env{HOME} environment variable is set to @file{/homeless-shelter}
 > > +during the build process.  This ensures builds are determistic and
 > > +highlights all uses of @env{HOME}.  Packages should not depend on the
 > > +pathname of a home directory.  Instead, modify the build phase to set
 > > +@env{HOME} to @file{/tmp}:
 > > +
 > > +@lisp
 > > +(modify-phases %standard-phases
 > > +  (add-before 'check 'fix-home-directory
 > > +    (lambda _
 > > +      (setenv "HOME" "/tmp"))))
 > > +@end lisp
 > 
 > This text is basically correct but we have to balance verbosity with
 > readability.
 > 
 > The important thing was to document /homeless-shelter, so that packagers
 > understand it comes from Guix, and to explain its rationale.
 > 
 > It's not 100% true that setting HOME=homeless-shelter ensures that
 > builds are deterministic and highlights all uses of $HOME, although it
 > does help with those goals.
 > 
 > I don't think we want to document the use of /tmp, or recommend it as an
 > authoritative workaround.
 > 
 > Rather, it's a common solution, but packagers must seek to understand
 > how the package build scripts are trying to use $HOME and make a
 > judgement.
 > 
 > Additionally, I don't think that Build Environment Setup is the right
 > place to document workarounds.
 
This all looks sound to me. Thanks for taking the time to work with me on this.