[bug#53447] [PATCH] doc: Unset environment variables considered harmful

[bug#53447] [PATCH] doc: Unset environment variables considered harmful

[Liliana Marie Prikler]

* doc/guix.texi (Application Setup): Add subsection for implicit environment
variables.
---
 doc/guix.texi | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/doc/guix.texi b/doc/guix.texi
index 912a8e3c5a..805e3b611f 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -2023,6 +2023,33 @@ want to avoid auto-loading the Emacs packages installed with Guix, you
 can do so by running Emacs with the @option{--no-site-file} option
 (@pxref{Init File,,, emacs, The GNU Emacs Manual}).
 
+@subsection Implicit Environment Variables
+
+Many environment variables are defined in a way that assumes a certain
+default value, even if unset.  For example, bash uses the current
+working directory as @env{PATH} if unset, or @env{XDG_CONFIG_HOME}
+expands to @file{$HOME/.config}.  Some of these defaults remain the same
+whether a package is used through Guix or not---however, environment
+variables referring to @file{/etc} or @file{/usr} typically have their
+meaning subtly changed in Guix.  An application installed via Guix might
+instead look up files in its own @file{etc} structure, or (if a
+search-path was specified) even override the default for packages from
+the foreign distro.
+
+In order to avoid problems coming from such implicitly defined
+environment variables, it is best to explicitly define them somewhere.
+Startup files like @file{~/.bash_profile} are usually a good location
+to do so.  However, not all setups treat them equally.  While on Guix
+System, the user's login shell is used to spawn a graphical session,
+other distros may choose a different approach.  For instance, systemd
+is capable of spawning graphical applications in a clean environment,
+but allows said environments to be modified through an
+@file{override.conf} placed in the right directory.
+
+Regardless of which method is used to set such environment variables,
+they typically should only be set once at login.  Placing them in files
+like @file{~/.bashrc} or similar has adverse effects on other commands
+that want to create clean environments, such as @command{guix shell}.
 
 @node Upgrading Guix
 @section Upgrading Guix
-- 
2.34.0

[bug#53447] [PATCH] doc: Unset environment variables considered harmful

[Ludovic Courtès]

Hi Liliana,

Liliana Marie Prikler <liliana.prikler@gmail.com> skribis:

> * doc/guix.texi (Application Setup): Add subsection for implicit environment
> variables.
> ---
>  doc/guix.texi | 27 +++++++++++++++++++++++++++
>  1 file changed, 27 insertions(+)
>
> diff --git a/doc/guix.texi b/doc/guix.texi
> index 912a8e3c5a..805e3b611f 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -2023,6 +2023,33 @@ want to avoid auto-loading the Emacs packages installed with Guix, you
>  can do so by running Emacs with the @option{--no-site-file} option
>  (@pxref{Init File,,, emacs, The GNU Emacs Manual}).
>  
> +@subsection Implicit Environment Variables
> +
> +Many environment variables are defined in a way that assumes a certain
> +default value, even if unset.  For example, bash uses the current
> +working directory as @env{PATH} if unset, or @env{XDG_CONFIG_HOME}
> +expands to @file{$HOME/.config}.  Some of these defaults remain the same
> +whether a package is used through Guix or not---however, environment
> +variables referring to @file{/etc} or @file{/usr} typically have their
> +meaning subtly changed in Guix.  An application installed via Guix might
> +instead look up files in its own @file{etc} structure, or (if a
> +search-path was specified) even override the default for packages from
> +the foreign distro.

I think I miss some context: what concrete problem is this trying to
solve?  What is it telling me to do?

I wonder to what extent this is actionable for a user, due to wording
that leaves it up to the reader to figure out how this applies to them:

  “Many environment variables”
  “a certain default value”
  “Some of these”
  “An application installed via Guix might”
  “problems coming from such implicitly defined”
  …

I think the “Application Setup” section should be as concrete as
possible, with clear instructions (“If X then type Y”), possibly
followed by explanations that curious readers can read and that others
can skip.

WDYT?

Ludo’.

[bug#53447] [PATCH] doc: Unset environment variables considered harmful

[Liliana Marie Prikler]

Hi Ludo,

Am Samstag, dem 22.01.2022 um 17:04 +0100 schrieb Ludovic Courtès:
> Hi Liliana,
> 
> Liliana Marie Prikler <liliana.prikler@gmail.com> skribis:
> 
> > * doc/guix.texi (Application Setup): Add subsection for implicit
> > environment variables.
> > ---
> >  doc/guix.texi | 27 +++++++++++++++++++++++++++
> >  1 file changed, 27 insertions(+)
> > 
> > diff --git a/doc/guix.texi b/doc/guix.texi
> > index 912a8e3c5a..805e3b611f 100644
> > --- a/doc/guix.texi
> > +++ b/doc/guix.texi
> > @@ -2023,6 +2023,33 @@ want to avoid auto-loading the Emacs
> > packages installed with Guix, you
> >  can do so by running Emacs with the @option{--no-site-file} option
> >  (@pxref{Init File,,, emacs, The GNU Emacs Manual}).
> >  
> > +@subsection Implicit Environment Variables
> > +
> > +Many environment variables are defined in a way that assumes a
> > certain
> > +default value, even if unset.  For example, bash uses the current
> > +working directory as @env{PATH} if unset, or @env{XDG_CONFIG_HOME}
> > +expands to @file{$HOME/.config}.  Some of these defaults remain
> > the same
> > +whether a package is used through Guix or not---however,
> > environment
> > +variables referring to @file{/etc} or @file{/usr} typically have
> > their
> > +meaning subtly changed in Guix.  An application installed via Guix
> > might
> > +instead look up files in its own @file{etc} structure, or (if a
> > +search-path was specified) even override the default for packages
> > from
> > +the foreign distro.
> 
> I think I miss some context: what concrete problem is this trying to
> solve?  What is it telling me to do?
> 
> I wonder to what extent this is actionable for a user, due to wording
> that leaves it up to the reader to figure out how this applies to
> them:
> 
>   “Many environment variables”
>   “a certain default value”
>   “Some of these”
>   “An application installed via Guix might”
>   “problems coming from such implicitly defined”
>   …
> 
> I think the “Application Setup” section should be as concrete as
> possible, with clear instructions (“If X then type Y”), possibly
> followed by explanations that curious readers can read and that
> others can skip.
> 
> WDYT?
I think there are too many examples to exhaustively list them all, but
to give an example, [1, 2, 3, 4] are all the same bug and in [2] Carlo
said we should document this under "Application Setup".  My personal
stance is that the Guix behaviour is not a bug and other distros are
weird for not explicitly binding it.

I know my wording is not the best here, but "If X then Y" is a little
too late when your session broke.  But if your session broke and you
read the manual saying "blah blah unset environment variables evil",
you are more likely to suspect "hmm, maybe evil environment variables
were evil".  Fortunately, with distros trying out Flatpaks and Snaps,
XDG_DATA_DIRS is less likely to break them, but still.  We never know
which variable will be the next to blow things up.

Cheers

[1] https://issues.guix.gnu.org/26202
[2] https://issues.guix.gnu.org/45360
[3] https://issues.guix.gnu.org/48300
[4] https://issues.guix.gnu.org/53379

[bug#53447] [PATCH] doc: Unset environment variables considered harmful

[Maxim Cournoyer]

Hi Liliana and Ludo,

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

> Hi Ludo,
>
> Am Samstag, dem 22.01.2022 um 17:04 +0100 schrieb Ludovic Courtès:
>> Hi Liliana,
>> 
>> Liliana Marie Prikler <liliana.prikler@gmail.com> skribis:
>> 
>> > * doc/guix.texi (Application Setup): Add subsection for implicit
>> > environment variables.
>> > ---
>> >  doc/guix.texi | 27 +++++++++++++++++++++++++++
>> >  1 file changed, 27 insertions(+)
>> > 
>> > diff --git a/doc/guix.texi b/doc/guix.texi
>> > index 912a8e3c5a..805e3b611f 100644
>> > --- a/doc/guix.texi
>> > +++ b/doc/guix.texi
>> > @@ -2023,6 +2023,33 @@ want to avoid auto-loading the Emacs
>> > packages installed with Guix, you
>> >  can do so by running Emacs with the @option{--no-site-file} option
>> >  (@pxref{Init File,,, emacs, The GNU Emacs Manual}).
>> >  
>> > +@subsection Implicit Environment Variables
>> > +
>> > +Many environment variables are defined in a way that assumes a
>> > certain
>> > +default value, even if unset.  For example, bash uses the current
>> > +working directory as @env{PATH} if unset, or @env{XDG_CONFIG_HOME}
>> > +expands to @file{$HOME/.config}.  Some of these defaults remain
>> > the same
>> > +whether a package is used through Guix or not---however,
>> > environment
>> > +variables referring to @file{/etc} or @file{/usr} typically have
>> > their
>> > +meaning subtly changed in Guix.  An application installed via Guix
>> > might
>> > +instead look up files in its own @file{etc} structure, or (if a
>> > +search-path was specified) even override the default for packages
>> > from
>> > +the foreign distro.
>> 
>> I think I miss some context: what concrete problem is this trying to
>> solve?  What is it telling me to do?
>> 
>> I wonder to what extent this is actionable for a user, due to wording
>> that leaves it up to the reader to figure out how this applies to
>> them:
>> 
>>   “Many environment variables”
>>   “a certain default value”
>>   “Some of these”
>>   “An application installed via Guix might”
>>   “problems coming from such implicitly defined”
>>   …
>> 
>> I think the “Application Setup” section should be as concrete as
>> possible, with clear instructions (“If X then type Y”), possibly
>> followed by explanations that curious readers can read and that
>> others can skip.
>> 
>> WDYT?

I initially thought it unnecessary and vague, but after reading the bug
reports listed below, it seems to make sense to document it.  And
reading it now, that's probably the issue I encountered myself in
https://issues.guix.gnu.org/53233.

> I think there are too many examples to exhaustively list them all, but
> to give an example, [1, 2, 3, 4] are all the same bug and in [2] Carlo
> said we should document this under "Application Setup".  My personal
> stance is that the Guix behaviour is not a bug and other distros are
> weird for not explicitly binding it.

I like to see this as a bug, so I've opened one as 53514; Guix should
strive to not mess with the host environment, and setting global
variables used by both Guix and a potentially foreign host goes against
this.  The proper fix would be to patch all applications in Guix to use
Guix-specific variables, such as GUIX_XDG_DATA_DIRS instead of
XDG_DATA_DIRS.

> I know my wording is not the best here, but "If X then Y" is a little
> too late when your session broke.  But if your session broke and you
> read the manual saying "blah blah unset environment variables evil",
> you are more likely to suspect "hmm, maybe evil environment variables
> were evil".  Fortunately, with distros trying out Flatpaks and Snaps,
> XDG_DATA_DIRS is less likely to break them, but still.  We never know
> which variable will be the next to blow things up.

Agreed that this is useful in the meantime.  Perhaps add a TODO comment
pointing to the bug I've opened.

Thank you,

Maxim

[bug#53447] [PATCH] doc: Unset environment variables considered harmful

[Ludovic Courtès]

Hi,

Liliana Marie Prikler <liliana.prikler@gmail.com> skribis:

> I think there are too many examples to exhaustively list them all, but
> to give an example, [1, 2, 3, 4] are all the same bug and in [2] Carlo
> said we should document this under "Application Setup".  My personal
> stance is that the Guix behaviour is not a bug and other distros are
> weird for not explicitly binding it.

How about focusing specifically on XDG_DATA_DIRS, which is the root
cause of the first two bugs you mention?

We could say that installing certain packages will cause
~/.guix-profile/etc/profile to define XDG_DATA_DIRS, that this can
interfere with software installed outside Guix, such as GNOME, and that
we recommend unsetting XDG_DATA_DIRS (is that what we recommend?
breaking Guix-installed packages instead of breaking distro-installed
packages?)?

Thanks,
Ludo’.

> [1] https://issues.guix.gnu.org/26202
> [2] https://issues.guix.gnu.org/45360
> [3] https://issues.guix.gnu.org/48300
> [4] https://issues.guix.gnu.org/53379

[bug#53447] Introducing ‘GUIX_’-prefixed environment variables

[Ludovic Courtès]

Hi,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> I like to see this as a bug, so I've opened one as 53514; Guix should
> strive to not mess with the host environment, and setting global
> variables used by both Guix and a potentially foreign host goes against
> this.  The proper fix would be to patch all applications in Guix to use
> Guix-specific variables, such as GUIX_XDG_DATA_DIRS instead of
> XDG_DATA_DIRS.

This approach has its appeal (like in the ‘GUIX_PYTHONPATH’ case), but
there’s a tension with our other unwritten (?) guideline that we should
modify packages as little as possible.

Such wide-ranging changes would have the unfortunate effect that they’d
make Guix packages “special”: documentation, bug reports, suggestions
you’d find online would apply to the “real” package, but maybe not to
the Guix one.  That is a situation we’d rather avoid IMO.


‘XDG_DATA_DIRS’ is a real problem though.  In large part that’s because
its purpose is too broad—what “data dirs” are we talking about?  It
would be ideal if we could progressively replace ‘XDG_DATA_DIRS’ search
path specifications with more specific environment variables, when they
exist, or perhaps by using ‘wrap-program’ instead of having search path
specs.

Right now there are 9 packages that have ‘XDG_DATA_DIRS’ as their search
path, including 4 Common Lisp packages.  Of the remaining ones, I guess
those we should first focus on are glib and qtbase.

Thoughts?

Ludo’.

[bug#53447] [PATCH] doc: Unset environment variables considered harmful

[Liliana Marie Prikler]

Hi,

Am Dienstag, dem 25.01.2022 um 08:39 +0100 schrieb Ludovic Courtès:
> We could say that installing certain packages will cause
> ~/.guix-profile/etc/profile to define XDG_DATA_DIRS, that this can
> interfere with software installed outside Guix, such as GNOME, and
> that we recommend unsetting XDG_DATA_DIRS (is that what we recommend?
> breaking Guix-installed packages instead of breaking distro-installed
> packages?)?
XDG_DATA_DIRS can be concatenated just like PATH.  The recommendation
is to explicitly set it before sourcing Guix profiles.  "Unset
environment variables considered harmful" should be read as "It's
harmful not to set these variables", not "those variables are harmful,
don't set them".

As for the GUIX_XDG_DATA_DIRS thing discussed elsewhere, I'd like to
raise the following: do we need GUIX_PATH to not let packages installed
with Guix shadow PATH?

Cheers

[bug#53447] Introducing ‘GUIX_’-prefixed environment variables

[Maxim Cournoyer]

Hi Ludovic,

Ludovic Courtès <ludo@gnu.org> writes:

> Hi,
>
> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> I like to see this as a bug, so I've opened one as 53514; Guix should
>> strive to not mess with the host environment, and setting global
>> variables used by both Guix and a potentially foreign host goes against
>> this.  The proper fix would be to patch all applications in Guix to use
>> Guix-specific variables, such as GUIX_XDG_DATA_DIRS instead of
>> XDG_DATA_DIRS.
>
> This approach has its appeal (like in the ‘GUIX_PYTHONPATH’ case), but
> there’s a tension with our other unwritten (?) guideline that we should
> modify packages as little as possible.
>
> Such wide-ranging changes would have the unfortunate effect that they’d
> make Guix packages “special”: documentation, bug reports, suggestions
> you’d find online would apply to the “real” package, but maybe not to
> the Guix one.  That is a situation we’d rather avoid IMO.

True; but after people would get a hang that in Guix most environment
variables would be prefixed with 'GUIX_', I don't think it'd be that big
of an issue.  Maintaining the patches would be where the burden would
lie, in my opinion.

> ‘XDG_DATA_DIRS’ is a real problem though.  In large part that’s because
> its purpose is too broad—what “data dirs” are we talking about?  It
> would be ideal if we could progressively replace ‘XDG_DATA_DIRS’ search
> path specifications with more specific environment variables, when they
> exist, or perhaps by using ‘wrap-program’ instead of having search path
> specs.

Are there any more specific environment variables that exist that can
replace XDG_DATA_DIRS?  I'm not too knowledgeable about the freedesktop
specs, but I'm somewhat skeptical?  If they don't yet exist, that makes
this idea much less actionable.

Thanks,

Maxim

[bug#53447] Introducing ‘GUIX_’-prefixed environment variables

[Ludovic Courtès]

Hi,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Hi,
>>
>> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>>
>>> I like to see this as a bug, so I've opened one as 53514; Guix should
>>> strive to not mess with the host environment, and setting global
>>> variables used by both Guix and a potentially foreign host goes against
>>> this.  The proper fix would be to patch all applications in Guix to use
>>> Guix-specific variables, such as GUIX_XDG_DATA_DIRS instead of
>>> XDG_DATA_DIRS.
>>
>> This approach has its appeal (like in the ‘GUIX_PYTHONPATH’ case), but
>> there’s a tension with our other unwritten (?) guideline that we should
>> modify packages as little as possible.
>>
>> Such wide-ranging changes would have the unfortunate effect that they’d
>> make Guix packages “special”: documentation, bug reports, suggestions
>> you’d find online would apply to the “real” package, but maybe not to
>> the Guix one.  That is a situation we’d rather avoid IMO.
>
> True; but after people would get a hang that in Guix most environment
> variables would be prefixed with 'GUIX_', I don't think it'd be that big
> of an issue.  Maintaining the patches would be where the burden would
> lie, in my opinion.

There are environment variables everywhere; are you suggesting to
GUIX_-prefix them en masse?

That seems neither viable nor desirable to me.  And, as Liliana put it,
when will we have GUIX_PATH?  :-)

>> ‘XDG_DATA_DIRS’ is a real problem though.  In large part that’s because
>> its purpose is too broad—what “data dirs” are we talking about?  It
>> would be ideal if we could progressively replace ‘XDG_DATA_DIRS’ search
>> path specifications with more specific environment variables, when they
>> exist, or perhaps by using ‘wrap-program’ instead of having search path
>> specs.
>
> Are there any more specific environment variables that exist that can
> replace XDG_DATA_DIRS?  I'm not too knowledgeable about the freedesktop
> specs, but I'm somewhat skeptical?  If they don't yet exist, that makes
> this idea much less actionable.

I don’t know.  Like I wrote, the two main cases are glib and qt.  Why do
we have them use XDG_DATA_DIRS for?  This is what we need to
investigate.

I would be less reluctant to patches that add an extra variable in these
two packages (which could be submitted upstream) than GUIX_-prefixing
it.

Ludo’.

[bug#53447] Introducing ‘GUIX_’-prefixed environment variables

[Liliana Marie Prikler]

Hi,

Am Mittwoch, dem 26.01.2022 um 13:05 +0100 schrieb Ludovic Courtès:
> > Are there any more specific environment variables that exist that can
> > replace XDG_DATA_DIRS?  I'm not too knowledgeable about the
> > freedesktop specs, but I'm somewhat skeptical?  If they don't yet
> > exist, that makes this idea much less actionable.
> 
> I don’t know.  Like I wrote, the two main cases are glib and qt.  Why
> do we have them use XDG_DATA_DIRS for?  This is what we need to
> investigate.
Applications based on GLib or Qt usually put their configuration in
XDG_CONFIG_HOME/XDG_CONFIG_DIRS and their data into
XDG_DATA_HOME/XDG_DATA_DIRS.  Yes, it's that broad, XDG wants it to be
that way :P
There are additional environment variables for some things – one
example would be GSETTINGS_SCHEMA_DIR – but many things are simply put
in those XDG directories.  For example, gio, which is part of GLib,
uses it to look up MIME stuff [1].  Icons and themes follow a similar
trend as far as I can see.

Note that according to the freedesktop specs, these directories are
ordered by preference, so Guix prepending stuff might not always be
what was asked for.  However, I'd also argue that if users wanted their
distro stuff to shadow what Guix has, they could first clear the
variable, then load the Guix stuff and finally prepend the distro
stuff.  And arguably if you spawn a `guix shell' you probably want that
shell to have effect over everything else in your system (or you add a
hack to your rc files to make it not so).  In any case, I'd find it
hard to argue that users have no choice here.  It's their shell, they
can source profiles in any order they wish.

WDYT?

[1] https://developer-old.gnome.org/gio/stable/running-gio-apps.html
[2]
https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html

[bug#53447] Introducing ‘GUIX_’-prefixed environment variables

[Maxim Cournoyer]

Hello,

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

> Hi,
>
> Am Mittwoch, dem 26.01.2022 um 13:05 +0100 schrieb Ludovic Courtès:
>> > Are there any more specific environment variables that exist that can
>> > replace XDG_DATA_DIRS?  I'm not too knowledgeable about the
>> > freedesktop specs, but I'm somewhat skeptical?  If they don't yet
>> > exist, that makes this idea much less actionable.
>> 
>> I don’t know.  Like I wrote, the two main cases are glib and qt.  Why
>> do we have them use XDG_DATA_DIRS for?  This is what we need to
>> investigate.
> Applications based on GLib or Qt usually put their configuration in
> XDG_CONFIG_HOME/XDG_CONFIG_DIRS and their data into
> XDG_DATA_HOME/XDG_DATA_DIRS.  Yes, it's that broad, XDG wants it to be
> that way :P
> There are additional environment variables for some things – one
> example would be GSETTINGS_SCHEMA_DIR – but many things are simply put
> in those XDG directories.  For example, gio, which is part of GLib,
> uses it to look up MIME stuff [1].  Icons and themes follow a similar
> trend as far as I can see.

This is what I perceive too; XDG variables are broad by design, and
there aren't more finer grain alternatives that currently exist.

The main problem to solve in my opinion is to not break the users
environment on foreign distribution.  Plugins for example probably ought
to be searched by GUIX_ prefixed environment variables rather than their
stock versions, else users may find binary incompatible versions to
crash their host-provided application, for example.

I don't really mind how to solve it, but using a prefix seems a
relatively straight forward, bullet proof way to isolate the host from
potentially undesirable effects of installing Guix components.

Imagine if Guix-provided Emacs was to use GUIX_EMACSLOADPATH instead of
EMACSLOADPATH.  This would have the benefit that users could continue
using their host provided Emacs (without having it see potentially
incompatible byte compiled .elc packages) in parallel to the one
installed with Guix.

Thanks,

Maxim