bug#23118: Duplicate entries in various environment variables

bug#23118: Duplicate entries in various environment variables

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 9664 bytes --]

Hi,

I've noticed that my environment variables contain duplicate (sometimes
more) entries.  This occurs regardless of whether the user logs in
directly via a tty or via a desktop environment like GNOME.

This behavior should be reproducible.  I created a new user to test
this.  When I logged into GNOME on GuixSD (built via "guix system
reconfigure" using git commit 43a58615bfc37a053ddaca1fcf11af5c14c0b4b6)
as the new user, I opened up the "Terminal" program (which launched bash
as an interactive, non-login shell).  I then typed

--8<---------------cut here---------------start------------->8---
env > /tmp/myenv
--8<---------------cut here---------------end--------------->8---

Then, I analyzed the results.  First, I defined a little helper function
to do the analysis.  For each environment variable, it counts the number
of times each colon-delimited entry occurs in that environment variable
and prints them out:

--8<---------------cut here---------------start------------->8---
count_env_var_occurrences () 
{ 
    for environment_variable in $(cat $1 | cut -d = -f 1 | sort);
    do
        echo $environment_variable;
        cat $1 | grep ^${environment_variable}= | cut -d = -f 2 | tr : '\n' | sort | uniq -c;
    done
}
--8<---------------cut here---------------end--------------->8---

Then I ran it, and got the following results:

--8<---------------cut here---------------start------------->8---
$ count_env_var_occurrences /tmp/myenv 
_
      1 /run/current-system/profile/bin/env
DBUS_FATAL_WARNINGS
      1 0
DBUS_SESSION_BUS_ADDRESS
      1 abstract
      1 unix
DISPLAY
      1 
      1 0.0
GIO_EXTRA_MODULES
      9 /gnu/store/lbkdykh3aa1469li01rwyya02q50r3b9-glib-2.46.1/lib/gio/modules
      1 /gnu/store/ygn4vavcgr79p86jrlmvdycjs81ki4dl-dconf-0.24.0/lib/gio/modules
      1 /run/current-system/profile/lib/gio/modules
GIT_SSL_CAINFO
      1 /etc/ssl/certs/ca-certificates.crt
GI_TYPELIB_PATH
      1 /gnu/store/1sp33d4j5gx2xd64mncc30q0x01hjbf4-cogl-1.22.0/lib/girepository-1.0
      1 /gnu/store/38ir3s81qxhbpa8v8rwf8cbjdcmsr2q0-atk-2.18.0/lib/girepository-1.0
      1 /gnu/store/3k8lhk39979kbi3m26fa191wq5jgxf7f-polkit-0.113/lib/girepository-1.0
      1 /gnu/store/5l1fab8vi6zpiy8m61aiv2qbr8gfsjha-json-glib-1.0.4/lib/girepository-1.0
      1 /gnu/store/67a1962vx46b5shf3myxi23jvziwaclj-gdm-3.18.2/lib/girepository-1.0
      1 /gnu/store/73wzvqzffyx867r1rbck5w65by6lbapy-gnome-bluetooth-3.18.1/lib/girepository-1.0
      1 /gnu/store/apf78hww8abnhsg0pr6d00gc5w1g7y5h-libgweather-3.18.1/lib/girepository-1.0
      1 /gnu/store/b2qdk8kxcb6ls2hdbzgv51smljk2bqh3-clutter-1.24.2/lib/girepository-1.0
      1 /gnu/store/bxb7i5qiml1vlxbsks4fxnv4vv5pzac3-gnome-desktop-3.18.1/lib/girepository-1.0
      1 /gnu/store/cr289j8247p0mv39j26l4gxbdf6b7g11-network-manager-1.0.10/lib/girepository-1.0
      1 /gnu/store/dzdqr0yh7csf3j4ash6akzxjdbqgfln2-network-manager-applet-1.0.10/lib/girepository-1.0
      1 /gnu/store/fqp33841n42qn6qx3ma1w9xyq329abyh-gst-plugins-base-1.6.3/lib/girepository-1.0
      1 /gnu/store/g7ii95bmy2aq4z5x17wvsqlz4f8lmn5a-gcr-3.18.0/lib/girepository-1.0
      1 /gnu/store/h4p0b9qp1v1pxnmlgdpcy53xxddjzajp-gobject-introspection-1.46.0/lib/girepository-1.0
      1 /gnu/store/hkl1hwx7cqgb23115rnacin4lywy39fq-libxklavier-5.3/lib/girepository-1.0
      1 /gnu/store/l2qcfja7vspdki8birvm80xs82acm2va-gdk-pixbuf-2.32.3/lib/girepository-1.0
      1 /gnu/store/m67pffz37qvkr63a7d5i92z10cwznsy9-accountsservice-0.6.40/lib/girepository-1.0
      1 /gnu/store/mfdgb0i95bf4j89zc4mnnbd6hp3v08yx-at-spi2-core-2.18.1/lib/girepository-1.0
      1 /gnu/store/p79p86324sb5yc80m21nk7q8cjsixb3s-libgee-0.18.0/lib/girepository-1.0
      1 /gnu/store/pc9w7sz63d0mims6hbxnzc47g3jy55id-pango-1.38.1/lib/girepository-1.0
      1 /gnu/store/pil657vzrprd2bfdhg7q8vhd3f6jwzpb-libsoup-2.52.1/lib/girepository-1.0
      1 /gnu/store/rdmyj3ylsfbw57kpdgi9iz15s7xfzxzr-libsecret-0.18.3/lib/girepository-1.0
      1 /gnu/store/rjhhm5ixkmgqpxfiik6k1rncwrvdz0z1-upower-0.99.3/lib/girepository-1.0
      1 /gnu/store/rsywwl626s8fmmwf154sn2x9si62b4k1-evolution-data-server-3.18.2/lib/girepository-1.0
      1 /gnu/store/s1nig50yjs5g6fnbl29dyi0d6cp1vza9-telepathy-glib-0.24.1/lib/girepository-1.0
      1 /gnu/store/sm1i06dhgpkhvgj2bhcn6nfi275slgjz-gsettings-desktop-schemas-3.18.0/lib/girepository-1.0
      1 /gnu/store/sqd3q1xq5fsbga00bwhghi9shi7xdaac-gtk+-3.18.2/lib/girepository-1.0
      1 /gnu/store/vmh2w8bsh5w3arrrkcvb468j98nvbjqf-gstreamer-1.6.3/lib/girepository-1.0
      1 /gnu/store/ymx1l8l2gm1jmljh5y24p4f5gk8i2j3c-caribou-0.4.19/lib/girepository-1.0
      1 /gnu/store/z2dh08qp3k1829dcsz7wcrinccmj6743-ibus-1.5.11/lib/girepository-1.0
      1 /gnu/store/zlq812qd3cykkizdip0l7qw7m2pffj1h-telepathy-logger-0.8.2/lib/girepository-1.0
      1 /gnu/store/zvi3k12q95i992gnvhddqs0iildyz4lp-harfbuzz-1.0.6/lib/girepository-1.0
GJS_DEBUG_OUTPUT
      1 stderr
GJS_DEBUG_TOPICS
      1 JS ERROR;JS LOG
GNOME_DESKTOP_SESSION_ID
      1 this-is-deprecated
GST_PLUGIN_PATH
      1 /home/alice/.guix-profile/lib/gstreamer-1.0
GTK_PATH
      3 /gnu/store/7hdyj0dkzp3dbl878g042g300yycasrx-libcanberra-0.30/lib/gtk-3.0
      9 /gnu/store/sqd3q1xq5fsbga00bwhghi9shi7xdaac-gtk+-3.18.2/lib/gtk-3.0
      3 /gnu/store/ymx1l8l2gm1jmljh5y24p4f5gk8i2j3c-caribou-0.4.19/lib/gtk-3.0
      3 /gnu/store/z2dh08qp3k1829dcsz7wcrinccmj6743-ibus-1.5.11/lib/gtk-3.0
GUILE_LOAD_COMPILED_PATH
      1 /run/current-system/profile/lib/guile/2.0/ccache
      1 /run/current-system/profile/share/guile/site/2.0
GUILE_LOAD_PATH
      1 /run/current-system/profile/share/guile/site/2.0
GUIX_GTK2_PATH
      1 /run/current-system/profile/lib/gtk-2.0
GUIX_GTK3_PATH
      1 /run/current-system/profile/lib/gtk-3.0
HOME
      1 /home/alice
INFOPATH
      1 /home/alice/.guix-profile/share/info
      2 /run/current-system/profile/share/info
LANG
      1 en_US.UTF-8
LINUX_MODULE_DIRECTORY
      1 /run/booted-system/kernel/lib/modules
LOGNAME
      1 alice
MAIL
      1 /var/mail/alice
MANPATH
      1 /home/alice/.guix-profile/share/man
      2 /run/current-system/profile/share/man
PATH
      1 /gnu/store/a9sqlwcl02z89kh6v5n6yzxvxw5qbs14-glib-2.46.1-bin/bin
      1 /home/alice/.guix-profile/bin
      1 /run/current-system/profile/bin
      1 /run/current-system/profile/sbin
      1 /run/setuid-programs
PS1
      1 \u@\h \w\$ 
PWD
      1 /home/alice
SESSION_MANAGER
      1 local/carbuncle
      1 /tmp/.ICE-unix/383
      1 @/tmp/.ICE-unix/383,unix/carbuncle
SHELL
      1 /gnu/store/7cdd8s466qyjh64m0byq0rz9gk1jid40-bash-4.3.42/bin/bash
SHLVL
      1 2
SSH_AUTH_SOCK
      1 /run/user/30011/keyring/ssh
SSL_CERT_DIR
      1 /etc/ssl/certs
SSL_CERT_FILE
      1 /etc/ssl/certs/ca-certificates.crt
TERM
      1 xterm-256color
TZ
      1 America/Los_Angeles
TZDIR
      1 /gnu/store/n80pk0smpzdlshkcgccgga8xy5p2vwz3-tzdata-2015g/share/zoneinfo
USER
      1 alice
VTE_VERSION
      1 4203
WINDOWID
      1 23068678
XAUTHORITY
      1 /home/alice/.Xauthority
XDG_CONFIG_DIRS
      1 /home/alice/.guix-profile/etc/xdg
      1 /run/current-system/profile/etc/xdg
XDG_CURRENT_DESKTOP
      1 GNOME
XDG_DATA_DIRS
      3 /gnu/store/27c82f3szfn2d97ms56hkpsb78q9myhd-gnome-settings-daemon-3.18.2/share
      3 /gnu/store/67a1962vx46b5shf3myxi23jvziwaclj-gdm-3.18.2/share
      3 /gnu/store/73wzvqzffyx867r1rbck5w65by6lbapy-gnome-bluetooth-3.18.1/share
      3 /gnu/store/apf78hww8abnhsg0pr6d00gc5w1g7y5h-libgweather-3.18.1/share
      5 /gnu/store/bahndy6sxfxw6g49djc10mpa7292na40-gnome-session-3.18.1.2/share
      3 /gnu/store/dzdqr0yh7csf3j4ash6akzxjdbqgfln2-network-manager-applet-1.0.10/share
      3 /gnu/store/g7ii95bmy2aq4z5x17wvsqlz4f8lmn5a-gcr-3.18.0/share
      3 /gnu/store/kc8j8zcwjzr04nlkm16gy44a7vfr26i8-mutter-3.18.1/share
      3 /gnu/store/km25kxpv999ihj058xnvniwycl1zhajl-gnome-control-center-3.18.2/share
      9 /gnu/store/lbkdykh3aa1469li01rwyya02q50r3b9-glib-2.46.1/share
      3 /gnu/store/rsywwl626s8fmmwf154sn2x9si62b4k1-evolution-data-server-3.18.2/share
      9 /gnu/store/sm1i06dhgpkhvgj2bhcn6nfi275slgjz-gsettings-desktop-schemas-3.18.0/share
      9 /gnu/store/sqd3q1xq5fsbga00bwhghi9shi7xdaac-gtk+-3.18.2/share
      1 /gnu/store/x2hz7q806ry8lmwdya0jlff01w7djbn2-gnome-terminal-3.18.2/share
      3 /gnu/store/xnkkvlwvwbv2zzvbca3sclf41m6jxzza-gnome-shell-3.18.3/share
      3 /gnu/store/ymx1l8l2gm1jmljh5y24p4f5gk8i2j3c-caribou-0.4.19/share
      3 /gnu/store/z2dh08qp3k1829dcsz7wcrinccmj6743-ibus-1.5.11/share
      3 /gnu/store/zlq812qd3cykkizdip0l7qw7m2pffj1h-telepathy-logger-0.8.2/share
      1 /home/alice/.guix-profile/share
      2 /run/current-system/profile/share
XDG_MENU_PREFIX
      1 gnome-
XDG_RUNTIME_DIR
      1 /run/user/30011
XDG_SEAT
      1 seat0
XDG_SESSION_ID
      1 c1
XDG_VTNR
      1 7
X_XFCE4_LIB_DIRS
      1 /run/current-system/profile/lib/xfce4
--8<---------------cut here---------------end--------------->8---

As you can see, there are some environment variables with up to 9
duplicate entries.  Is this expected?  Is it a problem?  Why is it
happening?  Assuming that it is not expected and that it is a problem,
how can we prevent it from happening?

I've tried opening multiple subshells (e.g., by running "bash" on the
command line) to see if the number of duplicate entries increases with
each subshell, but that is not the case.  Opening up a new subshell does
not seem to trigger (in the default GuixSD configuration with GNOME, at
least) whatever it is that it causing this duplication.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

bug#23118: Duplicate entries in various environment variables

[Ludovic Courtès]

Chris Marusich <cmmarusich@gmail.com> skribis:

> This behavior should be reproducible.  I created a new user to test
> this.  When I logged into GNOME on GuixSD (built via "guix system
> reconfigure" using git commit 43a58615bfc37a053ddaca1fcf11af5c14c0b4b6)
> as the new user, I opened up the "Terminal" program (which launched bash
> as an interactive, non-login shell).  I then typed

[...]

I think the main problem is that gnome-terminal is wrapped:

--8<---------------cut here---------------start------------->8---
$ cat /gnu/store/x2hz7q806ry8lmwdya0jlff01w7djbn2-gnome-terminal-3.18.2/bin/gnome-terminal 
#!/gnu/store/b1yqjimbdh5bf9jnizd4h7yf110744j2-bash-4.3.42/bin/bash
export XDG_DATA_DIRS="/gnu/store/lbkdykh3aa1469li01rwyya02q50r3b9-glib-2.46.1/share:/gnu/store/sm1i06dhgpkhvgj2bhcn6nfi275slgjz-gsettings-desktop-schemas-3.18.0/share:/gnu/store/sqd3q1xq5fsbga00bwhghi9shi7xdaac-gtk+-3.18.2/share:/gnu/store/x2hz7q806ry8lmwdya0jlff01w7djbn2-gnome-terminal-3.18.2/share${XDG_DATA_DIRS:+:}$XDG_DATA_DIRS"
export GTK_PATH="/gnu/store/sqd3q1xq5fsbga00bwhghi9shi7xdaac-gtk+-3.18.2/lib/gtk-3.0${GTK_PATH:+:}$GTK_PATH"
export GIO_EXTRA_MODULES="/gnu/store/lbkdykh3aa1469li01rwyya02q50r3b9-glib-2.46.1/lib/gio/modules:/gnu/store/ygn4vavcgr79p86jrlmvdycjs81ki4dl-dconf-0.24.0/lib/gio/modules${GIO_EXTRA_MODULES:+:}$GIO_EXTRA_MODULES"
exec -a "$0" "/gnu/store/x2hz7q806ry8lmwdya0jlff01w7djbn2-gnome-terminal-3.18.2/bin/.gnome-terminal-real" "$@"
--8<---------------cut here---------------end--------------->8---

This is done automatically by ‘glib-or-gtk-build-system’, but obviously
it’s not so great here.

I suppose that in practice, it would work to simply switch
gnome-terminal to ‘gnu-build-system’.

Could you try that?

There may still be other places that add extra entries, though.

Ludo’.

bug#23118: Duplicate entries in various environment variables

[Leo Famulari]

On Sat, Mar 26, 2016 at 07:53:09PM +0100, Ludovic Courtès wrote:
> Chris Marusich <cmmarusich@gmail.com> skribis:
> 
> > This behavior should be reproducible.  I created a new user to test
> > this.  When I logged into GNOME on GuixSD (built via "guix system
> > reconfigure" using git commit 43a58615bfc37a053ddaca1fcf11af5c14c0b4b6)
> > as the new user, I opened up the "Terminal" program (which launched bash
> > as an interactive, non-login shell).  I then typed
> 
> [...]
> 
> I think the main problem is that gnome-terminal is wrapped:
> 
> --8<---------------cut here---------------start------------->8---
> $ cat /gnu/store/x2hz7q806ry8lmwdya0jlff01w7djbn2-gnome-terminal-3.18.2/bin/gnome-terminal 
> #!/gnu/store/b1yqjimbdh5bf9jnizd4h7yf110744j2-bash-4.3.42/bin/bash
> export XDG_DATA_DIRS="/gnu/store/lbkdykh3aa1469li01rwyya02q50r3b9-glib-2.46.1/share:/gnu/store/sm1i06dhgpkhvgj2bhcn6nfi275slgjz-gsettings-desktop-schemas-3.18.0/share:/gnu/store/sqd3q1xq5fsbga00bwhghi9shi7xdaac-gtk+-3.18.2/share:/gnu/store/x2hz7q806ry8lmwdya0jlff01w7djbn2-gnome-terminal-3.18.2/share${XDG_DATA_DIRS:+:}$XDG_DATA_DIRS"
> export GTK_PATH="/gnu/store/sqd3q1xq5fsbga00bwhghi9shi7xdaac-gtk+-3.18.2/lib/gtk-3.0${GTK_PATH:+:}$GTK_PATH"
> export GIO_EXTRA_MODULES="/gnu/store/lbkdykh3aa1469li01rwyya02q50r3b9-glib-2.46.1/lib/gio/modules:/gnu/store/ygn4vavcgr79p86jrlmvdycjs81ki4dl-dconf-0.24.0/lib/gio/modules${GIO_EXTRA_MODULES:+:}$GIO_EXTRA_MODULES"
> exec -a "$0" "/gnu/store/x2hz7q806ry8lmwdya0jlff01w7djbn2-gnome-terminal-3.18.2/bin/.gnome-terminal-real" "$@"
> --8<---------------cut here---------------end--------------->8---
> 
> This is done automatically by ‘glib-or-gtk-build-system’, but obviously
> it’s not so great here.
> 
> I suppose that in practice, it would work to simply switch
> gnome-terminal to ‘gnu-build-system’.
> 
> Could you try that?

I tried it. I had to add glib:bin as a native-input.

Trying to run gnome-terminal produces this error (I wrapped the lines):

$ gnome-terminal
Error constructing proxy for
org.gnome.Terminal:/org/gnome/Terminal/Factory0: Error calling
StartServiceByName for org.gnome.Terminal:
GDBus.Error:org.freedesktop.DBus.Error.Spawn.ChildSignaled: Process
org.gnome.Terminal received signal 5

> 
> There may still be other places that add extra entries, though.
> 
> Ludo’.
> 
> 
> 

bug#23118: Duplicate entries in various environment variables

[Leo Prikler]

Hello, Chris

Am Freitag, den 25.03.2016, 20:59 -0700 schrieb Chris Marusich:
> Hi,
> 
> I've noticed that my environment variables contain duplicate
> (sometimes
> more) entries.  This occurs regardless of whether the user logs in
> directly via a tty or via a desktop environment like GNOME.
> 
> [...]
> 
> As you can see, there are some environment variables with up to 9
> duplicate entries.  Is this expected?  Is it a problem?  Why is it
> happening?  Assuming that it is not expected and that it is a
> problem,
> how can we prevent it from happening?
It has been pointed out, that this is somewhat expected when wrapping
the same environment variable multiple times.  For instance, you as a
GNOME user might already have gtk+ in your GTK path if you run GNOME,
but GNOME applications can not rely on that and thus need to add their
own.  As the number of software components, that use it increases, so
does the number of mentions.  
In the special case of gnome-terminal, this is user-visible by printing
out env, but other applications get launched in a similar manner all
the time without you noticing.  Perhaps one could patch GNOME terminal
to clear those variables before spawning the shell, but that's going to
be a bit fiddly.  Alternatively, one could enforce GTK_PATH by using
"=" instead of prefix.

More generally, this can become an issue when environment variables
reach a certain size (and has led to bug reports in Guix before, which
have since been fixed).  POSIX mandates a syntax, that would allow
removing already present components first (see [1]), but I can hardly
imagine what monstrosities we would need to cook up to do this
reliably.  Not to mention, that some otherwise POSIX-compliant shells
might not implement that syntax (correctly).

Regards,
Leo

[1] 
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_02

bug#23118: Duplicate entries in various environment variables

[Maxim Cournoyer]

Hi,

[...]

> More generally, this can become an issue when environment variables
> reach a certain size (and has led to bug reports in Guix before, which
> have since been fixed).  POSIX mandates a syntax, that would allow
> removing already present components first (see [1]), but I can hardly
> imagine what monstrosities we would need to cook up to do this
> reliably.  Not to mention, that some otherwise POSIX-compliant shells
> might not implement that syntax (correctly).

Why are we talking about POSIX here?  Our wrap-program procedure makes
use of Bash; we could make use of its features to filter duplicates,
right?  Unless I'm missing something.

-- 
Thanks,
Maxim