‘guix environment’ vs. ‘.bash_profile’

‘guix environment’ vs. ‘.bash_profile’

[Ludovic Courtès]

Hello Guix!

One thing that often trips up newcomers, especially on other distros, is
the interaction of ‘guix environment’ and ‘.bashrc’ etc. when it comes
to setting environment variables.  Often the first experience is “why
doesn’t ‘guix environment’ set environment variables correctly?”.

The manual does explain this in a footnote¹ but let’s face it, people
don’t read footnotes and consistently define environment variables in
the “wrong” shell startup file.

What can we do about that?  Should we invoke the shell such that it
sources our ‘etc/profile’ after its own initialization file?  Any other
ideas?

Thanks,
Ludo’.

¹ https://guix.gnu.org/manual/en/html_node/Invoking-guix-environment.html#FOOT10

Re: ‘guix environment’ vs. ‘.bash_profile’

[Brendan Tildesley]

Doom Emacs has a tool `doom doctor' for diagnosing common errors. 
Perhaps there could be a `guix doctor' that would check for such things. 
`guix offload test' is already somewhat like that but for offloading, 
althought it could improve. Any bug report from a user where the 
solution is to tell them to fix their environment instead of changing 
guix could also have a check added to guix doctor. Also, we could 
collect knowledge on aspects of GNU/Linux that are unique to Guix that 
one often doesn't learn about on other distributions and include a page 
in the manual. for example I never had any use for sudo -E or sudo -i 
until I started using Guix.

Re: ‘guix environment’ vs. ‘.bash_profile’

[Danny O'Brien]

Brendan Tildesley <mail@brendan.scot> writes:

> Doom Emacs has a tool `doom doctor' for diagnosing common 
> errors. Perhaps there
> could be a `guix doctor' that would check for such things. `guix 
> offload test'
> is already somewhat like that but for offloading, althought it 
> could improve.
> Any bug report from a user where the solution is to tell them to 
> fix their
> environment instead of changing guix could also have a check 
> added to guix
> doctor. Also, we could collect knowledge on aspects of GNU/Linux 
> that are unique
> to Guix that one often doesn't learn about on other 
> distributions and include a
> page in the manual. for example I never had any use for sudo -E 
> or sudo -i
> until I started using Guix.
>

I was going to suggest the same thing -- a "guix doctor" would 
have
helped me a great deal when I was struggling to set up guix at the 
very
start.

Also because I share my shell and other configuration files across
distributions, it's not uncommon for me to fix something in 
another
distribution which causes repeated trouble in guix (or 
vice-versa!). So
a program that I could repeatedly re-run to identify a root cause 
(or
even run as a test to check that a change doesn't break guix) 
would be
great. Just something that can tell you "oh, your environment 
variables
aren't what I would expect, here is what I think may be wrong" 
would be
very useful.

Another quick note: I've noticed a trend in a few FLOSS projects 
of
spending some concentrated time on improving the usability of 
error
messages. Elm and Rust are two examples that spring to mind. The
strategy appears to be to collect common mistakes and their 
errors, and
then iterate on making the error messages not only clearer about 
what
has happened, but also provide concrete suggestions on what can be 
done
to fix the problem -- to the extent of even providing suggested 
commands
that would correct the error.

https://elm-lang.org/news/compiler-errors-for-humans

https://blog.rust-lang.org/2016/08/10/Shape-of-errors-to-come.html

I find Guix error messages to be very good on the whole, but I 
wonder if
the strong connection between Guix and Guile might serve us well 
here.
Clearer error messages in the REPL, informed by Guix usage and
implemented by improving Guile error reporting, would benefit both
immensely. And I think it might makes sense to do this in tandem 
with a
more reactive "guix doctor" project.

I'm not a great coder, but I make a lot of common mistakes, so 
happy to
help in that respect!

d.

Re: ‘guix environment’ vs. ‘.bash_profile’

[Ludovic Courtès]

Hi!

Danny O'Brien <danny@spesh.com> skribis:

> Brendan Tildesley <mail@brendan.scot> writes:
>
>> Doom Emacs has a tool `doom doctor' for diagnosing common
>> errors. Perhaps there
>> could be a `guix doctor' that would check for such things. `guix
>> offload test'
>> is already somewhat like that but for offloading, althought it could
>> improve.
>> Any bug report from a user where the solution is to tell them to fix
>> their
>> environment instead of changing guix could also have a check added
>> to guix
>> doctor.

Interesting.  Any idea how this particular issue could be checked for?
If we can come up with an automated way to determine that “something’s
wrong”, we might as well get ‘guix environment’ to display a hint.

> Also because I share my shell and other configuration files across
> distributions, it's not uncommon for me to fix something in another
> distribution which causes repeated trouble in guix (or
> vice-versa!). So
> a program that I could repeatedly re-run to identify a root cause (or
> even run as a test to check that a change doesn't break guix) would be
> great. Just something that can tell you "oh, your environment
> variables
> aren't what I would expect, here is what I think may be wrong" would
> be
> very useful.

I see.

> Another quick note: I've noticed a trend in a few FLOSS projects of
> spending some concentrated time on improving the usability of error
> messages. Elm and Rust are two examples that spring to mind. The
> strategy appears to be to collect common mistakes and their errors,
> and
> then iterate on making the error messages not only clearer about what
> has happened, but also provide concrete suggestions on what can be
> done
> to fix the problem -- to the extent of even providing suggested
> commands
> that would correct the error.
>
> https://elm-lang.org/news/compiler-errors-for-humans
>
> https://blog.rust-lang.org/2016/08/10/Shape-of-errors-to-come.html

Yes, I very much admire what the Elm folks (and also the GCC folks,
closer to me) have achieved in recent years.

> I find Guix error messages to be very good on the whole, but I wonder
> if
> the strong connection between Guix and Guile might serve us well here.
> Clearer error messages in the REPL, informed by Guix usage and
> implemented by improving Guile error reporting, would benefit both
> immensely. And I think it might makes sense to do this in tandem with
> a
> more reactive "guix doctor" project.

We’ve tried to follow the trend above.  :-)  We’re still very far from
the quality of Elm or recent GCC messages, but I think that, as you
write, we can iterate on specific error messages and gradually improve
them, add hints, and so on.

For example, ‘guix’ displays hints for unbound variables or misnamed
modules because these are common mistakes.  Let’s collect more of those
“common mistakes” and improve the way they are reported.

Thanks,
Ludo’.

Re: ‘guix environment’ vs. ‘.bash_profile’

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

> Hi!
>
> Danny O'Brien <danny@spesh.com> skribis:
>
>> Brendan Tildesley <mail@brendan.scot> writes:
>>
>>> Doom Emacs has a tool `doom doctor' for diagnosing common
>>> errors. Perhaps there
>>> could be a `guix doctor' that would check for such things. `guix
>>> offload test'
>>> is already somewhat like that but for offloading, althought it could
>>> improve.
>>> Any bug report from a user where the solution is to tell them to fix
>>> their
>>> environment instead of changing guix could also have a check added
>>> to guix
>>> doctor.
>
> Interesting.  Any idea how this particular issue could be checked for?
> If we can come up with an automated way to determine that “something’s
> wrong”, we might as well get ‘guix environment’ to display a hint.

The checks would have to be pretty self-contained and carefully ordered.
Imagine LD_LIBRARY_PATH is set to something terribly incompatible: we
might not even be able to launch Guile then!  These tests would also
have to work for older Guixes, e.g. when “guix pull” was used but the
effective “guix” command is /usr/local/bin/guix or even
~/.guix-profile/bin/guix (from a misguided invocation of “guix install
guix”).

This is really tricky to get right when you can’t trust the execution
environment.

> For example, ‘guix’ displays hints for unbound variables or misnamed
> modules because these are common mistakes.

I think these hints are excellent!  I’m often surprised at how good and
useful they are.

I have sometimes been frustrated with the lack of *any* output in case
of errors.  It is possible to get Guix to just print an obscure error
and get neither a backtrace nor a hint.  Of course I can’t remember now
how to replicate this, but this could be a good exercise for new
contributors: do whatever you can to make Guix barf!

One thing that usually only hits contributors is a barrage of unrelated
error messages (with repeated hints to include different modules) that
culminate in something silly like “unknown package” for an unrelated
package that obviously exists.  I’m sure you’ve also encountered this
before while adding or upgrading packages.

-- 
Ricardo

Re: ‘guix environment’ vs. ‘.bash_profile’

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 638 bytes --]

Hey,

I was toying with the idea of automatically detecting discrepancies in
environment variables, so I tried the attached patch, which would check
/proc/PID/environ compared to the profile’s search path variables.

Unfortunately, that doesn’t work because Bash for instance does not
change its own environment variables (with setenv(3) or environ(7)) when
you type “export PATH=foo” or similar (which makes sense, after all).

I’m thinking the only solution left would be to grep ~/.bashrc and
similar for potentially problematic environment variable settings.
Should we do that every time?  Or…?

Ludo’.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 3163 bytes --]

diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scm
index ad50281eb2..33d5bd736a 100644
--- a/guix/scripts/environment.scm
+++ b/guix/scripts/environment.scm
@@ -43,7 +43,9 @@
   #:use-module (ice-9 format)
   #:use-module (ice-9 match)
   #:use-module (ice-9 rdelim)
+  #:use-module (ice-9 textual-ports)
   #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-9 gnu)
   #:use-module (srfi srfi-11)
   #:use-module (srfi srfi-26)
   #:use-module (srfi srfi-37)
@@ -424,6 +426,50 @@ regexps in WHITE-LIST."
     ((program . args)
      (apply execlp program program args))))
 
+(define-immutable-record-type <environment-discrepancy>
+  (environment-discrepancy path expected actual)
+  environment-discrepancy?
+  (path     environment-discrepancy-path)
+  (expected environment-discrepancy-expected)
+  (actual   environment-discrepancy-actual))
+
+(define (process-environment-variables pid)
+  (define split-on-nul
+    (cute string-tokenize <>
+          (char-set-complement (char-set #\nul))))
+
+  (define (split-on-= str)
+    (let ((offset (string-index str #\=)))
+      (cons (string-take str offset)
+            (string-drop str (+ 1 offset)))))
+
+  (call-with-input-file (string-append "/proc/" (number->string pid)
+                                       "/environ")
+    (lambda (port)
+      (map split-on-=
+           (split-on-nul (get-string-all port))))))
+
+(define (process-environment-discrepancies pid profile manifest)
+  (let ((variables (process-environment-variables pid))
+        (paths     (profile-search-paths profile manifest)))
+    (filter-map (match-lambda
+                  ((path . value)
+                   (let ((name (search-path-specification-variable
+                                path)))
+                     (match (assoc-ref variables name)
+                       (#f
+                        (environment-discrepancy path value #f))
+                       (actual
+                        (pk 'var name actual value)
+                        (and (not (string-prefix? value
+                                                  actual))
+                             (environment-discrepancy path
+                                                      value actual)))))))
+                paths)))
+
+(define (check-environment pid profile manifest)
+  (pk 'disc (process-environment-discrepancies pid profile manifest)))
+
 (define* (launch-environment/fork command profile manifest
                                   #:key pure? (white-list '()))
   "Run COMMAND in a new process with an environment containing PROFILE, with
@@ -434,8 +480,11 @@ regexps in WHITE-LIST."
     (0 (launch-environment command profile manifest
                            #:pure? pure?
                            #:white-list white-list))
-    (pid (match (waitpid pid)
-           ((_ . status) status)))))
+    (pid
+     (sleep 1)
+     (check-environment pid profile manifest)
+     (match (waitpid pid)
+       ((_ . status) status)))))
 
 (define* (launch-environment/container #:key command bash user user-mappings
                                        profile manifest link-profile? network?