bug#26366: Building Guix from within a container

bug#26366: Building Guix from within a container

[Clément Lassieur]

Hi,

I had a lot of trouble building Guix recently (is it related with Guile
2.2?).  Or it did build but then 'guix' command produced things like:

;;; WARNING: loading compiled file /home/clement/.config/guix/latest/guix/ui.go failed:
;;; ERROR: In procedure make_objcode_from_file: bad header on object file: "\x7fELF\x02\x01\x01�\x00\x00\x00\x00\x00\x00\x00\x00"
;;; note: source file /home/clement/.config/guix/latest/guix/ui.scm
;;;       newer than compiled /gnu/store/my64yq4r2ii0bblvsbfnjq5bnkivvn07-guix-0.12.0-5.1162/share/guile/site/2.0/guix/ui.go
;;; WARNING: loading compiled file /home/clement/.config/guix/latest/guix/utils.go failed:
;;; ERROR: In procedure make_objcode_from_file: bad header on object file: "\x7fELF\x02\x01\x01�\x00\x00\x00\x00\x00\x00\x00\x00"
;;; note: source file /home/clement/.config/guix/latest/guix/utils.scm
;;;       newer than compiled /gnu/store/my64yq4r2ii0bblvsbfnjq5bnkivvn07-guix-0.12.0-5.1162/share/guile/site/2.0/guix/utils.go
;;; WARNING: loading compiled file /home/clement/.config/guix/latest/guix/config.go failed:
;;; ERROR: In procedure make_objcode_from_file: bad header on object file: "\x7fELF\x02\x01\x01�\x00\x00\x00\x00\x00\x00\x00\x00"
;;; note: source file /home/clement/.config/guix/latest/guix/config.scm
;;;       newer than compiled /gnu/store/my64yq4r2ii0bblvsbfnjq5bnkivvn07-guix-0.12.0-5.1162/share/guile/site/2.0/guix/config.go

I finally got it to work by using 'guix environment -C -N guix'.  Is it
the right way to build Guix?  If so, here is a patch that updates the
documentation.

bug#26366: [PATCH] doc: Specify that Guix should be built from within a container.

[Clément Lassieur]

* doc/contributing.texi: add '--container' and '--network' options to the
'guix environment guix' command.
---
 doc/contributing.texi | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/doc/contributing.texi b/doc/contributing.texi
index bbc93424b..dbafa4139 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -45,19 +45,19 @@ the installation instructions (@pxref{Requirements}).
 @end itemize
 
 The easiest way to set up a development environment for Guix is, of
-course, by using Guix!  The following command starts a new shell where
-all the dependencies and appropriate environment variables are set up to
-hack on Guix:
+course, by using Guix!  The following command starts a new shell within
+an isolated container where all the dependencies and appropriate
+environment variables are set up to hack on Guix:
 
 @example
-guix environment guix
+guix environment --container --network guix
 @end example
 
 @xref{Invoking guix environment}, for more information on that command.
 Extra dependencies can be added with @option{--ad-hoc}:
 
 @example
-guix environment guix --ad-hoc help2man git strace
+guix environment --container --network guix --ad-hoc help2man git strace
 @end example
 
 Run @command{./bootstrap} to generate the build system infrastructure
-- 
2.12.2

bug#26366: [PATCH] doc: Specify that Guix should be built from within a container.

[Ludovic Courtès]

Clément Lassieur <clement@lassieur.org> skribis:

> * doc/contributing.texi: add '--container' and '--network' options to the
> 'guix environment guix' command.

[...]

> +course, by using Guix!  The following command starts a new shell within
> +an isolated container where all the dependencies and appropriate
> +environment variables are set up to hack on Guix:
>  
>  @example
> -guix environment guix
> +guix environment --container --network guix
>  @end example

What’s the rationale?  Was this to address the warnings in your first
message?

It’s possible and a good way to silence the warnings you were seeing,
but it should work fine without it too.

Thanks,
Ludo’.

bug#26366: [PATCH] doc: Specify that Guix should be built from within a container.

[Clément Lassieur]

Ludovic Courtès <ludo@gnu.org> writes:

> Clément Lassieur <clement@lassieur.org> skribis:
>
>> * doc/contributing.texi: add '--container' and '--network' options to the
>> 'guix environment guix' command.
>
> [...]
>
>> +course, by using Guix!  The following command starts a new shell within
>> +an isolated container where all the dependencies and appropriate
>> +environment variables are set up to hack on Guix:
>>  
>>  @example
>> -guix environment guix
>> +guix environment --container --network guix
>>  @end example
>
> What’s the rationale?  Was this to address the warnings in your first
> message?

This was both to address the warnings in my first message, and my build
error.  I apologize if my message is (still) not clear, but it is
difficult to reproduce those bugs, since they are state-dependent.

> It’s possible and a good way to silence the warnings you were seeing,
> but it should work fine without it too.

I don't understand why it should work fine.  Guix (or Guile-SSH, for
example) both depend on Guile 2.0 (which is an input).  But let's say
that on my system, Guile 2.2 is installed instead.  I want to build Guix
(or Guile-SSH).  I usually do 'guix environment guix' (or 'guix
environment guile-ssh').  Then, in my opinion, 'guile --version' should
output 2.0, but it outputs 2.2.  Is it supposed to build with the wrong
Guile?  (If I do 'guix environment -C guix', then 'guile --version'
outputs 2.0.)

bug#26366: [PATCH] doc: Specify that Guix should be built from within a container.

[Ludovic Courtès]

Hello!

Clément Lassieur <clement@lassieur.org> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Clément Lassieur <clement@lassieur.org> skribis:
>>
>>> * doc/contributing.texi: add '--container' and '--network' options to the
>>> 'guix environment guix' command.
>>
>> [...]
>>
>>> +course, by using Guix!  The following command starts a new shell within
>>> +an isolated container where all the dependencies and appropriate
>>> +environment variables are set up to hack on Guix:
>>>  
>>>  @example
>>> -guix environment guix
>>> +guix environment --container --network guix
>>>  @end example
>>
>> What’s the rationale?  Was this to address the warnings in your first
>> message?
>
> This was both to address the warnings in my first message, and my build
> error.  I apologize if my message is (still) not clear, but it is
> difficult to reproduce those bugs, since they are state-dependent.

Yeah, I see.

>> It’s possible and a good way to silence the warnings you were seeing,
>> but it should work fine without it too.
>
> I don't understand why it should work fine.  Guix (or Guile-SSH, for
> example) both depend on Guile 2.0 (which is an input).  But let's say
> that on my system, Guile 2.2 is installed instead.  I want to build Guix
> (or Guile-SSH).  I usually do 'guix environment guix' (or 'guix
> environment guile-ssh').  Then, in my opinion, 'guile --version' should
> output 2.0, but it outputs 2.2.  Is it supposed to build with the wrong
> Guile?  (If I do 'guix environment -C guix', then 'guile --version'
> outputs 2.0.)

‘guix environment’ definitely prepends things to PATH et al.

Could it be that you have variable definitions in the wrong Bash init
files as noted in
<https://www.gnu.org/software/guix/manual/html_node/Invoking-guix-environment.html#FOOT14>?

HTH!

Ludo’.

bug#26366: [PATCH] doc: Specify that Guix should be built from within a container.

[Clément Lassieur]

Ludovic Courtès <ludo@gnu.org> writes:

> Hello!
>
> Clément Lassieur <clement@lassieur.org> skribis:
>
>> Ludovic Courtès <ludo@gnu.org> writes:
>>
>>> Clément Lassieur <clement@lassieur.org> skribis:
>>>
>>>> * doc/contributing.texi: add '--container' and '--network' options to the
>>>> 'guix environment guix' command.
>>>
>>> [...]
>>>
>>>> +course, by using Guix!  The following command starts a new shell within
>>>> +an isolated container where all the dependencies and appropriate
>>>> +environment variables are set up to hack on Guix:
>>>>  
>>>>  @example
>>>> -guix environment guix
>>>> +guix environment --container --network guix
>>>>  @end example
>>>
>>> What’s the rationale?  Was this to address the warnings in your first
>>> message?
>>
>> This was both to address the warnings in my first message, and my build
>> error.  I apologize if my message is (still) not clear, but it is
>> difficult to reproduce those bugs, since they are state-dependent.
>
> Yeah, I see.
>
>>> It’s possible and a good way to silence the warnings you were seeing,
>>> but it should work fine without it too.
>>
>> I don't understand why it should work fine.  Guix (or Guile-SSH, for
>> example) both depend on Guile 2.0 (which is an input).  But let's say
>> that on my system, Guile 2.2 is installed instead.  I want to build Guix
>> (or Guile-SSH).  I usually do 'guix environment guix' (or 'guix
>> environment guile-ssh').  Then, in my opinion, 'guile --version' should
>> output 2.0, but it outputs 2.2.  Is it supposed to build with the wrong
>> Guile?  (If I do 'guix environment -C guix', then 'guile --version'
>> outputs 2.0.)
>
> ‘guix environment’ definitely prepends things to PATH et al.
>
> Could it be that you have variable definitions in the wrong Bash init
> files as noted in
> <https://www.gnu.org/software/guix/manual/html_node/Invoking-guix-environment.html#FOOT14>?

Oh...  yes that's it :)  Sorry the waste of time, I should have followed
the documentation.  I close the bug, since I think it's all related to
this.

Thanks,
Clément

bug#26366: [PATCH] doc: Specify that Guix should be built from within a container.

[Ludovic Courtès]

Clément Lassieur <clement@lassieur.org> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:

[...]

>> ‘guix environment’ definitely prepends things to PATH et al.
>>
>> Could it be that you have variable definitions in the wrong Bash init
>> files as noted in
>> <https://www.gnu.org/software/guix/manual/html_node/Invoking-guix-environment.html#FOOT14>?
>
> Oh...  yes that's it :)  Sorry the waste of time, I should have followed
> the documentation.  I close the bug, since I think it's all related to
> this.

No problem!  This one often bites people…

Ludo’.

bug#26366: Building Guix from within a container

[Clément Lassieur]

Clément Lassieur <clement@lassieur.org> writes:

> Hi,
>
> I had a lot of trouble building Guix recently (is it related with Guile
> 2.2?).  Or it did build but then 'guix' command produced things like:
>
> ;;; WARNING: loading compiled file /home/clement/.config/guix/latest/guix/ui.go failed:
> ;;; ERROR: In procedure make_objcode_from_file: bad header on object file: "\x7fELF\x02\x01\x01�\x00\x00\x00\x00\x00\x00\x00\x00"
> ;;; note: source file /home/clement/.config/guix/latest/guix/ui.scm
> ;;;       newer than compiled /gnu/store/my64yq4r2ii0bblvsbfnjq5bnkivvn07-guix-0.12.0-5.1162/share/guile/site/2.0/guix/ui.go
> ;;; WARNING: loading compiled file /home/clement/.config/guix/latest/guix/utils.go failed:
> ;;; ERROR: In procedure make_objcode_from_file: bad header on object file: "\x7fELF\x02\x01\x01�\x00\x00\x00\x00\x00\x00\x00\x00"
> ;;; note: source file /home/clement/.config/guix/latest/guix/utils.scm
> ;;;       newer than compiled /gnu/store/my64yq4r2ii0bblvsbfnjq5bnkivvn07-guix-0.12.0-5.1162/share/guile/site/2.0/guix/utils.go
> ;;; WARNING: loading compiled file /home/clement/.config/guix/latest/guix/config.go failed:
> ;;; ERROR: In procedure make_objcode_from_file: bad header on object file: "\x7fELF\x02\x01\x01�\x00\x00\x00\x00\x00\x00\x00\x00"
> ;;; note: source file /home/clement/.config/guix/latest/guix/config.scm
> ;;;       newer than compiled /gnu/store/my64yq4r2ii0bblvsbfnjq5bnkivvn07-guix-0.12.0-5.1162/share/guile/site/2.0/guix/config.go
>
> I finally got it to work by using 'guix environment -C -N guix'.  Is it
> the right way to build Guix?  If so, here is a patch that updates the
> documentation.

I forgot to say that I can't reproduce those issues anymore.  I'm pretty
confident --container helped, but I can't say why for sure.

bug#26366: Building Guix from within a container

[Clément Lassieur]

Clément Lassieur <clement@lassieur.org> writes:
> Clément Lassieur <clement@lassieur.org> writes:
>
>> Hi,
>>
>> I had a lot of trouble building Guix recently (is it related with Guile
>> 2.2?).  Or it did build but then 'guix' command produced things like:
>>
>> ;;; WARNING: loading compiled file /home/clement/.config/guix/latest/guix/ui.go failed:
>> ;;; ERROR: In procedure make_objcode_from_file: bad header on object file: "\x7fELF\x02\x01\x01�\x00\x00\x00\x00\x00\x00\x00\x00"
>> ;;; note: source file /home/clement/.config/guix/latest/guix/ui.scm
>> ;;;       newer than compiled /gnu/store/my64yq4r2ii0bblvsbfnjq5bnkivvn07-guix-0.12.0-5.1162/share/guile/site/2.0/guix/ui.go
>> ;;; WARNING: loading compiled file /home/clement/.config/guix/latest/guix/utils.go failed:
>> ;;; ERROR: In procedure make_objcode_from_file: bad header on object file: "\x7fELF\x02\x01\x01�\x00\x00\x00\x00\x00\x00\x00\x00"
>> ;;; note: source file /home/clement/.config/guix/latest/guix/utils.scm
>> ;;;       newer than compiled /gnu/store/my64yq4r2ii0bblvsbfnjq5bnkivvn07-guix-0.12.0-5.1162/share/guile/site/2.0/guix/utils.go
>> ;;; WARNING: loading compiled file /home/clement/.config/guix/latest/guix/config.go failed:
>> ;;; ERROR: In procedure make_objcode_from_file: bad header on object file: "\x7fELF\x02\x01\x01�\x00\x00\x00\x00\x00\x00\x00\x00"
>> ;;; note: source file /home/clement/.config/guix/latest/guix/config.scm
>> ;;;       newer than compiled /gnu/store/my64yq4r2ii0bblvsbfnjq5bnkivvn07-guix-0.12.0-5.1162/share/guile/site/2.0/guix/config.go
>>
>> I finally got it to work by using 'guix environment -C -N guix'.  Is it
>> the right way to build Guix?  If so, here is a patch that updates the
>> documentation.
>
> I forgot to say that I can't reproduce those issues anymore.  I'm pretty
> confident --container helped, but I can't say why for sure.

Actually, the error described here on #guix
(https://gnunet.org/bot/log/guix/2017-04-06#T1346052) might be the one I
had:

checking how to run the C preprocessor... /lib/cpp” -> configure: error. 

It seems that --pure solved it, which might be an alternative solution.
But anyway it looks like there are a few corner cases for which just
running 'guix environment guix' won't work.

bug#26366: Building Guix from within a container

[Clément Lassieur]

Clément Lassieur <clement@lassieur.org> writes:
> Clément Lassieur <clement@lassieur.org> writes:
>> Clément Lassieur <clement@lassieur.org> writes:
>>
>>> Hi,
>>>
>>> I had a lot of trouble building Guix recently (is it related with Guile
>>> 2.2?).  Or it did build but then 'guix' command produced things like:
>>>
>>> ;;; WARNING: loading compiled file /home/clement/.config/guix/latest/guix/ui.go failed:
>>> ;;; ERROR: In procedure make_objcode_from_file: bad header on object file: "\x7fELF\x02\x01\x01�\x00\x00\x00\x00\x00\x00\x00\x00"
>>> ;;; note: source file /home/clement/.config/guix/latest/guix/ui.scm
>>> ;;;       newer than compiled /gnu/store/my64yq4r2ii0bblvsbfnjq5bnkivvn07-guix-0.12.0-5.1162/share/guile/site/2.0/guix/ui.go
>>> ;;; WARNING: loading compiled file /home/clement/.config/guix/latest/guix/utils.go failed:
>>> ;;; ERROR: In procedure make_objcode_from_file: bad header on object file: "\x7fELF\x02\x01\x01�\x00\x00\x00\x00\x00\x00\x00\x00"
>>> ;;; note: source file /home/clement/.config/guix/latest/guix/utils.scm
>>> ;;;       newer than compiled /gnu/store/my64yq4r2ii0bblvsbfnjq5bnkivvn07-guix-0.12.0-5.1162/share/guile/site/2.0/guix/utils.go
>>> ;;; WARNING: loading compiled file /home/clement/.config/guix/latest/guix/config.go failed:
>>> ;;; ERROR: In procedure make_objcode_from_file: bad header on object file: "\x7fELF\x02\x01\x01�\x00\x00\x00\x00\x00\x00\x00\x00"
>>> ;;; note: source file /home/clement/.config/guix/latest/guix/config.scm
>>> ;;;       newer than compiled /gnu/store/my64yq4r2ii0bblvsbfnjq5bnkivvn07-guix-0.12.0-5.1162/share/guile/site/2.0/guix/config.go
>>>
>>> I finally got it to work by using 'guix environment -C -N guix'.  Is it
>>> the right way to build Guix?  If so, here is a patch that updates the
>>> documentation.
>>
>> I forgot to say that I can't reproduce those issues anymore.  I'm pretty
>> confident --container helped, but I can't say why for sure.
>
> Actually, the error described here on #guix
> (https://gnunet.org/bot/log/guix/2017-04-06#T1346052) might be the one I
> had:
>
> checking how to run the C preprocessor... /lib/cpp” -> configure: error. 
>
> It seems that --pure solved it, which might be an alternative solution.
> But anyway it looks like there are a few corner cases for which just
> running 'guix environment guix' won't work.

Another example just happened here:
https://gnunet.org/bot/log/guix/2017-04-13#T1353026.

--8<---------------cut here---------------start------------->8---
I'm trying to build guix from git. I use 'guix environment guix', and
get 'configure: error: C preprocessor "/lib/cpp" fails sanity
check'. Anything else I need? I'm trying to follow
https://www.gnu.org/software/guix/manual/html_node/Building-from-Git.html
--8<---------------cut here---------------end--------------->8---

To be clear, it seems that there are different bugs, with one solution
(--container).  That's why I talk about it in this thread.

bug#26366: Building Guix from within a container

[Ludovic Courtès]

Clément Lassieur <clement@lassieur.org> skribis:

> I'm trying to build guix from git. I use 'guix environment guix', and
> get 'configure: error: C preprocessor "/lib/cpp" fails sanity
> check'. Anything else I need? I'm trying to follow
> https://www.gnu.org/software/guix/manual/html_node/Building-from-Git.html

Could you send config.log?  I’m not sure what’s happening here.

TIA,
Ludo’.

bug#26366: Building Guix from within a container

[Clément Lassieur]

Ludovic Courtès <ludo@gnu.org> writes:

> Clément Lassieur <clement@lassieur.org> skribis:
>
>> I'm trying to build guix from git. I use 'guix environment guix', and
>> get 'configure: error: C preprocessor "/lib/cpp" fails sanity
>> check'. Anything else I need? I'm trying to follow
>> https://www.gnu.org/software/guix/manual/html_node/Building-from-Git.html
>
> Could you send config.log?  I’m not sure what’s happening here.

This was a quote from the chat (#guix) from today.  I can't reproduce
this issue.

bug#26366: Building Guix from within a container

[Ludovic Courtès]

Hi Clément,

Clément Lassieur <clement@lassieur.org> skribis:

> I had a lot of trouble building Guix recently (is it related with Guile
> 2.2?).  Or it did build but then 'guix' command produced things like:
>
> ;;; WARNING: loading compiled file /home/clement/.config/guix/latest/guix/ui.go failed:
> ;;; ERROR: In procedure make_objcode_from_file: bad header on object file: "\x7fELF\x02\x01\x01�\x00\x00\x00\x00\x00\x00\x00\x00"
> ;;; note: source file /home/clement/.config/guix/latest/guix/ui.scm
> ;;;       newer than compiled /gnu/store/my64yq4r2ii0bblvsbfnjq5bnkivvn07-guix-0.12.0-5.1162/share/guile/site/2.0/guix/ui.go
> ;;; WARNING: loading compiled file /home/clement/.config/guix/latest/guix/utils.go failed:
> ;;; ERROR: In procedure make_objcode_from_file: bad header on object file: "\x7fELF\x02\x01\x01�\x00\x00\x00\x00\x00\x00\x00\x00"
> ;;; note: source file /home/clement/.config/guix/latest/guix/utils.scm
> ;;;       newer than compiled /gnu/store/my64yq4r2ii0bblvsbfnjq5bnkivvn07-guix-0.12.0-5.1162/share/guile/site/2.0/guix/utils.go
> ;;; WARNING: loading compiled file /home/clement/.config/guix/latest/guix/config.go failed:
> ;;; ERROR: In procedure make_objcode_from_file: bad header on object file: "\x7fELF\x02\x01\x01�\x00\x00\x00\x00\x00\x00\x00\x00"
> ;;; note: source file /home/clement/.config/guix/latest/guix/config.scm
> ;;;       newer than compiled /gnu/store/my64yq4r2ii0bblvsbfnjq5bnkivvn07-guix-0.12.0-5.1162/share/guile/site/2.0/guix/config.go

These warnings come from Guile 2.0 trying to load Guile 2.2 .go files
(in 2.2, .go files are ELF files; in 2.0, it’s a custom format.)

They’re annoying but harmless and just mean that you have a mixture of
2.2 and 2.0 files in GUILE_LOAD_COMPILED_PATH.

HTH!

Ludo’.

bug#26366: Building Guix from within a container

[Pjotr Prins]

On Sat, Apr 08, 2017 at 02:38:40PM +0200, Ludovic Courtès wrote:
> These warnings come from Guile 2.0 trying to load Guile 2.2 .go files
> (in 2.2, .go files are ELF files; in 2.0, it’s a custom format.)
> 
> They’re annoying but harmless and just mean that you have a mixture of
> 2.2 and 2.0 files in GUILE_LOAD_COMPILED_PATH.

To be honest I think we should get rid of harmless messages - or only
show the first or last one (perhaps state that there are more similar
messages which can be seen in debug mode). I understand that this is a
guile thing, but the same holds for guix with all the warnings we get
when sylinks/files are duplicated in the store. 

Irony is that sometimes we don't get warnings when we need them. Such
as when you specify a substitute-url and if the server does not exist
there is no indication. I have wasted many a time on figuring that
problem out.

It would be nice to have a policy where we do not show all harmless
warnings by default, but only in debug mode, as well as missing
services etc. I am happy to run --debug when I actually face a
problem.

Pj.

bug#26366: Building Guix from within a container

[Ludovic Courtès]

Heya,

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> To be honest I think we should get rid of harmless messages - or only
> show the first or last one (perhaps state that there are more similar
> messages which can be seen in debug mode). I understand that this is a
> guile thing, but the same holds for guix with all the warnings we get
> when sylinks/files are duplicated in the store. 

So, one thing at a time.  :-)

I think this specific Guile warning makes some sense, but it’s not a
discussion for Guix here.

The “sylinks/files are duplicated in the store” thing you’re referring
to is when you have the same file multiple times in a profile and you
get a warning (“arbitrarily choosing…”) when building the profile right?

I’ve discussed a fix long ago that would raise an error when you have
real conflicts in a profile (e.g., same package twice but with different
versions), rather than having these warnings.  I haven’t gotten around
to implementing it yet.

> Irony is that sometimes we don't get warnings when we need them. Such
> as when you specify a substitute-url and if the server does not exist
> there is no indication. I have wasted many a time on figuring that
> problem out.

The idea here is that --substitute-urls="https://foo https://bar" would
pick whichever of these servers is available, and silently ignore the
other one (for the DNS resolution and the initial HTTP request;
subsequent HTTP requests do lead to an error/warning if they fail.)
We could revisit that, but no discussion will take place if there’s not
a bug report in the first place.  :-)

> It would be nice to have a policy where we do not show all harmless
> warnings by default, but only in debug mode, as well as missing
> services etc. I am happy to run --debug when I actually face a
> problem.

“Missing services”?

I think everyone agrees on the goal.  What we need is to precise list of
these issues and discuss possible solutions for each of them.

Thanks in advance for the upcoming bug reports!  ;-)

Ludo’.