bug#35691: remote configure failure due to bad pathnames for crt*.o

bug#35691: remote configure failure due to bad pathnames for crt*.o

[Nelson H. F. Beebe]

I built a new virtual machine (VM) on 2-May-2019 from the
newly-announced GUIX 1.0 release in the

	 guix-system-install-1.0.0.x86_64-linux.iso 

image file. The VM runs on VirtManager (QEMU/KVM) on CentOS 7.

I initially tried to select 3 desktop managers, but later in the
installation, one of them died and the system could not recover, nor
could it boot from the virtual disk.  I tried again, choosing only
xfce as the desktop manager, and that succeeded.

Since them, I've been busy trying to understand the GUIX model of
UNIX, and testing software installations with a goal to being able to
announce a TeX Live 2019 port to GUIX 1.0 (I'm part of the TeX Live
development team).  My build reports for TeX Live appear here:

	http://www.math.utah.edu/pub/texlive-utah-2019/

On GUIX 1.0, I found that I could take arbitrary GNU packages (and
many of my own) and invoke the usual commands

	tar xf FOO-x.y.z.tar.gz
	cd FOO-x.y.z
	./configure && make all check && make install

AS LONG AS I am logged in on a terminal window via ssh from another
system.

However, if I instead execute those commands remotely with

	% ssh guix10.vm.example.com 'cd FOO-x.y.z ; ./configure && make all check && make install'

then the configure script fails like this:

	checking whether the C compiler works... no
	configure: error: in `/local/build/cc/dw-2.00':
	configure: error: C compiler cannot create executables
	See `config.log' for more details
	...

The config.log file contains these messages:

	ld: cannot find crt1.o: No such file or directory
	ld: cannot find crti.o: No such file or directory
	collect2: error: ld returned 1 exit status

In separate experiments, I verified that my shell enviroment (as
reported by "env | sort") was the same in both cases.

I then installed the strace package and ran

	% ssh guix10.vm.example.com 'cd FOO-x.y.z ; strace -f -o foo.strace ./configure'

That identified the problem:

	% fgrep crt1.o foo.strace | grep 'lib.*crt'
	...
	28734 access("/gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/libcrt1.o", R_OK) = -1 ENOENT (No such file or directory)

There is a missing path separator in the word "libcrt1.o"; the correct
file is this one:

	$ ls -l /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/crt1.o
	-r--r--r-- 2 root root 2024 Dec 31  1969 /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/crt1.o

The bad pathname may be generated by GUIX's ld, but it is unclear to
me why it works in an interactive login, but fails as shown in an ssh
batch command execution.

Until this problem is fixed, I cannot incorporate GUIX in my massive
build farm testing on 250+ flavors of Unix that I routinely do for
many GNU, and other, packages.


-------------------------------------------------------------------------------
- Nelson H. F. Beebe                    Tel: +1 801 581 5254                  -
- University of Utah                    FAX: +1 801 581 4148                  -
- Department of Mathematics, 110 LCB    Internet e-mail: beebe@math.utah.edu  -
- 155 S 1400 E RM 233                       beebe@acm.org  beebe@computer.org -
- Salt Lake City, UT 84112-0090, USA    URL: http://www.math.utah.edu/~beebe/ -
-------------------------------------------------------------------------------

bug#35691: remote configure failure due to bad pathnames for crt*.o

[Ludovic Courtès]

Hi Nelson,

"Nelson H. F. Beebe" <beebe@math.utah.edu> skribis:

> However, if I instead execute those commands remotely with
>
> 	% ssh guix10.vm.example.com 'cd FOO-x.y.z ; ./configure && make all check && make install'
>
> then the configure script fails like this:
>
> 	checking whether the C compiler works... no
> 	configure: error: in `/local/build/cc/dw-2.00':
> 	configure: error: C compiler cannot create executables
> 	See `config.log' for more details
> 	...
>
> The config.log file contains these messages:
>
> 	ld: cannot find crt1.o: No such file or directory
> 	ld: cannot find crti.o: No such file or directory
> 	collect2: error: ld returned 1 exit status

You need to install the ‘gcc-toolchain’ package, not just ‘gcc’:

  https://www.gnu.org/software/guix/manual/en/html_node/Application-Setup.html#The-GCC-toolchain

I hope this helps!

Thanks for testing,
Ludo’.

bug#35691: remote configure failure due to bad pathnames for crt*.o

[Ludovic Courtès]

Hi Nelson,

"Nelson H. F. Beebe" <beebe@math.utah.edu> skribis:

> 	configure: error: C compiler cannot create executables
> 	See `config.log' for more details

What does ‘config.log’ show?

What does ‘guix package --list-installed’ return?

Please make sure that only ‘gcc-toolchain’ is in your profile, and not ‘gcc’.

HTH!

Ludo’.

bug#35691: remote configure failure due to bad pathnames for crt*.o

[Nelson H. F. Beebe]

>> ...
>> >       configure: error: C compiler cannot create executables
>> >       See `config.log' for more details
>> 
>> What does config.log show?
>> 
>> What does guix package --list-installed return?
>> 
>> Please make sure that only gcc-toolchain in your profile, and not gcc.
>> ...

The config.log fragment is

	configure:2095: gcc  -I${prefix}/include  -I${prefix}/include -L/usr/uumath/lib64 -Wl,-rpath,/usr/uumath/lib64 conftest.c  >&5
	ld: cannot find crt1.o: No such file or directory
	ld: cannot find crti.o: No such file or directory
	collect2: error: ld returned 1 exit status

I tried

	guix remove gcc

Same problem.  I then did

	guix remove gcc-toolchain

Same problem: there is still a gcc visible: /u/sy/beebe/.guix-profile/bin/gcc.

The strace output shows that it tries to load .../libcrt1.o, instead of .../lib/crt1.o.

Here is what I find about installed gcc-related packages:

	$ guix package --list-installed |grep gcc
	gccgo   4.9.4   out     /gnu/store/7108fl31hfj9lb4y76g89sxdb43c1wpc-gccgo-4.9.4
	gcc-objc        8.3.0   out     /gnu/store/44r4xl10nmix25bv6bv974r280vc232c-gcc-objc-8.3.0
	gcc-objc++      8.3.0   out     /gnu/store/wqycqqp4almqcmmsj7gys56lyk9girvq-gcc-objc++-8.3.0
	gcc-bootstrap   0       out     /gnu/store/fgarfy5392vz5hik1ag5b8ccv0r4cm01-gcc-bootstrap-0
	gcc-toolchain   9.1.0   out     /gnu/store/aqv6njg05w2yqh8krgrjy0wcbfxb4nn3-gcc-toolchain-9.1.0

Maybe it is related to another problem that I wanted to eventually
raise on a guix list.  Namely, after installing the new GUIX 1.0 VM, I
found that /usr/bin, /usr/sbin, and /sbin do not exist, and /bin
contains only /bin/sh.  Even after numerous "guix install" runs, /bin
continues to have only one entry.

This seems bizarre, because it is long-standing practice that
shell scripts should set PATH to a limited value to avoid
trojan attacks, and things like "/bin/rm foo" should always work.

On your GUIX systems, do you have a reasonably populated /bin?  Ditto
the other three.

For reference, here is the system configuration:

;; This is an operating system configuration generated
;; by the graphical installer.

(use-modules (gnu))
(use-service-modules desktop networking ssh xorg)

(operating-system
  (locale "en_US.utf8")
  (timezone "America/Denver")
  (keyboard-layout
    (keyboard-layout "us" "altgr-intl"))
  (bootloader
    (bootloader-configuration
      (bootloader grub-bootloader)
      (target "/dev/sda")
      (keyboard-layout keyboard-layout)))
  (file-systems
    (cons* (file-system
             (mount-point "/")
             (device
               (uuid "a7e44cea-8fee-4431-9658-59644e18c9ec"
                     'ext4))
             (type "ext4"))
           %base-file-systems))
  (host-name "guix10.vm.math.utah.")
 
  ;; ===========================================================
  ;; [02-May-2019] Utah additions
  ;; After updating this file, run
  ;;	guix system reconfigure /etc/config.scm
  ;; to propagate the changes into assorted system files.
  ;; ===========================================================

;  (groups (cons*    
;	   (user-group (name "beebe") (id 887))
;	   (user-group (name "bowman") (id 888))
;	   (user-group (name "sysstaff") (id 810))))

  (users (cons*
	  (user-account
           (name "x")
           (comment "X")
           (group "users")
           (home-directory "/home/x")
           (supplementary-groups '("wheel" "netdev" "audio" "video")))

	  (user-account
	   (name "beebe")
	   (comment "Nelson H F Beebe")
	   (uid 887)
	   (group "users")
	   (home-directory "/u/sy/beebe")
	   (supplementary-groups '("wheel" "netdev" "audio" "video")))

	  (user-account
	   (name "bowman")
	   (comment "Pieter Bowman")
	   (uid 888)
	   (group "users")
	   (home-directory "/u/sy/bowman")
	   (supplementary-groups '("wheel" "netdev" "audio" "video")))

	  (user-account
	   (name "sysstaff")
	   (comment "System Staff")
	   (uid 810)
	   (group "users")
	   (home-directory "/u/sy/sysstaff")
	   (supplementary-groups '("wheel" "netdev" "audio" "video")))

          %base-user-accounts))

  (packages
    (list (specification->package "nss-certs")))

  (services
    (append
      (list (service xfce-desktop-service-type)
            (service openssh-service-type)
            (set-xorg-configuration
              (xorg-configuration
                (keyboard-layout keyboard-layout))))
      %desktop-services))
  )

-------------------------------------------------------------------------------
- Nelson H. F. Beebe                    Tel: +1 801 581 5254                  -
- University of Utah                    FAX: +1 801 581 4148                  -
- Department of Mathematics, 110 LCB    Internet e-mail: beebe@math.utah.edu  -
- 155 S 1400 E RM 233                       beebe@acm.org  beebe@computer.org -
- Salt Lake City, UT 84112-0090, USA    URL: http://www.math.utah.edu/~beebe/ -
-------------------------------------------------------------------------------

bug#35691: remote configure failure due to bad pathnames for crt*.o

[Ludovic Courtès]

Hi Nelson,

(Please keep the bug Cc’d.)

"Nelson H. F. Beebe" <beebe@math.utah.edu> skribis:

> Here is what I find about installed gcc-related packages:
>
> 	$ guix package --list-installed |grep gcc
> 	gccgo   4.9.4   out     /gnu/store/7108fl31hfj9lb4y76g89sxdb43c1wpc-gccgo-4.9.4
> 	gcc-objc        8.3.0   out     /gnu/store/44r4xl10nmix25bv6bv974r280vc232c-gcc-objc-8.3.0
> 	gcc-objc++      8.3.0   out     /gnu/store/wqycqqp4almqcmmsj7gys56lyk9girvq-gcc-objc++-8.3.0
> 	gcc-bootstrap   0       out     /gnu/store/fgarfy5392vz5hik1ag5b8ccv0r4cm01-gcc-bootstrap-0
> 	gcc-toolchain   9.1.0   out     /gnu/store/aqv6njg05w2yqh8krgrjy0wcbfxb4nn3-gcc-toolchain-9.1.0

It’s crowded here.  Please remove everything but ‘gcc-toolchain’.

Now, if you want to have Objective-C support, you of course have to
install ‘gcc-objc’ and/or ‘gcc-objc++’.  However, you’ll have to make
sure that they come after ‘gcc-toolchain’ in your profile, like so:

  guix install gcc-toolchain gcc-objc

> Maybe it is related to another problem that I wanted to eventually
> raise on a guix list.  Namely, after installing the new GUIX 1.0 VM, I
> found that /usr/bin, /usr/sbin, and /sbin do not exist, and /bin
> contains only /bin/sh.  Even after numerous "guix install" runs, /bin
> continues to have only one entry.

Correct.

> This seems bizarre, because it is long-standing practice that
> shell scripts should set PATH to a limited value to avoid
> trojan attacks, and things like "/bin/rm foo" should always work.

The GNU build system purposefully makes no assumptions on the location
of tools: ‘configure.ac’ files have AC_PROG_SED, AC_PATH_PROG, and
similar calls.

> On your GUIX systems, do you have a reasonably populated /bin?  Ditto
> the other three.

No.  By default, one gets /bin/sh, and nothing more.

This is a radical departure from Unix tradition, but (1) GNU’s Not Unix
;-), and (2) the lack of a global name space gives rise to many
interesting properties: transactional upgrades and rollbacks are now
possible, per-user profiles, on-the-fly environments with ‘guix
environment’, etc.

>   (packages
>     (list (specification->package "nss-certs")))

Note that this is bogus, due to
<https://issues.guix.gnu.org/issue/35541>, which will be fixed in
Guix 1.0.1 in the coming days.

You should write:

  (packages (append (list (specification->package "nss-certs"))
                    %base-packages))

to get the full set of “base packages” (Coreutils, grep, etc.) in $PATH
by default.

Once you’ve made that change, you have to reconfigure your system:

  guix system reconfigure /etc/config.scm

HTH!

Thanks,
Ludo’.

bug#35691: remote configure failure due to bad pathnames for crt*.o

[Nelson H. F. Beebe]

OK, I've made this change to /etc/config.scm

;; Old clause from GUIX 1.0 installation
;;  (packages
;;    (list (specification->package "nss-certs")))
;; New clause suggested in e-mail from Ludo Cortes Wed, 15 May 2019 18:29:06 +0200
  (packages (append (list (specification->package "nss-certs"))
                    %base-packages))

and then ran

	% guix remove gccgo gcc-objc gcc-objc++ gcc-bootstrap
	% guix package --list-installed | grep gcc
	gcc-toolchain   9.1.0   out     /gnu/store/aqv6njg05w2yqh8krgrjy0wcbfxb4nn3-gcc-toolchain-9.1.0

	% sudo -s
	# guix system reconfigure /etc/config.scm
	# reboot	

Even after that major update, and reboot, I still find that, from a
remote ssh connection, configure fails to find a C compiler, because
of the faulty construction of the path to crt1.o.

Can you folks reproduce that behavior at your end with the simple test
bundle that I put up at

	http://www.math.utah.edu/~beebe/guix-bug.tar.gz

?


-------------------------------------------------------------------------------
- Nelson H. F. Beebe                    Tel: +1 801 581 5254                  -
- University of Utah                    FAX: +1 801 581 4148                  -
- Department of Mathematics, 110 LCB    Internet e-mail: beebe@math.utah.edu  -
- 155 S 1400 E RM 233                       beebe@acm.org  beebe@computer.org -
- Salt Lake City, UT 84112-0090, USA    URL: http://www.math.utah.edu/~beebe/ -
-------------------------------------------------------------------------------

bug#35691: remote configure failure due to bad pathnames for crt*.o

[Ricardo Wurmus]

Hi Nelson,

> Even after that major update, and reboot, I still find that, from a
> remote ssh connection, configure fails to find a C compiler, because
> of the faulty construction of the path to crt1.o.

Earlier you wrote this

    there is still a gcc visible: /u/sy/beebe/.guix-profile/bin/gcc.

Where does that come from?  Are you activating this profile in some
shell initialisation file?  Does that profile contain “gcc-toolchain” or
“gcc”?

What does PATH look like?

You can check with “guix package -p /u/sy/beebe/.guix-profile -l”.

(Or is $HOME equivalent to /u/sy/beebe on this system and the profile is
thus the default?)

I’m interested in the activation of that profile.  We recommend adding
these lines to ~/.bash_profile (not .bashrc) in the manual:

   export GUIX_PROFILE=$HOME/.guix-profile
   source $GUIX_PROFILE/etc/profile

(Leaving out the definition of GUIX_PROFILE will change the behaviour
slightly and you would end up with potentially outdated absolute store
file names in your PATH.)

--
Ricardo

bug#35691: remote configure failure due to bad pathnames for crt*.o

[Ludovic Courtès]

Hi Nelson,

"Nelson H. F. Beebe" <beebe@math.utah.edu> skribis:

> Even after that major update, and reboot, I still find that, from a
> remote ssh connection, configure fails to find a C compiler, because
> of the faulty construction of the path to crt1.o.

I cannot reproduce the problem.  With ‘guix environment’, as show below,
I build an isolated environment that contains nothing but
‘gcc-toolchain’ and Bash:

--8<---------------cut here---------------start------------->8---
$ guix environment --ad-hoc -C gcc-toolchain
[env]$ echo 'int main() { }' > t.c
[env]$ gcc t.c
[env]$ ./a.out
--8<---------------cut here---------------end--------------->8---

As you can see, I can compile and link in that environment.

This environment defines PATH, CPATH, and LIBRARY_PATH appropriately:

--8<---------------cut here---------------start------------->8---
$ guix environment --ad-hoc -C gcc-toolchain grep coreutils
[env]$ env | grep PATH
CPATH=/gnu/store/26ng85mn5rs22cw654bvh62wh0yjx89h-profile/include
LIBRARY_PATH=/gnu/store/26ng85mn5rs22cw654bvh62wh0yjx89h-profile/lib
PATH=/gnu/store/26ng85mn5rs22cw654bvh62wh0yjx89h-profile/bin:/gnu/store/26ng85mn5rs22cw654bvh62wh0yjx89h-profile/sbin
--8<---------------cut here---------------end--------------->8---

Can you check the value of CPATH and LIBRARY_PATH in your user shell?

What does ‘guix package --search-paths’ return?

(See also the bits about search paths and ~/.guix-profile/etc/profile at
<https://gnu.org/software/guix/manual/en/html_node/Invoking-guix-package.html>.)

HTH,
Ludo’.

bug#35691: remote configure failure due to bad pathnames for crt*.o

[Giovanni Biscuolo]

[-- Attachment #1: Type: text/plain, Size: 1759 bytes --]

Hello Nelson and Ludovic,

AFAIU this is an environment issue 

Ludovic Courtès <ludo@gnu.org> writes:

> "Nelson H. F. Beebe" <beebe@math.utah.edu> skribis:
>
>> Even after that major update, and reboot, I still find that, from a
>> remote ssh connection

Nelson please inspect the output you remotely get from:

 remote-machine$ ssh guix10.vm.example.com 'env'

compare it with the one you locally get from:

 guix10.vm.example.com$ env

and compare it with the one you remotely get from

 remote-machine$ ssh guix10.vm.example.com 'bash -l -c "env"'

(the part before the $ prompt is a fictional hostname)

User environment - essential for Guix to work - is not passed
(permitted) usually by default via ssh; the easiest option is to wrap
your command in a remote login shell invocation; in your case should be:

 remote-machine$ ssh guix10.vm.example.com 'bash -l -c "cd FOO-x.y.z ; ./configure && make all check && make install"'

[...]

> I cannot reproduce the problem.  With ‘guix environment’, as show below,
> I build an isolated environment that contains nothing but
> ‘gcc-toolchain’ and Bash:
>
> --8<---------------cut here---------------start------------->8---
> $ guix environment --ad-hoc -C gcc-toolchain
> [env]$ echo 'int main() { }' > t.c
> [env]$ gcc t.c
> [env]$ ./a.out
> --8<---------------cut here---------------end--------------->8---
>
> As you can see, I can compile and link in that environment.

Nelson was also able to compile and link locally, he is just having
problems running compilation commands remotely via ssh

Do we need to document this on the manual, if not already documented?

HTH!
Gio'.

[...]

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]