bug#26949: Python path conflict

bug#26949: Python path conflict

[Pjotr Prins]

Note that I do most of my buiding with --no-grafts.

Not sure they are related:

Using the Guix tree from source 4a3495d57c08dff9287fe559482a6d2009109304

  ./pre-inst-env guix package -i python@2.7.13 

renders

Backtrace:
In ice-9/eval.scm:
 432: 19 [eval # #]
In ice-9/boot-9.scm:
2412: 18 [save-module-excursion #<procedure 1f8e900 at ice-9/boot-9.scm:4084:3 ()>]
4089: 17 [#<procedure 1f8e900 at ice-9/boot-9.scm:4084:3 ()>]
1734: 16 [%start-stack load-stack ...]
1739: 15 [#<procedure 1fa0b70 ()>]
In unknown file:
   ?: 14 [primitive-load "/gnu/export/local/home/wrk/izip/git/opensource/genenetwork/gn-latest-guix/scripts/guix"]
In guix/ui.scm:
1257: 13 [run-guix-command package "-i" "python@2.7.13"]
In ice-9/boot-9.scm:
 160: 12 [catch srfi-34 #<procedure 2984020 at guix/ui.scm:442:2 ()> ...]
 160: 11 [catch system-error ...]
In guix/scripts/package.scm:
 898: 10 [#<procedure 2984080 at guix/scripts/package.scm:886:4 ()>]
 865: 9 [process-actions #<build-daemon 256.97 29acb80> (# # # # ...)]
In guix/ui.scm:
 713: 8 [show-manifest-transaction # # # ...]
In srfi/srfi-1.scm:
 598: 7 [map #<procedure upgrade-string (name old-version new-version output item)> ...]
In guix/ui.scm:
 689: 6 [upgrade-string "python" "3.5.2" ...]
In guix/packages.scm:
1174: 5 [package-output # # "out" ...]
 817: 4 [cache! #<weak-key-hash-table 2844460 352/443> # # ...]
In guix/grafts.scm:
 312: 3 [graft-derivation # # # ...]
 267: 2 [#<procedure 52d30a0 at guix/grafts.scm:267:2 (state)> #<vlist ()>]
 272: 1 [#<procedure 5ae7c60 at guix/grafts.scm:267:2 (t-1b71430ac-17aa)> #]
In unknown file:
   ?: 0 [# # #<procedure dependency-grafts (item)> #]

ERROR: In procedure #<syntax-transformer mapm>:
ERROR: Wrong type to apply: #<syntax-transformer mapm>

and

  ./pre-inst-env guix package --no-grafts -i python@2.7.13

renders

The following package will be downgraded:
   python	3.5.2 -> 2.7.13	/gnu/store/q5kdj7gpawi94pqd15x3wizjq0nx4zhx-python-2.7.13

The following derivation will be built:
   /gnu/store/hrl55dd3m9v8jqwf3jn5g0j36ix5n1n5-profile.drv
warning: collision encountered: /gnu/store/36nzwbcn09kd72aywi4idpasrlnpcphy-gdk-pixbuf+svg-2.34.0/share/gir-1.0/GdkPixbuf-2.0.gir /gnu/store/8mv63hjx5ba8f1mgqb7hkx6xmlfamaxp-gdk-pixbuf-2.32.3/share/gir-1.0/GdkPixbuf-2.0.gir 
warning: arbitrarily choosing /gnu/store/36nzwbcn09kd72aywi4idpasrlnpcphy-gdk-pixbuf+svg-2.34.0/share/gir-1.0/GdkPixbuf-2.0.gir
(...)
warning: collision encountered: /gnu/store/dag4djb807a57vb7s9kkj04bz7v4c3ji-mysql-5.7.13/share/man/man1/mysqltest.1.gz /gnu/store/vdvwj57w1rnay7khvi0c4wp05f35gqcl-mysql-5.6.25/share/man/man1/mysqltest.1.gz 
warning: arbitrarily choosing /gnu/store/dag4djb807a57vb7s9kkj04bz7v4c3ji-mysql-5.7.13/share/man/man1/mysqltest.1.gz
Backtrace:
In ice-9/boot-9.scm:
 160: 17 [catch #t #<catch-closure 8c2080> ...]
In unknown file:
   ?: 16 [apply-smob/1 #<catch-closure 8c2080>]
In ice-9/boot-9.scm:
  66: 15 [call-with-prompt prompt0 ...]
In ice-9/eval.scm:
 432: 14 [eval # #]
In ice-9/boot-9.scm:
2412: 13 [save-module-excursion #<procedure 8e67c0 at ice-9/boot-9.scm:4084:3 ()>]
4089: 12 [#<procedure 8e67c0 at ice-9/boot-9.scm:4084:3 ()>]
1734: 11 [%start-stack load-stack #<procedure 8f6de0 at ice-9/boot-9.scm:4080:10 ()>]
1739: 10 [#<procedure 8f86c0 ()>]
In unknown file:
   ?: 9 [primitive-load "/gnu/store/jcgmg7mnfzqhnwf8svrydcv67z13fpdb-profile-builder"]
In ./guix/build/profiles.scm:
 137: 8 [build-profile "/gnu/store/b5rq0sl2jsqpqjxkhi68dy3racz1wfpv-profile" # ...]
In unknown file:
   ?: 7 [hash-for-each #<procedure d5b150 at ./guix/build/union.scm:150:21 (file dirs-with-file)> ...]
   ?: 6 [hash-for-each #<procedure d4b120 at ./guix/build/union.scm:150:21 (file dirs-with-file)> ...]
   ?: 5 [hash-for-each #<procedure e25450 at ./guix/build/union.scm:150:21 (file dirs-with-file)> ...]
   ?: 4 [hash-for-each #<procedure ec02a0 at ./guix/build/union.scm:150:21 (file dirs-with-file)> ...]
In ./guix/build/union.scm:
 117: 3 [union "/gnu/store/b5rq0sl2jsqpqjxkhi68dy3racz1wfpv-profile/share/man/man1/python.1" ...]
In unknown file:
   ?: 2 [partition #<procedure file-is-directory? (file)> #]
In ./guix/build/union.scm:
  50: 1 [file-is-directory? "/gnu/store/q5kdj7gpawi94pqd15x3wizjq0nx4zhx-python-2.7.13/share/man/man1/python.1"]
In unknown file:
   ?: 0 [stat "/gnu/store/q5kdj7gpawi94pqd15x3wizjq0nx4zhx-python-2.7.13/share/man/man1/python.1" ...]

ERROR: In procedure stat:
ERROR: In procedure stat: No such file or directory: "/gnu/store/q5kdj7gpawi94pqd15x3wizjq0nx4zhx-python-2.7.13/share/man/man1/python.1"
builder for `/gnu/store/hrl55dd3m9v8jqwf3jn5g0j36ix5n1n5-profile.drv' failed with exit code 1
guix package: error: build failed: build of `/gnu/store/hrl55dd3m9v8jqwf3jn5g0j36ix5n1n5-profile.drv' failed

-- 

bug#26949: Python path conflict

[Pjotr Prins]

The two errors are not related. When I choose a different target
profile the second error goes away. 

In a clean profile both errors will probably go away...

-- 

bug#26949: Python path conflict

[Ludovic Courtès]

Hi,

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> Note that I do most of my buiding with --no-grafts.

This is an unsafe thing to do.

> Using the Guix tree from source 4a3495d57c08dff9287fe559482a6d2009109304
>
>   ./pre-inst-env guix package -i python@2.7.13 
>
> renders

[...]

> In unknown file:
>    ?: 0 [# # #<procedure dependency-grafts (item)> #]
>
> ERROR: In procedure #<syntax-transformer mapm>:
> ERROR: Wrong type to apply: #<syntax-transformer mapm>

This is due to an ABI change a few weeks ago.  Just run “make clean-go
&& make”.

> and
>
>   ./pre-inst-env guix package --no-grafts -i python@2.7.13
>
> renders
>
> The following package will be downgraded:
>    python	3.5.2 -> 2.7.13	/gnu/store/q5kdj7gpawi94pqd15x3wizjq0nx4zhx-python-2.7.13
>
> The following derivation will be built:
>    /gnu/store/hrl55dd3m9v8jqwf3jn5g0j36ix5n1n5-profile.drv

[...]

> In ./guix/build/union.scm:
>   50: 1 [file-is-directory? "/gnu/store/q5kdj7gpawi94pqd15x3wizjq0nx4zhx-python-2.7.13/share/man/man1/python.1"]
> In unknown file:
>    ?: 0 [stat "/gnu/store/q5kdj7gpawi94pqd15x3wizjq0nx4zhx-python-2.7.13/share/man/man1/python.1" ...]
>
> ERROR: In procedure stat:
> ERROR: In procedure stat: No such file or directory: "/gnu/store/q5kdj7gpawi94pqd15x3wizjq0nx4zhx-python-2.7.13/share/man/man1/python.1"
> builder for `/gnu/store/hrl55dd3m9v8jqwf3jn5g0j36ix5n1n5-profile.drv' failed with exit code 1
> guix package: error: build failed: build of `/gnu/store/hrl55dd3m9v8jqwf3jn5g0j36ix5n1n5-profile.drv' failed

I can’t seem to reproduce it:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix package -p foo -i python@2.7
The following package will be installed:
   python	2.7.13	/gnu/store/p0ygwmwspjpn4pwnla3p26n0xrypzdxv-python-2.7.13

substitute: updating list of substitutes from 'https://bayfront.guixsd.org'... 100.0%
substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%
The following derivations will be built:
   /gnu/store/x0pffs6dzzzrajvhajicwgw6aj0g34zl-profile.drv
   /gnu/store/xi5js16n6f9v5vr4cdqsqnh9vcm3yppn-manual-database.drv
substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%
Creating manual page database for 1 packages... done in 0.015 s
find-files: /gnu/store/p0ygwmwspjpn4pwnla3p26n0xrypzdxv-python-2.7.13/share/man/man1/python.1: No such file or directory
1 package in profile
The following environment variable definitions may be needed:
   export PATH="foo/bin${PATH:+:}$PATH"
   export PYTHONPATH="foo/lib/python2.7/site-packages${PYTHONPATH:+:}$PYTHONPATH"
$ git describe
v0.12.0-3905-g516bfed7a
--8<---------------cut here---------------end--------------->8---

Could you check if the problem occurs with current Guix, and if it does,
whether the above command is easy to reproduce it?

Thanks for reporting it!

Ludo’.

bug#26949: Python path conflict

[Pjotr Prins]

On Wed, May 17, 2017 at 03:08:05PM +0200, Ludovic Court??s wrote:
> Hi,
> 
> Pjotr Prins <pjotr.public12@thebird.nl> skribis:
> 
> > Note that I do most of my buiding with --no-grafts.
> 
> This is an unsafe thing to do.

Depends on the machine and what it is used for. Yes, for services that
can be vulnerable you need to use grafts using an older guix tree. But
I mostly build against git checkouts of guix - which means I am using
the latest versions of packages anyway. Agree?

> 
> This is due to an ABI change a few weeks ago.  Just run ???make clean-go
> && make???.

clean-go is a new one for me.

> 
> > In ./guix/build/union.scm:
> >   50: 1 [file-is-directory? "/gnu/store/q5kdj7gpawi94pqd15x3wizjq0nx4zhx-python-2.7.13/share/man/man1/python.1"]
> > In unknown file:
> >    ?: 0 [stat "/gnu/store/q5kdj7gpawi94pqd15x3wizjq0nx4zhx-python-2.7.13/share/man/man1/python.1" ...]
> >
> > ERROR: In procedure stat:
> > ERROR: In procedure stat: No such file or directory: "/gnu/store/q5kdj7gpawi94pqd15x3wizjq0nx4zhx-python-2.7.13/share/man/man1/python.1"
> > builder for `/gnu/store/hrl55dd3m9v8jqwf3jn5g0j36ix5n1n5-profile.drv' failed with exit code 1
> > guix package: error: build failed: build of `/gnu/store/hrl55dd3m9v8jqwf3jn5g0j36ix5n1n5-profile.drv' failed
> 
> I can???t seem to reproduce it:
 
> --8<---------------cut here---------------start------------->8---
> $ ./pre-inst-env guix package -p foo -i python@2.7
> The following package will be installed:
>    python	2.7.13	/gnu/store/p0ygwmwspjpn4pwnla3p26n0xrypzdxv-python-2.7.13
> 
> substitute: updating list of substitutes from 'https://bayfront.guixsd.org'... 100.0%
> substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%
> The following derivations will be built:
>    /gnu/store/x0pffs6dzzzrajvhajicwgw6aj0g34zl-profile.drv
>    /gnu/store/xi5js16n6f9v5vr4cdqsqnh9vcm3yppn-manual-database.drv
> substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%
> Creating manual page database for 1 packages... done in 0.015 s
> find-files: /gnu/store/p0ygwmwspjpn4pwnla3p26n0xrypzdxv-python-2.7.13/share/man/man1/python.1: No such file or directory
> 1 package in profile
> The following environment variable definitions may be needed:
>    export PATH="foo/bin${PATH:+:}$PATH"
>    export PYTHONPATH="foo/lib/python2.7/site-packages${PYTHONPATH:+:}$PYTHONPATH"
> $ git describe
> v0.12.0-3905-g516bfed7a
> --8<---------------cut here---------------end--------------->8---
> 
> Could you check if the problem occurs with current Guix, and if it does,
> whether the above command is easy to reproduce it?

I am using current Guix. The problem occurs on an existing profile. On
a new profile it does not happen.

I remember the python.1 path is a symlink not pointing anywhere. We
dealt with that a year ago. Maybe it now confuses the man db builder.
Does it somehow use the (existing) profile?

Note that python-2.7.3 is the newly installed package.

Pj.

bug#26949: Python path conflict

[Ludovic Courtès]

Hello,

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> On Wed, May 17, 2017 at 03:08:05PM +0200, Ludovic Court??s wrote:
>> Hi,
>> 
>> Pjotr Prins <pjotr.public12@thebird.nl> skribis:
>> 
>> > Note that I do most of my buiding with --no-grafts.
>> 
>> This is an unsafe thing to do.
>
> Depends on the machine and what it is used for. Yes, for services that
> can be vulnerable you need to use grafts using an older guix tree. But
> I mostly build against git checkouts of guix - which means I am using
> the latest versions of packages anyway. Agree?

Yes, sure; I just wanted to make sure you had check the “I understand
the risks” box.  ;-)

>> > In ./guix/build/union.scm:
>> >   50: 1 [file-is-directory? "/gnu/store/q5kdj7gpawi94pqd15x3wizjq0nx4zhx-python-2.7.13/share/man/man1/python.1"]
>> > In unknown file:
>> >    ?: 0 [stat "/gnu/store/q5kdj7gpawi94pqd15x3wizjq0nx4zhx-python-2.7.13/share/man/man1/python.1" ...]
>> >
>> > ERROR: In procedure stat:
>> > ERROR: In procedure stat: No such file or directory: "/gnu/store/q5kdj7gpawi94pqd15x3wizjq0nx4zhx-python-2.7.13/share/man/man1/python.1"
>> > builder for `/gnu/store/hrl55dd3m9v8jqwf3jn5g0j36ix5n1n5-profile.drv' failed with exit code 1
>> > guix package: error: build failed: build of `/gnu/store/hrl55dd3m9v8jqwf3jn5g0j36ix5n1n5-profile.drv' failed
>> 
>> I can???t seem to reproduce it:
>  
>> --8<---------------cut here---------------start------------->8---
>> $ ./pre-inst-env guix package -p foo -i python@2.7
>> The following package will be installed:
>>    python	2.7.13	/gnu/store/p0ygwmwspjpn4pwnla3p26n0xrypzdxv-python-2.7.13
>> 
>> substitute: updating list of substitutes from 'https://bayfront.guixsd.org'... 100.0%
>> substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%
>> The following derivations will be built:
>>    /gnu/store/x0pffs6dzzzrajvhajicwgw6aj0g34zl-profile.drv
>>    /gnu/store/xi5js16n6f9v5vr4cdqsqnh9vcm3yppn-manual-database.drv
>> substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%
>> Creating manual page database for 1 packages... done in 0.015 s
>> find-files: /gnu/store/p0ygwmwspjpn4pwnla3p26n0xrypzdxv-python-2.7.13/share/man/man1/python.1: No such file or directory
>> 1 package in profile
>> The following environment variable definitions may be needed:
>>    export PATH="foo/bin${PATH:+:}$PATH"
>>    export PYTHONPATH="foo/lib/python2.7/site-packages${PYTHONPATH:+:}$PYTHONPATH"
>> $ git describe
>> v0.12.0-3905-g516bfed7a
>> --8<---------------cut here---------------end--------------->8---
>> 
>> Could you check if the problem occurs with current Guix, and if it does,
>> whether the above command is easy to reproduce it?
>
> I am using current Guix. The problem occurs on an existing profile. On
> a new profile it does not happen.
>
> I remember the python.1 path is a symlink not pointing anywhere. We
> dealt with that a year ago. Maybe it now confuses the man db builder.
> Does it somehow use the (existing) profile?

Yes, it could be that it happens with a particular combination of
packages in the profile.

Could you try to see if you could reproduce the problem in a fresh
profile starting from what the offending profile has and gradually
reducing the package set?

Thanks,
Ludo’.

bug#26949: Python path conflict

[Pjotr Prins]

On Wed, May 17, 2017 at 09:54:07PM +0200, Ludovic Court??s wrote:
> > I remember the python.1 path is a symlink not pointing anywhere. We
> > dealt with that a year ago. Maybe it now confuses the man db builder.
> > Does it somehow use the (existing) profile?
> 
> Yes, it could be that it happens with a particular combination of
> packages in the profile.
> 
> Could you try to see if you could reproduce the problem in a fresh
> profile starting from what the offending profile has and gradually
> reducing the package set?

I can reproduce the problem, but I can not reproduce the environment.
I tried.

All I can see is that when the profile gets updated the new path

  $profile/share/man/man1/python.1

is a symlink pointing to nothing. You can see that when installing
python-2.7.13.

All I can tell is that the version that was installed (2.7.11).
actually pointed to something. When overwriting the old version,
strangely, the man pages generator balks.

Strangely because installing in a fresh profile gives no complaints.

The old package path was

  /gnu/store/vcx1n5nj4gr52xx5m6gvi7zrwngy06s3-python-2.7.11/share/man/man1/python.1

Pj.

-- 

bug#26949: Python path conflict

[Ludovic Courtès]

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> In ./guix/build/union.scm:
>  117: 3 [union "/gnu/store/b5rq0sl2jsqpqjxkhi68dy3racz1wfpv-profile/share/man/man1/python.1" ...]
> In unknown file:
>    ?: 2 [partition #<procedure file-is-directory? (file)> #]
> In ./guix/build/union.scm:
>   50: 1 [file-is-directory? "/gnu/store/q5kdj7gpawi94pqd15x3wizjq0nx4zhx-python-2.7.13/share/man/man1/python.1"]
> In unknown file:
>    ?: 0 [stat "/gnu/store/q5kdj7gpawi94pqd15x3wizjq0nx4zhx-python-2.7.13/share/man/man1/python.1" ...]
>
> ERROR: In procedure stat:
> ERROR: In procedure stat: No such file or directory: "/gnu/store/q5kdj7gpawi94pqd15x3wizjq0nx4zhx-python-2.7.13/share/man/man1/python.1"

Ooh, got it.  I managed to reproduce it with a toy example.  Should be
fixed in 22ef06b801b284760b4ffd9587ea1a3dffd31baa.  Can you confirm?

Thank you!

Ludo’.

bug#26949: Python path conflict

[Pjotr Prins]

On Thu, May 18, 2017 at 11:43:31AM +0200, Ludovic Court??s wrote:
> Ooh, got it.  I managed to reproduce it with a toy example.  Should be
> fixed in 22ef06b801b284760b4ffd9587ea1a3dffd31baa.  Can you confirm?

Fixed!

Pj.