bug#22013: python2 versions of packages failing as inputs

bug#22013: python2 versions of packages failing as inputs

[Efraim Flashner]

[-- Attachment #1.1: Type: text/plain, Size: 820 bytes --]

Guix build python2-cryptography builds fine, guix build python2-oauthlib
rebuilds python2-cryptography, and fails to build. Python2-cryptography has
an additional input of python2-ipaddress which doesn't get carried along when
python2-cryptography is an input and not the final build.

My guess is that package-with-python2 needs some tweaking. It looks like
python-oauthlib uses python-cryptography, python2-oauthlib uses
(package-with-python2 python-cryptography) instead of python2-cryptography.

I've enclosed `guix graph python2-cryptography python2-oauthlib` as a
visualization of the bug.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #1.2: graph2 --]
[-- Type: application/octet-stream, Size: 5732 bytes --]

digraph "Guix package" {
  "56124544" [label = "python2-oauthlib-1.0.3", shape = box, fontname = Helvetica];
  "56124544" -> "57090240" [color = red];
  "56124544" -> "56350400" [color = red];
  "56124544" -> "58765056" [color = red];
  "56124544" -> "57091200" [color = red];
  "56124544" -> "67195648" [color = red];
  "56124544" -> "54476800" [color = red];
  "56124544" -> "56125696" [color = red];
  "56124544" -> "58763520" [color = red];
  "57090240" [label = "python2-setuptools-18.3.1", shape = box, fontname = Helvetica];
  "56350400" [label = "python2-coverage-3.7.1", shape = box, fontname = Helvetica];
  "56350400" -> "57090240" [color = red];
  "58765056" [label = "python2-nose-1.3.4", shape = box, fontname = Helvetica];
  "58765056" -> "57090240" [color = red];
  "57091200" [label = "python2-mock-1.0.1", shape = box, fontname = Helvetica];
  "67195648" [label = "python2-blinker-1.3", shape = box, fontname = Helvetica];
  "67195648" -> "57090240" [color = red];
  "54476800" [label = "python2-cryptography-1.0.2", shape = box, fontname = Helvetica];
  "54476800" -> "54478720" [color = red];
  "54476800" -> "57090240" [color = red];
  "54476800" -> "54479872" [color = red];
  "54476800" -> "60484992" [color = red];
  "54476800" -> "35188480" [color = red];
  "54476800" -> "58762752" [color = red];
  "54476800" -> "55629120" [color = red];
  "54476800" -> "56123968" [color = red];
  "54476800" -> "80761600" [color = red];
  "54476800" -> "57092544" [color = red];
  "54476800" -> "54692672" [color = red];
  "54478720" [label = "python2-cryptography-vectors-1.0.2", shape = box, fontname = Helvetica];
  "54478720" -> "57090240" [color = red];
  "54479872" [label = "python2-pretend-1.0.8", shape = box, fontname = Helvetica];
  "54479872" -> "57090240" [color = red];
  "60484992" [label = "python2-pytest-2.6.1", shape = box, fontname = Helvetica];
  "60484992" -> "57090240" [color = red];
  "60484992" -> "58761984" [color = red];
  "60484992" -> "58765056" [color = red];
  "60484992" -> "57091200" [color = red];
  "58761984" [label = "python2-py-1.4.23", shape = box, fontname = Helvetica];
  "58761984" -> "57090240" [color = red];
  "35188480" [label = "openssl-1.0.2d", shape = box, fontname = Helvetica];
  "35188480" -> "37608448" [color = red];
  "37608448" [label = "perl-5.16.1", shape = box, fontname = Helvetica];
  "58762752" [label = "python2-cffi-1.2.1", shape = box, fontname = Helvetica];
  "58762752" -> "44623936" [color = red];
  "58762752" -> "67198528" [color = red];
  "58762752" -> "60484992" [color = red];
  "58762752" -> "57090240" [color = red];
  "58762752" -> "44622592" [color = red];
  "58762752" -> "65499136" [color = red];
  "44623936" [label = "pkg-config-0.28", shape = box, fontname = Helvetica];
  "67198528" [label = "python2-sphinx-1.2.3", shape = box, fontname = Helvetica];
  "67198528" -> "57090240" [color = red];
  "67198528" -> "35201408" [color = red];
  "67198528" -> "65500672" [color = red];
  "67198528" -> "65499712" [color = red];
  "35201408" [label = "python2-jinja2-2.7.3", shape = box, fontname = Helvetica];
  "35201408" -> "57090240" [color = red];
  "35201408" -> "35202368" [color = red];
  "35202368" [label = "python2-markupsafe-0.23", shape = box, fontname = Helvetica];
  "35202368" -> "57090240" [color = red];
  "65500672" [label = "python2-docutils-0.12", shape = box, fontname = Helvetica];
  "65500672" -> "57090240" [color = red];
  "65499712" [label = "python2-pygments-1.6", shape = box, fontname = Helvetica];
  "65499712" -> "57090240" [color = red];
  "44622592" [label = "libffi-3.1", shape = box, fontname = Helvetica];
  "65499136" [label = "python2-pycparser-2.10", shape = box, fontname = Helvetica];
  "65499136" -> "44623936" [color = red];
  "65499136" -> "57090240" [color = red];
  "55629120" [label = "python2-six-1.9.0", shape = box, fontname = Helvetica];
  "55629120" -> "57090240" [color = red];
  "56123968" [label = "python2-pyasn1-0.1.8", shape = box, fontname = Helvetica];
  "80761600" [label = "python2-enum34-1.0", shape = box, fontname = Helvetica];
  "80761600" -> "57090240" [color = red];
  "57092544" [label = "python2-idna-2.0", shape = box, fontname = Helvetica];
  "57092544" -> "57090240" [color = red];
  "54692672" [label = "python2-iso8601-0.1.10", shape = box, fontname = Helvetica];
  "54692672" -> "57090240" [color = red];
  "56125696" [label = "python2-pyjwt-1.4.0", shape = box, fontname = Helvetica];
  "56125696" -> "57090240" [color = red];
  "56125696" -> "60482112" [color = red];
  "60482112" [label = "python2-pytest-runner-2.6.2", shape = box, fontname = Helvetica];
  "60482112" -> "60484992" [color = red];
  "60482112" -> "57090240" [color = red];
  "60482112" -> "55628544" [color = red];
  "55628544" [label = "python2-setuptools-scm-1.9.0", shape = box, fontname = Helvetica];
  "55628544" -> "57090240" [color = red];
  "58763520" [label = "python2-unittest2-0.5.1", shape = box, fontname = Helvetica];
  "58763520" -> "57090240" [color = red];
  "35204672" [label = "python2-cryptography-1.0.2", shape = box, fontname = Helvetica];
  "35204672" -> "54478720" [color = red];
  "35204672" -> "57090240" [color = red];
  "35204672" -> "54479872" [color = red];
  "35204672" -> "60484992" [color = red];
  "35204672" -> "35188480" [color = red];
  "35204672" -> "58762752" [color = red];
  "35204672" -> "55629120" [color = red];
  "35204672" -> "56123968" [color = red];
  "35204672" -> "80761600" [color = red];
  "35204672" -> "57092544" [color = red];
  "35204672" -> "54692672" [color = red];
  "35204672" -> "56366976" [color = red];
  "56366976" [label = "python2-ipaddress-1.0.14", shape = box, fontname = Helvetica];
  "56366976" -> "57090240" [color = red];

}

[-- Attachment #1.3: graph2.pdf --]
[-- Type: application/pdf, Size: 16363 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

bug#22013: python2 versions of packages failing as inputs

[Ricardo Wurmus]

Efraim Flashner <efraim@flashner.co.il> writes:

> Guix build python2-cryptography builds fine, guix build python2-oauthlib
> rebuilds python2-cryptography, and fails to build. Python2-cryptography has
> an additional input of python2-ipaddress which doesn't get carried along when
> python2-cryptography is an input and not the final build.

I think that’s pretty normal.  We have the same situation in
‘python2-openssl’ where we have to explicitly add ‘python2-cryptography’
and remove ‘python-cryptography’.  This is also why any python2 variant
of a package with ‘python-numpy’ among its inputs has to be treated
specially.

It would, of course, be nice if package-with-python2 could handle this
automatically.

~~ Ricardo

bug#22013: python2 versions of packages failing as inputs

[Ludovic Courtès]

Ricardo Wurmus <rekado@elephly.net> skribis:

> Efraim Flashner <efraim@flashner.co.il> writes:
>
>> Guix build python2-cryptography builds fine, guix build python2-oauthlib
>> rebuilds python2-cryptography, and fails to build. Python2-cryptography has
>> an additional input of python2-ipaddress which doesn't get carried along when
>> python2-cryptography is an input and not the final build.
>
> I think that’s pretty normal.  We have the same situation in
> ‘python2-openssl’ where we have to explicitly add ‘python2-cryptography’
> and remove ‘python-cryptography’.

Yes, and this is due to the fact that ‘python2-cryptography’ is not just
an automatically-translated package:

  (define-public python2-cryptography
    (let ((crypto (package-with-python2 python-cryptography)))
      (package (inherit crypto)
        (propagated-inputs
         `(("python2-ipaddress" ,python2-ipaddress)
           ,@(package-propagated-inputs crypto))))))

If it were simply equal to (package-with-python2 python-cryptography),
there would be no need for the manual tweak in ‘python-pyopenssl’.

Efraim, could you apply the same strategy as for ‘python2-pyopenssl’ in
‘python2-oauthlib’?

> It would, of course, be nice if package-with-python2 could handle this
> automatically.

I think we would need a way to register “translation” results.
Currently ‘package-with-python2’ internally uses a hash table to memoize
conversion results.  We could imagine adding something like:

  (register-python2-variant PY3VARIANT PY2VARIANT)

to explicitly add a pair to that hash table.

The downside of this approach is that this would break referential
transparency.

WDYT?

Thanks,
Ludo’.

bug#22013: python2 versions of packages failing as inputs

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 2741 bytes --]

On Thu, 26 Nov 2015 09:59:12 +0100
ludo@gnu.org (Ludovic Courtès) wrote:

> Ricardo Wurmus <rekado@elephly.net> skribis:
> 
> > Efraim Flashner <efraim@flashner.co.il> writes:
> >  
> >> Guix build python2-cryptography builds fine, guix build python2-oauthlib
> >> rebuilds python2-cryptography, and fails to build. Python2-cryptography has
> >> an additional input of python2-ipaddress which doesn't get carried along when
> >> python2-cryptography is an input and not the final build.  
> >
> > I think that’s pretty normal.  We have the same situation in
> > ‘python2-openssl’ where we have to explicitly add ‘python2-cryptography’
> > and remove ‘python-cryptography’.  
> 
> Yes, and this is due to the fact that ‘python2-cryptography’ is not just
> an automatically-translated package:
> 
>   (define-public python2-cryptography
>     (let ((crypto (package-with-python2 python-cryptography)))
>       (package (inherit crypto)
>         (propagated-inputs
>          `(("python2-ipaddress" ,python2-ipaddress)
>            ,@(package-propagated-inputs crypto))))))
> 
> If it were simply equal to (package-with-python2 python-cryptography),
> there would be no need for the manual tweak in ‘python-pyopenssl’.
> 
> Efraim, could you apply the same strategy as for ‘python2-pyopenssl’ in
> ‘python2-oauthlib’?

Ok, I understand now. I thought I had to edit python2-cryptography to fix
python2-oauthlib, now I realize everytime I use python2-cryptography I need
to make sure it pulls the right one.

> 
> > It would, of course, be nice if package-with-python2 could handle this
> > automatically.  
> 
> I think we would need a way to register “translation” results.
> Currently ‘package-with-python2’ internally uses a hash table to memoize
> conversion results.  We could imagine adding something like:
> 
>   (register-python2-variant PY3VARIANT PY2VARIANT)
> 
> to explicitly add a pair to that hash table.
> 
> The downside of this approach is that this would break referential
> transparency.
> 
> WDYT?
> 
> Thanks,
> Ludo’.

We already have the python- and python2- variants as the same packages, just
with the occasional additional input to make it build. Mapping the two
together might speed up package-with-python2.

I'm going to keep on looking at guix/build-systems/python.scm to see if I can
come up with something to not need to specifically declare python2-foo every
time python-foo ends up being used.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

bug#22013: python2 versions of packages failing as inputs

[Ludovic Courtès]

Efraim Flashner <efraim@flashner.co.il> skribis:

> On Thu, 26 Nov 2015 09:59:12 +0100
> ludo@gnu.org (Ludovic Courtès) wrote:
>
>> Ricardo Wurmus <rekado@elephly.net> skribis:
>> 
>> > Efraim Flashner <efraim@flashner.co.il> writes:
>> >  
>> >> Guix build python2-cryptography builds fine, guix build python2-oauthlib
>> >> rebuilds python2-cryptography, and fails to build. Python2-cryptography has
>> >> an additional input of python2-ipaddress which doesn't get carried along when
>> >> python2-cryptography is an input and not the final build.  
>> >
>> > I think that’s pretty normal.  We have the same situation in
>> > ‘python2-openssl’ where we have to explicitly add ‘python2-cryptography’
>> > and remove ‘python-cryptography’.  
>> 
>> Yes, and this is due to the fact that ‘python2-cryptography’ is not just
>> an automatically-translated package:
>> 
>>   (define-public python2-cryptography
>>     (let ((crypto (package-with-python2 python-cryptography)))
>>       (package (inherit crypto)
>>         (propagated-inputs
>>          `(("python2-ipaddress" ,python2-ipaddress)
>>            ,@(package-propagated-inputs crypto))))))
>> 
>> If it were simply equal to (package-with-python2 python-cryptography),
>> there would be no need for the manual tweak in ‘python-pyopenssl’.
>> 
>> Efraim, could you apply the same strategy as for ‘python2-pyopenssl’ in
>> ‘python2-oauthlib’?
>
> Ok, I understand now. I thought I had to edit python2-cryptography to fix
> python2-oauthlib, now I realize everytime I use python2-cryptography I need
> to make sure it pulls the right one.

AFAICS this was fixed in 0066de6, so I’m closing it.

>> > It would, of course, be nice if package-with-python2 could handle this
>> > automatically.  
>> 
>> I think we would need a way to register “translation” results.
>> Currently ‘package-with-python2’ internally uses a hash table to memoize
>> conversion results.  We could imagine adding something like:
>> 
>>   (register-python2-variant PY3VARIANT PY2VARIANT)
>> 
>> to explicitly add a pair to that hash table.
>> 
>> The downside of this approach is that this would break referential
>> transparency.
>> 
>> WDYT?

For this more general issue, we should open a different bug.

Thanks,
Ludo’.