bug#36813: 'lower-gexp' does not respect 'system' or 'target' keywords

bug#36813: 'lower-gexp' does not respect 'system' or 'target' keywords

[Jakob L. Kreuze]

[-- Attachment #1.1: Type: text/plain, Size: 864 bytes --]

Hi,

I believe there is an issue with 'lower-gexp'. Running the following
snippet to lower a G-Expression for "i686-linux" yields output that
references store paths built for x86_64. In this case, the Guile
interpreter used is an x86_64 binary.

#+BEGIN_SRC scheme
(define (display-exp exp)
  (mlet* %store-monad ((lowered (lower-gexp exp
                                            #:system "i686-linux"
                                            #:target "i686-linux"))
                       (to-build -> (cons (lowered-gexp-guile lowered)
                                          (lowered-gexp-inputs lowered)))
                       (_ (built-derivations to-build)))
    (return (format #t "~a~%" (lowered-gexp-sexp lowered)))))

(with-store store
  (run-with-store store
    (display-exp #~(primitive-load #$(switch-system-program %system)))))
#+END_SRC


[-- Attachment #1.2: Command-line output. --]
[-- Type: text/plain, Size: 663 bytes --]

jakob@Epsilon ~ $ guile ~/test.scm 
(primitive-load /gnu/store/v7v1b7375j9j82dvfycv56v36nv5jq3y-switch-to-system.scm)
jakob@Epsilon ~ $ cat /gnu/store/v7v1b7375j9j82dvfycv56v36nv5jq3y-switch-to-system.scm
#!/gnu/store/9alic3caqhay3h8mx4iihpmyj6ymqpcx-guile-2.2.4/bin/guile --no-auto-compile
!#
...
jakob@Epsilon ~ $ file /gnu/store/9alic3caqhay3h8mx4iihpmyj6ymqpcx-guile-2.2.4/bin/guile
/gnu/store/9alic3caqhay3h8mx4iihpmyj6ymqpcx-guile-2.2.4/bin/guile: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, not stripped

[-- Attachment #1.3: Type: text/plain, Size: 309 bytes --]


This doesn't seem to be an issue if '%current-system' is parameterized,
as in the following.

#+BEGIN_SRC scheme
(parameterize ((%current-system "i686-linux"))
  (with-store store
    (run-with-store store
      (display-exp #~(primitive-load #$(switch-system-program %system))))))
#+END_SRC

Regards,
Jakob

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

bug#36813: 'lower-gexp' does not respect 'system' or 'target' keywords

[Ludovic Courtès]

Hello Jakob,

zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) skribis:

> I believe there is an issue with 'lower-gexp'. Running the following
> snippet to lower a G-Expression for "i686-linux" yields output that
> references store paths built for x86_64. In this case, the Guile
> interpreter used is an x86_64 binary.
>
> #+BEGIN_SRC scheme
> (define (display-exp exp)
>   (mlet* %store-monad ((lowered (lower-gexp exp
>                                             #:system "i686-linux"
>                                             #:target "i686-linux"))
>                        (to-build -> (cons (lowered-gexp-guile lowered)
>                                           (lowered-gexp-inputs lowered)))
>                        (_ (built-derivations to-build)))
>     (return (format #t "~a~%" (lowered-gexp-sexp lowered)))))
>
> (with-store store
>   (run-with-store store
>     (display-exp #~(primitive-load #$(switch-system-program %system)))))
> #+END_SRC

Note: #:target must be a “GNU triplet” like “arm-linux-gnueabihf”, not a
system type like “i686-linux”.

> jakob@Epsilon ~ $ guile ~/test.scm 
> (primitive-load /gnu/store/v7v1b7375j9j82dvfycv56v36nv5jq3y-switch-to-system.scm)
> jakob@Epsilon ~ $ cat /gnu/store/v7v1b7375j9j82dvfycv56v36nv5jq3y-switch-to-system.scm
> #!/gnu/store/9alic3caqhay3h8mx4iihpmyj6ymqpcx-guile-2.2.4/bin/guile --no-auto-compile
> !#
> ...
> jakob@Epsilon ~ $ file /gnu/store/9alic3caqhay3h8mx4iihpmyj6ymqpcx-guile-2.2.4/bin/guile
> /gnu/store/9alic3caqhay3h8mx4iihpmyj6ymqpcx-guile-2.2.4/bin/guile: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, not stripped

That’s because the Guile used here comes from the #:guile-for-build
parameter.

So the caller is responsible for doing the right thing here.  In fact,
if you do:

  (lower-gexp exp #:system whatever #:guile-for-build #f)

it will automatically take care of computing the right Guile for this
system.  For consistency, I don’t think we should change the default,
though.

WDYT?

Thanks,
Ludo’.

bug#36813: 'lower-gexp' does not respect 'system' or 'target' keywords

[Jakob L. Kreuze]

[-- Attachment #1: Type: text/plain, Size: 1044 bytes --]

Hi Ludo,

Ludovic Courtès <ludo@gnu.org> writes:

> Note: #:target must be a “GNU triplet” like “arm-linux-gnueabihf”, not
> a system type like “i686-linux”.

Thanks, is 'nix-system->gnu-triplet' the suggested way of obtaining the
triplet for a system?

> That’s because the Guile used here comes from the #:guile-for-build
> parameter.
>
> So the caller is responsible for doing the right thing here.  In fact,
> if you do:
>
>   (lower-gexp exp #:system whatever #:guile-for-build #f)
>
> it will automatically take care of computing the right Guile for this
> system.  For consistency, I don’t think we should change the default,
> though.
>
> WDYT?

Changing it to the following:

#+BEGIN_SRC scheme
(lower-gexp exp
            #:system "i686-linux"
            #:target "i686-unknown-linux-gnu"
            #:guile-for-build #f)
#+END_SRC

The Guile used is still a 64-bit LSB executable. Similarly, the
<operating-system> that's ungexp'd has a profile containing x86_64
executables.


[-- Attachment #2: output --]
[-- Type: text/plain, Size: 1347 bytes --]

jakob@Epsilon ~ $ readlink /gnu/store/6z5hdxjr8db4qm4d578lly3l87mlgkpv-system/profile/bin/* | xargs file
/gnu/store/5s2nib1lrd2101bbrivcl17kjx1mspw6-coreutils-8.30/bin/[:                 ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, not stripped
/gnu/store/9y5cvqnincp2ax5kxyv43zr7gdd89vs2-man-db-2.8.5/bin/apropos:             symbolic link to whatis
/gnu/store/lm3i15cvw4ybsnf2lsam5nj76kqbjg2k-libtasn1-4.13/bin/asn1Coding:         ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, not stripped
/gnu/store/lm3i15cvw4ybsnf2lsam5nj76kqbjg2k-libtasn1-4.13/bin/asn1Decoding:       ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, not stripped
/gnu/store/lm3i15cvw4ybsnf2lsam5nj76kqbjg2k-libtasn1-4.13/bin/asn1Parser:         ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, not stripped
...

[-- Attachment #3: Type: text/plain, Size: 16 bytes --]


Regards,
Jakob

bug#36813: 'lower-gexp' does not respect 'system' or 'target' keywords

[Ludovic Courtès]

zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) skribis:

> Changing it to the following:
>
> #+BEGIN_SRC scheme
> (lower-gexp exp
>             #:system "i686-linux"
>             #:target "i686-unknown-linux-gnu"
>             #:guile-for-build #f)
> #+END_SRC
>
> The Guile used is still a 64-bit LSB executable.

I can’t reproduce it on current ‘master’:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,run-in-store (lower-gexp #~(+ 2 3) #:guile-for-build #f #:system "mips64el-linux")
$6 = #<<lowered-gexp> sexp: (+ 2 3) inputs: () sources: () guile: #<<derivation-input> drv: #<derivation /gnu/store/jsnihqsz6nxwv88pr41i2y3403f959cf-guile-2.2.4.drv => /gnu/store/n2570pg8cahc8k9iqrg5qngyzf6j0xzr-guile-2.2.4-debug /gnu/store/j9a8dx25cj045yl5l32ajkkjf92ib3y1-guile-2.2.4 46e3230> sub-derivations: ("out")> load-path: () load-compiled-path: ()>
scheme@(guile-user)> (derivation-system (derivation-input-derivation (lowered-gexp-guile $6)))
$7 = "mips64el-linux"
--8<---------------cut here---------------end--------------->8---

Not even when cross-compiling (BTW, #:target probably doesn’t make sense
above, since it’s cross-compiling from i686 to i686):

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,run-in-store (lower-gexp #~(+ 2 3) #:guile-for-build #f #:system "mips64el-linux" #:target "i586-pc-gnu")
$8 = #<<lowered-gexp> sexp: (+ 2 3) inputs: () sources: () guile: #<<derivation-input> drv: #<derivation /gnu/store/jsnihqsz6nxwv88pr41i2y3403f959cf-guile-2.2.4.drv => /gnu/store/n2570pg8cahc8k9iqrg5qngyzf6j0xzr-guile-2.2.4-debug /gnu/store/j9a8dx25cj045yl5l32ajkkjf92ib3y1-guile-2.2.4 46e3230> sub-derivations: ("out")> load-path: () load-compiled-path: ()>
scheme@(guile-user)> (derivation-system (derivation-input-derivation (lowered-gexp-guile $8)))
$9 = "mips64el-linux"
--8<---------------cut here---------------end--------------->8---

Am I missing something?

Thanks,
Ludo’.

bug#36813: 'lower-gexp' does not respect 'system' or 'target' keywords

[Jakob L. Kreuze]

[-- Attachment #1: Type: text/plain, Size: 2390 bytes --]

Could you please try to reproduce using the example I provided? The derivation itself is handled appropriately, but its references are not.

On July 26, 2019 11:28:11 AM EDT, "Ludovic Courtès" <ludo@gnu.org> wrote:
>zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) skribis:
>
>> Changing it to the following:
>>
>> #+BEGIN_SRC scheme
>> (lower-gexp exp
>>             #:system "i686-linux"
>>             #:target "i686-unknown-linux-gnu"
>>             #:guile-for-build #f)
>> #+END_SRC
>>
>> The Guile used is still a 64-bit LSB executable.
>
>I can’t reproduce it on current ‘master’:
>
>--8<---------------cut here---------------start------------->8---
>scheme@(guile-user)> ,run-in-store (lower-gexp #~(+ 2 3)
>#:guile-for-build #f #:system "mips64el-linux")
>$6 = #<<lowered-gexp> sexp: (+ 2 3) inputs: () sources: () guile:
>#<<derivation-input> drv: #<derivation
>/gnu/store/jsnihqsz6nxwv88pr41i2y3403f959cf-guile-2.2.4.drv =>
>/gnu/store/n2570pg8cahc8k9iqrg5qngyzf6j0xzr-guile-2.2.4-debug
>/gnu/store/j9a8dx25cj045yl5l32ajkkjf92ib3y1-guile-2.2.4 46e3230>
>sub-derivations: ("out")> load-path: () load-compiled-path: ()>
>scheme@(guile-user)> (derivation-system (derivation-input-derivation
>(lowered-gexp-guile $6)))
>$7 = "mips64el-linux"
>--8<---------------cut here---------------end--------------->8---
>
>Not even when cross-compiling (BTW, #:target probably doesn’t make
>sense
>above, since it’s cross-compiling from i686 to i686):
>
>--8<---------------cut here---------------start------------->8---
>scheme@(guile-user)> ,run-in-store (lower-gexp #~(+ 2 3)
>#:guile-for-build #f #:system "mips64el-linux" #:target "i586-pc-gnu")
>$8 = #<<lowered-gexp> sexp: (+ 2 3) inputs: () sources: () guile:
>#<<derivation-input> drv: #<derivation
>/gnu/store/jsnihqsz6nxwv88pr41i2y3403f959cf-guile-2.2.4.drv =>
>/gnu/store/n2570pg8cahc8k9iqrg5qngyzf6j0xzr-guile-2.2.4-debug
>/gnu/store/j9a8dx25cj045yl5l32ajkkjf92ib3y1-guile-2.2.4 46e3230>
>sub-derivations: ("out")> load-path: () load-compiled-path: ()>
>scheme@(guile-user)> (derivation-system (derivation-input-derivation
>(lowered-gexp-guile $8)))
>$9 = "mips64el-linux"
>--8<---------------cut here---------------end--------------->8---
>
>Am I missing something?
>
>Thanks,
>Ludo’.

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 2875 bytes --]

bug#36813: 'lower-gexp' does not respect 'system' or 'target' keywords

[Jakob L. Kreuze]

[-- Attachment #1: Type: text/plain, Size: 2805 bytes --]

Apologies, I should clarify -- the Guile for the S-Expression appears to be fine, but the Guile referenced in the shebang of the ungexp'd program-file does not reflect the system.

On July 26, 2019 11:37:20 AM EDT, "Jakob L. Kreuze" <zerodaysfordays.sdf.org@sdf.org> wrote:
>Could you please try to reproduce using the example I provided? The
>derivation itself is handled appropriately, but its references are not.
>
>On July 26, 2019 11:28:11 AM EDT, "Ludovic Courtès" <ludo@gnu.org>
>wrote:
>>zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) skribis:
>>
>>> Changing it to the following:
>>>
>>> #+BEGIN_SRC scheme
>>> (lower-gexp exp
>>>             #:system "i686-linux"
>>>             #:target "i686-unknown-linux-gnu"
>>>             #:guile-for-build #f)
>>> #+END_SRC
>>>
>>> The Guile used is still a 64-bit LSB executable.
>>
>>I can’t reproduce it on current ‘master’:
>>
>>--8<---------------cut here---------------start------------->8---
>>scheme@(guile-user)> ,run-in-store (lower-gexp #~(+ 2 3)
>>#:guile-for-build #f #:system "mips64el-linux")
>>$6 = #<<lowered-gexp> sexp: (+ 2 3) inputs: () sources: () guile:
>>#<<derivation-input> drv: #<derivation
>>/gnu/store/jsnihqsz6nxwv88pr41i2y3403f959cf-guile-2.2.4.drv =>
>>/gnu/store/n2570pg8cahc8k9iqrg5qngyzf6j0xzr-guile-2.2.4-debug
>>/gnu/store/j9a8dx25cj045yl5l32ajkkjf92ib3y1-guile-2.2.4 46e3230>
>>sub-derivations: ("out")> load-path: () load-compiled-path: ()>
>>scheme@(guile-user)> (derivation-system (derivation-input-derivation
>>(lowered-gexp-guile $6)))
>>$7 = "mips64el-linux"
>>--8<---------------cut here---------------end--------------->8---
>>
>>Not even when cross-compiling (BTW, #:target probably doesn’t make
>>sense
>>above, since it’s cross-compiling from i686 to i686):
>>
>>--8<---------------cut here---------------start------------->8---
>>scheme@(guile-user)> ,run-in-store (lower-gexp #~(+ 2 3)
>>#:guile-for-build #f #:system "mips64el-linux" #:target "i586-pc-gnu")
>>$8 = #<<lowered-gexp> sexp: (+ 2 3) inputs: () sources: () guile:
>>#<<derivation-input> drv: #<derivation
>>/gnu/store/jsnihqsz6nxwv88pr41i2y3403f959cf-guile-2.2.4.drv =>
>>/gnu/store/n2570pg8cahc8k9iqrg5qngyzf6j0xzr-guile-2.2.4-debug
>>/gnu/store/j9a8dx25cj045yl5l32ajkkjf92ib3y1-guile-2.2.4 46e3230>
>>sub-derivations: ("out")> load-path: () load-compiled-path: ()>
>>scheme@(guile-user)> (derivation-system (derivation-input-derivation
>>(lowered-gexp-guile $8)))
>>$9 = "mips64el-linux"
>>--8<---------------cut here---------------end--------------->8---
>>
>>Am I missing something?
>>
>>Thanks,
>>Ludo’.
>
>-- 
>Sent from my Android device with K-9 Mail. Please excuse my brevity.

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 3335 bytes --]

bug#36813: 'lower-gexp' does not respect 'system' or 'target' keywords

[Ludovic Courtès]

"Jakob L. Kreuze" <zerodaysfordays.sdf.org@sdf.org> skribis:

> Apologies, I should clarify -- the Guile for the S-Expression appears to be fine, but the Guile referenced in the shebang of the ungexp'd program-file does not reflect the system.

Got it now.  This is fixed by 2e8cabb8d630a8423e2e5a3bf150c1c0310b945d.

Thanks!

Ludo’.

bug#36813: 'lower-gexp' does not respect 'system' or 'target' keywords

[Jakob L. Kreuze]

[-- Attachment #1: Type: text/plain, Size: 495 bytes --]


Ludovic Courtès <ludo@gnu.org> writes:

> "Jakob L. Kreuze" <zerodaysfordays.sdf.org@sdf.org> skribis:
>
>> Apologies, I should clarify -- the Guile for the S-Expression appears to be fine, but the Guile referenced in the shebang of the ungexp'd program-file does not reflect the system.
>
> Got it now.  This is fixed by 2e8cabb8d630a8423e2e5a3bf150c1c0310b945d.
>
> Thanks!
>
> Ludo’.

Just had the opportunity to test it out -- works great! Thanks, Ludo!

Regards,
Jakob

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]