Question: wrap-program without #:sh can be ok even when cross-compiling?

Question: wrap-program without #:sh can be ok even when cross-compiling?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 2777 bytes --]

About <https://issues.guix.gnu.org/47869>
(the ‘wrap-program #:sh’ patch series):

While looking at guix/build/gnu-build-system.scm,
I noticed the 'patch-shebangs' phase, which is run
after the install phase. IIUC, this phase is for
changing shebangs ...

 #!/usr/bin/stuff

to ...

 #!/gnu/store/...-stuff-1.0/bin/stuff

IIUC, this phase tries to find the interpreter
in 'inputs'. So, if the original interpreter
was for SYSTEM, the new interpreter is for TARGET.

As such, when cross-compiling, when bash is present
in the inputs, when the patch-shebangs phase isn't
disabled and we use wrap-program without a #:sh
argument, even though wrap-program will use an
interpreter for SYSTEM, the patch-shebangs will
automatically correct it for us to an interpreter
for TARGET.

So, is the patch series in bug#47869 still useful?

  * if, for some reason, patch-shebangs? is #f,
    then we need to explicitely #:sh set --> patch useful

    patch-shebangs? doesn't seem to be set to #f anywhere though.

  * some build systems might not have a patch-shebangs phase
    (I'm not aware of any such build systems though) --> patch useful

  * explicit is better than implicit [citation needed, see e.g. python]
    --> patch useful

    The extra verbosity seems acceptable in build systems
    (see e.g. usage of wrap-program in
    guix/build/glib-or-gtk-build-system.scm
    and guix/build/python-build-system.scm).

    Dunno about package definitions though.

So I'd would say yes! But should we explicitely set
  #:sh (search-input-file inputs "bin/bash")
in package definitions?

Cons for explicit / pros for implicit (relying on patch-shebangs):

  * explicit form is a bit verbose
  * in the transition, many, many package definitions need to be adjusted.

Pros for explicit / cons for explicit:

  * by explicitely writing #:sh (search-input-file inputs "bin/bash"),
    it should be clear that bash-minimal (or bash) needs to be added
    to the package inputs

    (Note that I intend to separate 'inputs' from 'native-inputs' in
    build phases even when compiling natively. Haven't gotten around to
    trying it though, seems complicated ...)

    Alternative: write a linter checking that wrap-program is only used
    if "bash" (or "bash-minimal") is in the package inputs (native-inputs
    doesn't count here).

  * Using the explicit form is always correct. The implicit #:sh (which "bin/bash")
    is not always corrected by the patch-shebangs phase.

    Note that when cross-compiling, not setting #:sh and when bash
    is absent from inputs, the patch-shebang phase merely emits a warning
    (which can easily get lost in the noise) and _not_ an error.

Thoughts?

Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Question: wrap-program without #:sh can be ok even when cross-compiling?

[Leo Prikler]

I think we might want to export a utility procedure
    (patch-shebangs files inputs)
so that files used during build (e.g. configure, Makefile, etc.) can do
(patch-shebangs build-stuff native-inputs) and the rest implicitly gets
(patch-shebangs files inputs) during the patch-shebangs phase.  WDYT?

Re: Question: wrap-program without #:sh can be ok even when cross-compiling?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1586 bytes --]

Leo Prikler schreef op zo 06-06-2021 om 09:39 [+0200]:
> I think we might want to export a utility procedure
>     (patch-shebangs files inputs)

This procedure already exists, but is undocumented and
unexported. It is in (guix build gnu-build-system).

(define* (patch-shebangs #:key inputs outputs (patch-shebangs? #t)
                         #:allow-other-keys)
  [...])

> so that files used during build (e.g. configure, Makefile, etc.) can do
> (patch-shebangs build-stuff native-inputs) and the rest implicitly gets
> (patch-shebangs files inputs) during the patch-shebangs phase.  WDYT?

Looking at %standard-phases, we have

(define %standard-phases
  ;; Standard build phases, as a list of symbol/procedure pairs.
  [...]
    (phases [...] unpack bootstrap patch-usr-bin-file
            patch-source-shebangs configure patch-generated-file-shebangs
            build check install
            patch-shebangs [...]))

Here, patch-source-shebangs calls patch-shebang for every file
in the source code. As #:path is not set, $PATH is used. Thus,
when cross-compiling, native-inputs (+ some implicit inputs) is used,
and when compiling natively, the union of native-inputs and inputs (+
some implicit inputs) is used (*).

Thus, the files used during build (configure, Makefile, ...) already
get a ‘good’ interpreter. (Unless I'm mistaken, I didn't test this.)

(*) I looked into separating 'native-inputs' and 'inputs' even when compiling
    natively but it turned out to be more complicated than first expected.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Question: wrap-program without #:sh can be ok even when cross-compiling?

[Leo Prikler]

Am Sonntag, den 06.06.2021, 12:14 +0200 schrieb Maxime Devos:
> Leo Prikler schreef op zo 06-06-2021 om 09:39 [+0200]:
> > I think we might want to export a utility procedure
> >     (patch-shebangs files inputs)
> 
> This procedure already exists, but is undocumented and
> unexported. It is in (guix build gnu-build-system).
> 
> (define* (patch-shebangs #:key inputs outputs (patch-shebangs? #t)
>                          #:allow-other-keys)
>   [...])
This patch-shebangs doesn't take a list of files as arguments, it
instead just iterates over all the files.  I don't think this is
particularly helpful in a cross-compiling context, where you want some
files (most notably scripts, that don't get wrapped) to have its
shebang drawn from inputs rather than native-inputs.

You could use the patch-shebang primitive from (guix build utils), but
you'd have to wrap it in something like

(let ((path (search-path-as-list '("bin" "sbin") (map cdr INPUTS))))
  (for-each (cute patch-shebang <> path) (find-files ...)))

where INPUTS are inputs or native-inputs, whichever you want.  Not very
friendly imo.
> > so that files used during build (e.g. configure, Makefile, etc.)
> > can do
> > (patch-shebangs build-stuff native-inputs) and the rest implicitly
> > gets
> > (patch-shebangs files inputs) during the patch-shebangs
> > phase.  WDYT?
> 
> Looking at %standard-phases, we have
> 
> (define %standard-phases
>   ;; Standard build phases, as a list of symbol/procedure pairs.
>   [...]
>     (phases [...] unpack bootstrap patch-usr-bin-file
>             patch-source-shebangs configure patch-generated-file-
> shebangs
>             build check install
>             patch-shebangs [...]))
> 
> Here, patch-source-shebangs calls patch-shebang for every file
> in the source code. As #:path is not set, $PATH is used. Thus,
> when cross-compiling, native-inputs (+ some implicit inputs) is used,
> and when compiling natively, the union of native-inputs and inputs (+
> some implicit inputs) is used (*).
> 
> Thus, the files used during build (configure, Makefile, ...) already
> get a ‘good’ interpreter. (Unless I'm mistaken, I didn't test this.)
I agree, that's the status quo.  We want that to be retained, but we
also want our cross-compilation story to be better.  Hence imo allowing
to pass files, constructing path from inputs and exporting this
procedure might make sense.

> (*) I looked into separating 'native-inputs' and 'inputs' even when
> compiling
>     natively but it turned out to be more complicated than first
> expected.
Fair enough, nobody expects you to do everything at once.

Regards,
Leo