unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / Atom feed
* Re: avoid wrapper scripts when possible
@ 2021-09-07 18:52 Attila Lendvai
  2021-09-07 20:41 ` Maxime Devos
  0 siblings, 1 reply; 15+ messages in thread
From: Attila Lendvai @ 2021-09-07 18:52 UTC (permalink / raw)
  To: guix-devel

[-- Attachment #1: Type: text/plain, Size: 729 bytes --]

pardon me for reviving a years old discussion, but i'm facing the same problem once again.

i have updated trezor support, and i wanted to test generating a new gpg key:

$ trezor-gpg init "foobar <foo@bar.com>"

leading to: OSError: Cannot find '.trezor-gpg-gpg-agent' in $PATH

upstream relies on sys.argv[0] in a non-trivial way:

https://github.com/romanz/trezor-agent/blob/338a075ed500f210a707cb3adf90104602e35c48/libagent/gpg/__init__.py#L124

is there any chance to revive this effort?

original discussion: https://lists.gnu.org/archive/html/guix-devel/2017-11/msg00017.html

shall i open an issue? for this specific problem, or the wider situation discussed in the mailing list thread?

- attila
PGP: 5D5F 45C7 DFCD 0A39

[-- Attachment #2: Type: text/html, Size: 1349 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: avoid wrapper scripts when possible
  2021-09-07 18:52 avoid wrapper scripts when possible Attila Lendvai
@ 2021-09-07 20:41 ` Maxime Devos
  2021-09-08  9:20   ` Attila Lendvai
  0 siblings, 1 reply; 15+ messages in thread
From: Maxime Devos @ 2021-09-07 20:41 UTC (permalink / raw)
  To: Attila Lendvai, guix-devel

[-- Attachment #1: Type: text/plain, Size: 1293 bytes --]

Hi,

Attila Lendvai schreef op di 07-09-2021 om 18:52 [+0000]:
> pardon me for reviving a years old discussion, but i'm facing the same problem once again.
> 
> i have updated trezor support, and i wanted to test generating a new gpg key:
> 
> $ trezor-gpg init "foobar <foo@bar.com>"
> 
> leading to: OSError: Cannot find '.trezor-gpg-gpg-agent' in $PATH
> 
> upstream relies on sys.argv[0] in a non-trivial way:
> 
> https://github.com/romanz/trezor-agent/blob/338a075ed500f210a707cb3adf90104602e35c48/libagent/gpg/__init__.py#L124

You could patch trezor-agent with something like

  # Python syntax might be incorrect
  if sys.argv[0] == ".trezor-gpg'
    sys.argv[0] = "trezor-gpg"
  device_name = os.path.basename(sys.argv[0]).rsplit('-',1)[0]

Would that work?

> is there any chance to revive this effort?
> 
> original discussion: https://lists.gnu.org/archive/html/guix-devel/2017-11/msg00017.html
> 
> shall i open an issue? for this specific problem, or the wider situation discussed in the mailing list thread?

FWIW, there's 'wrap-program' and there is 'wrap-script'.
wrap-script is less likely to cause errors here (it doesn't rename the executable)
but less general.  Maybe wrap-script can be used instead of wrap-program?

Greetings,
Maime

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: avoid wrapper scripts when possible
  2021-09-07 20:41 ` Maxime Devos
@ 2021-09-08  9:20   ` Attila Lendvai
  2021-09-08 10:39     ` Maxime Devos
  0 siblings, 1 reply; 15+ messages in thread
From: Attila Lendvai @ 2021-09-08  9:20 UTC (permalink / raw)
  To: Maxime Devos; +Cc: guix-devel

thanks for the ideas Maxime!


> You could patch trezor-agent with something like
>
> Python syntax might be incorrect
> ================================
>
> if sys.argv[0] == ".trezor-gpg'
> sys.argv[0] = "trezor-gpg"
> device_name = os.path.basename(sys.argv[0]).rsplit('-',1)[0]
>
> Would that work?


most probably, but i thought first i'll pursue a more general fix for
this. i will fall back to just patching upstream if i fail.


> FWIW, there's 'wrap-program' and there is 'wrap-script'.
>
> wrap-script is less likely to cause errors here (it doesn't rename the executable)
> but less general. Maybe wrap-script can be used instead of wrap-program?


not sure how to do this. looking at python-build-system.scm it looks
like there is no way to tell it to use wrap-script, so i went ahead
and just changed the wrap-program call to wrap-script globally, and
gave it a try.

the first error i encountered was in the package called scons. it has
.bat files that wrap-script fails on. if i delete them in a snippet,
then scons' own build system errors out.

any hints on how to proceed from here?

was this what you meant at all?

would it be a worthwhile improvement to hack my way through this path
at all?

shall i try to make it configurable from the package's definition,
change the default, and force it back to wrap-program for packages
that fail with wrap-script?

- attila



^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: avoid wrapper scripts when possible
  2021-09-08  9:20   ` Attila Lendvai
@ 2021-09-08 10:39     ` Maxime Devos
  0 siblings, 0 replies; 15+ messages in thread
From: Maxime Devos @ 2021-09-08 10:39 UTC (permalink / raw)
  To: Attila Lendvai; +Cc: guix-devel

[-- Attachment #1: Type: text/plain, Size: 2841 bytes --]

Attila Lendvai schreef op wo 08-09-2021 om 09:20 [+0000]:
> thanks for the ideas Maxime!
> 
> 
> > You could patch trezor-agent with something like
> > 
> > Python syntax might be incorrect
> > ================================
> > 
> > if sys.argv[0] == ".trezor-gpg'
> > sys.argv[0] = "trezor-gpg"
> > device_name = os.path.basename(sys.argv[0]).rsplit('-',1)[0]
> > 
> > Would that work?
> 
> most probably, but i thought first i'll pursue a more general fix for
> this. i will fall back to just patching upstream if i fail.
> 
> 
> > FWIW, there's 'wrap-program' and there is 'wrap-script'.
> > 
> > wrap-script is less likely to cause errors here (it doesn't rename the executable)
> > but less general. Maybe wrap-script can be used instead of wrap-program?
> 
> not sure how to do this. looking at python-build-system.scm it looks
> like there is no way to tell it to use wrap-script, so i went ahead
> and just changed the wrap-program call to wrap-script globally, and
> gave it a try.

Are you referring to the lines

    (for-each (lambda (dir)
                (let ((files (list-of-files dir)))
                  (for-each (cut wrap-program <> var)
                            files)))
              bindirs)

in the procedure 'wrap' from (guix build python-build-system)?

> the first error i encountered was in the package called scons. it has
> .bat files that wrap-script fails on. if i delete them in a snippet,
> then scons' own build system errors out.

The .bat are only for Windows, so you could remove them after the 'install'
phase or before the 'wrap' phase.  They seem to be useless on GNU/{Linux,Hurd}.

> any hints on how to proceed from here?

Instead of modifying 'wrap', you could replace the 'wrap' phase with
something like

  `(#:phases
    (modify-phases %standard-phases
      (replace 'wrap
        (lambda _
          ... customised wrapping code ...))))

Also, replacing wrap-program with wrap-script globally won't work
because wrap-script is less general, it only spports python, R,
perl and bash.  So something like

  (define (wrap-script-or-program ...)
    (if it-uses-python-or-ruby-or...
        (wrap-script ...)
        (wrap-program ...)))

may be required.

> was this what you meant at all?
> 
> would it be a worthwhile improvement to hack my way through this path
> at all?

There were some changes on core-updates to reduce double-wrapping,
so possibly it's already solved.

> shall i try to make it configurable from the package's definition,
> change the default, and force it back to wrap-program for packages
> that fail with wrap-script?

Perhaps try something like 'wrap-script-or-program' first?  That should
reduce the number of package definitions that need to be adjusted.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: avoid wrapper scripts when possible
  2017-11-02 21:31 Ricardo Wurmus
  2017-11-03 18:54 ` Christopher Baines
  2017-11-04 10:12 ` Hartmut Goebel
@ 2017-11-06 15:53 ` Dave Love
  2 siblings, 0 replies; 15+ messages in thread
From: Dave Love @ 2017-11-06 15:53 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: guix-devel

Ricardo Wurmus <rekado@elephly.net> writes:

> For binaries (like emacs) we’d still create shell wrappers where needed,
> because it’s harder to do this natively.

You could link in a constructor function to replace a wrapper that just
sets the environment, couldn't you?  I've done that at times and it
allows you to run gdb directly, for instance.

How easy it is presumably depends on how the program is built,
e.g. autoconf LIBS.  If it's difficult to separate configure and build,
you can use a dummy linker script at configure time.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: avoid wrapper scripts when possible
  2017-11-05 11:00     ` Hartmut Goebel
@ 2017-11-05 17:06       ` Ludovic Courtès
  0 siblings, 0 replies; 15+ messages in thread
From: Ludovic Courtès @ 2017-11-05 17:06 UTC (permalink / raw)
  To: Hartmut Goebel; +Cc: guix-devel

Hi!

Hartmut Goebel <h.goebel@crazy-compilers.com> skribis:

> answering on a mail which Ricardo CCed to me and which did not make it
> to the list yet. Thus I full-quote.
>
> Am 04.11.2017 um 23:30 schrieb Ricardo Wurmus:
>> How about this:
>>
>> --8<---------------cut here---------------start------------->8---
>> #!/home/rekado/.guix-profile/bin/guile --no-auto-compile
>> #!# (let (($0 (car (command-line)))) (execl "/home/rekado/.guix-profile/bin/python3" $0 $0))
>> #!/home/rekado/.guix-profile/bin/python3
>> import sys; print("hello from python: "+sys.argv[0])
>> --8<---------------cut here---------------end--------------->8---
>>
>> The first two lines are Guile code, but they are also line comments in
>> shell, Perl, Python, Ruby, and R.  The Guile code in this example calls
>> the script again as a Python script.  Before doing that it can set
>> environment variables, like so:
>>
>> --8<---------------cut here---------------start------------->8---
>> #!/home/rekado/.guix-profile/bin/guile --no-auto-compile
>> #!#
>> #\- (setenv "PYTHONPATH" (string-append "/gnu/store/foo:/gnu/store/bar:" (getenv "PYTHONPATH")))
>> #\- (let (($0 (car (command-line)))) (execl "/home/rekado/.guix-profile/bin/python3" $0 $0))
>> #!/home/rekado/.guix-profile/bin/python3
>> import sys; print("hello from python: "+sys.argv[0])
>> print(sys.path)
>> --8<---------------cut here---------------end--------------->8---
>>
>> Did I overlook something?  Or could we use this hack to remove separate
>> wrapper scripts for Perl, Python, Ruby, and R?

Overall this looks good to me (with the adjustments Hartmut mentions).

Now, we should have a white list of languages for which this can be
done.  ‘wrap-program’ would grab the shebang and check if the
basename of the interpreter is in that white list.

Thoughts?

Ludo’.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: avoid wrapper scripts when possible
  2017-11-03 21:17   ` Ricardo Wurmus
       [not found]     ` <874lqbghlz.fsf@elephly.net>
@ 2017-11-05 11:10     ` Hartmut Goebel
  1 sibling, 0 replies; 15+ messages in thread
From: Hartmut Goebel @ 2017-11-05 11:10 UTC (permalink / raw)
  To: guix-devel

[-- Attachment #1: Type: text/plain, Size: 1183 bytes --]

Hi,

Ricado proposed a new and better solution already. Nevertheless I want
to comment on this proposal, sint it includes a pitfall and would not work:

Am 03.11.2017 um 22:17 schrieb Ricardo Wurmus:
> exec /home/rekado/.guix-profile/bin/python <(/run/current-system/profile/bin/tail -n +4 "$0")
[...]
> This seems to work just fine and off the top of my head I can’t think of
> a situation where this would fail — unless the wrapped script performed
> the same kind of trick of reading its contents through $0.  (How likely
> is that?)

Indeed for Python programs it is quite common to revert to the filename
using the variable "__file__". And if feeding in the script from stdin
would set __file__ to "<stdin>". Using __file__ is more often used in
package modules than in scripts (which could use $0 (sys.argv[0]).
Anyway this is a pitfall we should avoid.

Please not that Ricardo's new solution, based on guile and execl does
not suffer from this problem (I verified).

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |


[-- Attachment #2: Type: text/html, Size: 2029 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: avoid wrapper scripts when possible
       [not found]   ` <87tvy9ptge.fsf@elephly.net>
@ 2017-11-05 11:00     ` Hartmut Goebel
  2017-11-05 17:06       ` Ludovic Courtès
  0 siblings, 1 reply; 15+ messages in thread
From: Hartmut Goebel @ 2017-11-05 11:00 UTC (permalink / raw)
  To: guix-devel

Hi,

answering on a mail which Ricardo CCed to me and which did not make it
to the list yet. Thus I full-quote.

Am 04.11.2017 um 23:30 schrieb Ricardo Wurmus:
> How about this:
>
> --8<---------------cut here---------------start------------->8---
> #!/home/rekado/.guix-profile/bin/guile --no-auto-compile
> #!# (let (($0 (car (command-line)))) (execl "/home/rekado/.guix-profile/bin/python3" $0 $0))
> #!/home/rekado/.guix-profile/bin/python3
> import sys; print("hello from python: "+sys.argv[0])
> --8<---------------cut here---------------end--------------->8---
>
> The first two lines are Guile code, but they are also line comments in
> shell, Perl, Python, Ruby, and R.  The Guile code in this example calls
> the script again as a Python script.  Before doing that it can set
> environment variables, like so:
>
> --8<---------------cut here---------------start------------->8---
> #!/home/rekado/.guix-profile/bin/guile --no-auto-compile
> #!#
> #\- (setenv "PYTHONPATH" (string-append "/gnu/store/foo:/gnu/store/bar:" (getenv "PYTHONPATH")))
> #\- (let (($0 (car (command-line)))) (execl "/home/rekado/.guix-profile/bin/python3" $0 $0))
> #!/home/rekado/.guix-profile/bin/python3
> import sys; print("hello from python: "+sys.argv[0])
> print(sys.path)
> --8<---------------cut here---------------end--------------->8---
>
> Did I overlook something?  Or could we use this hack to remove separate
> wrapper scripts for Perl, Python, Ruby, and R?


I testes this and (beside some necessary adjustments) this works. Indeed
this a very elegant hack :-). My respect! Compared to my idea this is
more elegant since it does not require a separate file. 

Some fine-tuning needs to be done, though:

1) The setenv-line should only append the existing env-var, if this was
set. (At least in Python, an empty PYTHONPATH part is interpreted as
"current directory", thus adding an empty part would change the desired
behavior.)

2) Arguments need to be passed to the script, too.

Since I was eager to learn how this can be done, here is my draft fixing
these two issues:

--8<---------------cut here---------------start------------->8---

#!/bin/guile --no-auto-compile
#!#; Guix wrapping header
#\- (define (append-env name path) (let ((var (getenv name))) (setenv
name (if var (string-append path ":" var) path))))
#\- (append-env "PYTHONPATH" "/gnu/store/foo:/gnu/store/bar")
#\- (apply execl "/bin/python3" (car (command-line)) (command-line))
#!/home/rekado/.guix-profile/bin/python3

print("This is the real program")
import sys, os
print("sys.path[:4]", sys.path[:4])
print("argumentes:", sys.argv)
print("ppid", os.getppid())

--8<---------------cut here---------------end--------------->8---

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: avoid wrapper scripts when possible
  2017-11-03 18:54 ` Christopher Baines
  2017-11-03 19:22   ` Jan Nieuwenhuizen
  2017-11-03 21:17   ` Ricardo Wurmus
@ 2017-11-04 21:42   ` Eric Bavier
  2 siblings, 0 replies; 15+ messages in thread
From: Eric Bavier @ 2017-11-04 21:42 UTC (permalink / raw)
  To: Christopher Baines; +Cc: guix-devel

[-- Attachment #1: Type: text/plain, Size: 1698 bytes --]

On Fri, 3 Nov 2017 18:54:16 +0000
Christopher Baines <mail@cbaines.net> wrote:

> However, I think that the file wrapping approach has advantages for
> visibility. Maybe it could be tweaked to keep ensure the wrapper script
> has the same name as the script its wrapping, e.g. when wrapping foo,
> replace foo with a bash script, and move the real script
> to .wrapped-bin/foo.

We already try to do just that, by using the -a flag when exec'ing the
real program from the wrapper.  But...

> the environment activation feature of our “conda” package currently
> fails.  This is because the “deactivate” shell script is wrapped in
> another shell script.  This leads to the actual shell script to be
> called “.deactivate-real”.  The script compares the value of “$0” with
> the expected name “deactivate”.  This fails so conda misbehaves.

... it doesn't behave correctly when the "real" program is
shebang-interpreted.  It works fine for compiled programs.  

E.g. our "emacs-25.3" is a wrapper that sets XDG and GTK env vars, but
top/htop/ps show the proper "emacs-25.3".  OTOH we have the
"deactivate" example from Ricardo, which is a shell script, and e.g.
youtube-dl, whose --help output includes "Usage: .youtube-dl-real ...".

So the interpreters are not passing on the intended zeroth argument.

IMHO it'd be nice to find some way to fix that.  A while ago I had
looked into patching Perl libraries/programs.  But like Hartmut
mentioned for python, I noticed after some research that patching perl
paths would be nontrivial, and would require a perl parser to determine
where and how to patch the library paths correctly.

2c,
`~Eric

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: avoid wrapper scripts when possible
  2017-11-02 21:31 Ricardo Wurmus
  2017-11-03 18:54 ` Christopher Baines
@ 2017-11-04 10:12 ` Hartmut Goebel
       [not found]   ` <87tvy9ptge.fsf@elephly.net>
  2017-11-06 15:53 ` Dave Love
  2 siblings, 1 reply; 15+ messages in thread
From: Hartmut Goebel @ 2017-11-04 10:12 UTC (permalink / raw)
  To: guix-devel

Hallo Ricardo,

Am 02.11.2017 um 22:31 schrieb Ricardo Wurmus:
> This made me wonder if we could avoid shell wrappers in more cases.

This is an interesting challenge :-)

> If
> the target script is a Python script we could set the environment
> variables right after the Python shebang with Python code.  If the
> target script is a shell script we can set environment variables right
> there after the shebang.

Basically thsi would be a good solution IMO. But I'm afraid it might be
complicated to insert the code "right after the […] shebang". For
example Python has "doc-strings" [1], this is: if the first statement of
the script is a string (expression statement), this string is put into
the variable __doc__. So for at least for Python we might need a
language-aware scanner to insert the code at the correct position. We
might need similar tor other languages.

But taking your challenge :-) I had another idea which should work for
all script languages:

The real script's she-bang delegates to the wrapper (instead of
/bin/sh), which execs "interpreter $0". Here is a proof on concept:

....8<------- real.py
#!/tmp/wrapper.sh
print("This is real")
import sys, os
print("sys.path[:2]", sys.path[:2])
print("argumentes:", sys.argv)
print("ppid", os.getppid())
....8<-------

....8<-------  wrapper.sh
#!/bin/sh
echo "This is the wrapper"
echo "arguments in wrapper:" "$@"
echo "ppid:" $PPID
export PYTHONPATH=/foo/bar
exec python3 /tmp/real.py "${@:2}" # <- replace first argument by full path
....8<-------

I have to admit that this still adds another script and is less elegant,
but should be easier to implement.

[]1] https://www.python.org/dev/peps/pep-0257/#what-is-a-docstring

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: avoid wrapper scripts when possible
       [not found]     ` <874lqbghlz.fsf@elephly.net>
@ 2017-11-03 22:41       ` Ricardo Wurmus
  0 siblings, 0 replies; 15+ messages in thread
From: Ricardo Wurmus @ 2017-11-03 22:41 UTC (permalink / raw)
  To: Christopher Baines; +Cc: guix-devel


Ricardo Wurmus <rekado@elephly.net> writes:

> Ricardo Wurmus <rekado@elephly.net> writes:
>
>> I wonder if there’s a way to have a script header that starts out as a
>> shell script and then calls itself again but skipping over the header,
>> thus calling the original script.
>>
>> Take this Python script as an example:
>>
>> --8<---------------cut here---------------start------------->8---
>> #!/home/rekado/.guix-profile/bin/python
>> print("hello from python")
>> --8<---------------cut here---------------end--------------->8---
>>
>> We then prepend this header:
>>
>> --8<---------------cut here---------------start------------->8---
>> #!/run/current-system/profile/bin/bash
>> echo "hello from bash"
>> exec /home/rekado/.guix-profile/bin/python <(/run/current-system/profile/bin/tail -n +4 "$0")
>> --8<---------------cut here---------------end--------------->8---
>>
>> When we execute the resulting thing it will be a shell script, and do
>> shell things printing “hello from bash”.  Then it scrubs itself off the
>> script and removes the original script’s Python shebang using tail (with
>> “-n +4” where 4 is the number of lines of the shell script header + 1),
>> and replaces itself with Python calling the original script.
>>
>> This seems to work just fine […]
>
> Erm, it does not.  “$0” becomes a file descriptor like “/dev/fd/63”.
> “exec -a $0” also doesn’t help here.

How about this:

--8<---------------cut here---------------start------------->8---
#!/home/rekado/.guix-profile/bin/guile --no-auto-compile
#!# (let (($0 (car (command-line)))) (execl "/home/rekado/.guix-profile/bin/python3" $0 $0))
#!/home/rekado/.guix-profile/bin/python3
import sys; print("hello from python: "+sys.argv[0])
--8<---------------cut here---------------end--------------->8---

The first two lines are Guile code, but they are also line comments in
shell, Perl, Python, Ruby, and R.  The Guile code in this example calls
the script again as a Python script.  Before doing that it can set
environment variables, like so:

--8<---------------cut here---------------start------------->8---
#!/home/rekado/.guix-profile/bin/guile --no-auto-compile
#!#
#\- (setenv "PYTHONPATH" (string-append "/gnu/store/foo:/gnu/store/bar:" (getenv "PYTHONPATH")))
#\- (let (($0 (car (command-line)))) (execl "/home/rekado/.guix-profile/bin/python3" $0 $0))
#!/home/rekado/.guix-profile/bin/python3
import sys; print("hello from python: "+sys.argv[0])
print(sys.path)
--8<---------------cut here---------------end--------------->8---

Did I overlook something?  Or could we use this hack to remove separate
wrapper scripts for Perl, Python, Ruby, and R?

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: avoid wrapper scripts when possible
  2017-11-03 18:54 ` Christopher Baines
  2017-11-03 19:22   ` Jan Nieuwenhuizen
@ 2017-11-03 21:17   ` Ricardo Wurmus
       [not found]     ` <874lqbghlz.fsf@elephly.net>
  2017-11-05 11:10     ` Hartmut Goebel
  2017-11-04 21:42   ` Eric Bavier
  2 siblings, 2 replies; 15+ messages in thread
From: Ricardo Wurmus @ 2017-11-03 21:17 UTC (permalink / raw)
  To: Christopher Baines; +Cc: guix-devel


Christopher Baines <mail@cbaines.net> writes:

> However, I think that the file wrapping approach has advantages for
> visibility. Maybe it could be tweaked to keep ensure the wrapper script
> has the same name as the script its wrapping, e.g. when wrapping foo,
> replace foo with a bash script, and move the real script
> to .wrapped-bin/foo.

I agree that having a separate wrapper is certainly nicer than patching
upstream scripts automatically.  The idea of moving original scripts out
of the way sounds intriguing, but I fear that we would have a similar
set of problems.

In the case of conda’s “activate” script there would be an obvious
problem: the script tries to determine the directory that contains it.
If that were a different directory than the one the developers intended,
it would fail in unexpected ways.

I wonder if there’s a way to have a script header that starts out as a
shell script and then calls itself again but skipping over the header,
thus calling the original script.

Take this Python script as an example:

--8<---------------cut here---------------start------------->8---
#!/home/rekado/.guix-profile/bin/python
print("hello from python")
--8<---------------cut here---------------end--------------->8---

We then prepend this header:

--8<---------------cut here---------------start------------->8---
#!/run/current-system/profile/bin/bash
echo "hello from bash"
exec /home/rekado/.guix-profile/bin/python <(/run/current-system/profile/bin/tail -n +4 "$0")
--8<---------------cut here---------------end--------------->8---

When we execute the resulting thing it will be a shell script, and do
shell things printing “hello from bash”.  Then it scrubs itself off the
script and removes the original script’s Python shebang using tail (with
“-n +4” where 4 is the number of lines of the shell script header + 1),
and replaces itself with Python calling the original script.

This seems to work just fine and off the top of my head I can’t think of
a situation where this would fail — unless the wrapped script performed
the same kind of trick of reading its contents through $0.  (How likely
is that?)

What do you think?  Is this too weird to consider?  Weirder than
renaming binaries and replacing them with scripts?

One side effect would be that “file” would be rather confused about
these scripts, but maybe that’s acceptable.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: avoid wrapper scripts when possible
  2017-11-03 18:54 ` Christopher Baines
@ 2017-11-03 19:22   ` Jan Nieuwenhuizen
  2017-11-03 21:17   ` Ricardo Wurmus
  2017-11-04 21:42   ` Eric Bavier
  2 siblings, 0 replies; 15+ messages in thread
From: Jan Nieuwenhuizen @ 2017-11-03 19:22 UTC (permalink / raw)
  To: Christopher Baines; +Cc: guix-devel

Christopher Baines writes:

> However, I think that the file wrapping approach has advantages for
> visibility. Maybe it could be tweaked to keep ensure the wrapper script
> has the same name as the script its wrapping, e.g. when wrapping foo,
> replace foo with a bash script, and move the real script
> to .wrapped-bin/foo.

To probably state the obvious, yes it would be great if top/ps would
show `foo' instead of .foo-x.y-real; also great for `killall'.

Greetings,
janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: avoid wrapper scripts when possible
  2017-11-02 21:31 Ricardo Wurmus
@ 2017-11-03 18:54 ` Christopher Baines
  2017-11-03 19:22   ` Jan Nieuwenhuizen
                     ` (2 more replies)
  2017-11-04 10:12 ` Hartmut Goebel
  2017-11-06 15:53 ` Dave Love
  2 siblings, 3 replies; 15+ messages in thread
From: Christopher Baines @ 2017-11-03 18:54 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: guix-devel

[-- Attachment #1: Type: text/plain, Size: 2689 bytes --]

On Thu, 02 Nov 2017 22:31:16 +0100
Ricardo Wurmus <rekado@elephly.net> wrote:

> Hi Guix,
> 
> the environment activation feature of our “conda” package currently
> fails.  This is because the “deactivate” shell script is wrapped in
> another shell script.  This leads to the actual shell script to be
> called “.deactivate-real”.  The script compares the value of “$0” with
> the expected name “deactivate”.  This fails so conda misbehaves.
> 
> Since “deactivate” is really just a shell script I think we could avoid
> the renaming and the external wrapper by setting the environment
> variables in the “deactivate” script itself.
> 
> This made me wonder if we could avoid shell wrappers in more cases.  If
> the target script is a Python script we could set the environment
> variables right after the Python shebang with Python code.  If the
> target script is a shell script we can set environment variables right
> there after the shebang.
> 
> For binaries (like emacs) we’d still create shell wrappers where needed,
> because it’s harder to do this natively.
> 
> What do you think?

I recently tried to implement automatic wrapping for the ruby build
system [1]. I ended up implementing a ruby specific wrapping function,
that generated a ruby wrapper, rather than a bash wrapper.

I first tried using a bash wrapper, but it broke using the -S flag
with ruby to execute the script, as when -S is passed to ruby, it
expects the script on the PATH to use ruby in the shebang, and not bash.

1: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=28773

After I wrote this, I also stumbled upon #27003: Generalized wrap phase
for perl, python. I haven't read through the thread enough to work out
what the state of it is.

2: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=27003

I've also found some other bugs relating to wrapping, [3] is about the
issue you mention, and [4] relates to the native inputs problem, that I
came across when I was working on the ruby-build-system wrapping. 

3: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26752
4: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25235

I haven't read through all of these in much detail though.

Back to your specific question about avoiding shell wrappers. For ruby,
I found this necessary to avoid breaking one way of invoking the
scripts.

However, I think that the file wrapping approach has advantages for
visibility. Maybe it could be tweaked to keep ensure the wrapper script
has the same name as the script its wrapping, e.g. when wrapping foo,
replace foo with a bash script, and move the real script
to .wrapped-bin/foo.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* avoid wrapper scripts when possible
@ 2017-11-02 21:31 Ricardo Wurmus
  2017-11-03 18:54 ` Christopher Baines
                   ` (2 more replies)
  0 siblings, 3 replies; 15+ messages in thread
From: Ricardo Wurmus @ 2017-11-02 21:31 UTC (permalink / raw)
  To: guix-devel

Hi Guix,

the environment activation feature of our “conda” package currently
fails.  This is because the “deactivate” shell script is wrapped in
another shell script.  This leads to the actual shell script to be
called “.deactivate-real”.  The script compares the value of “$0” with
the expected name “deactivate”.  This fails so conda misbehaves.

Since “deactivate” is really just a shell script I think we could avoid
the renaming and the external wrapper by setting the environment
variables in the “deactivate” script itself.

This made me wonder if we could avoid shell wrappers in more cases.  If
the target script is a Python script we could set the environment
variables right after the Python shebang with Python code.  If the
target script is a shell script we can set environment variables right
there after the shebang.

For binaries (like emacs) we’d still create shell wrappers where needed,
because it’s harder to do this natively.

What do you think?

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2021-09-08 10:49 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-07 18:52 avoid wrapper scripts when possible Attila Lendvai
2021-09-07 20:41 ` Maxime Devos
2021-09-08  9:20   ` Attila Lendvai
2021-09-08 10:39     ` Maxime Devos
  -- strict thread matches above, loose matches on Subject: below --
2017-11-02 21:31 Ricardo Wurmus
2017-11-03 18:54 ` Christopher Baines
2017-11-03 19:22   ` Jan Nieuwenhuizen
2017-11-03 21:17   ` Ricardo Wurmus
     [not found]     ` <874lqbghlz.fsf@elephly.net>
2017-11-03 22:41       ` Ricardo Wurmus
2017-11-05 11:10     ` Hartmut Goebel
2017-11-04 21:42   ` Eric Bavier
2017-11-04 10:12 ` Hartmut Goebel
     [not found]   ` <87tvy9ptge.fsf@elephly.net>
2017-11-05 11:00     ` Hartmut Goebel
2017-11-05 17:06       ` Ludovic Courtès
2017-11-06 15:53 ` Dave Love

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).