Alternatives to --emulate-fhs on foreign distros

Alternatives to --emulate-fhs on foreign distros

[Kyle Andrews]

Dear Guix,

The (many) shell scripts in my software depend on bashisms. In the code right now I typically use /usr/bin/env bash since I had read that was the most portable way of referencing a shell. However, I really don't think I have a good handle on what my choices are. Maybe the tedious solution of just typing `bash script.sh' in all of the system calls referencing this scripts might be the right approach in the long run?

Should I be aiming to automatically patch my source code to point to a specific bash? Why doesn't this happen automatically? Is there a way to upgrade from /bin/sh to bash within a shell script? I know I can use --emulate-fhs to ensure that /bin/bash and /usr/bin/env bash are available, but that seems to preclude including the software directly in Guix without a container.

I'm curious what other options might be. The main bashism I use is default environment variables ${FOO:-bar}. On a native Guix system I could just add /usr/bin/env back in. However, that doesn't seem to be possible with only Guix shell. Maybe I am wrong here?

Thanks,
Kyle

Re: Alternatives to --emulate-fhs on foreign distros

[Andreas Enge]

Hello,

Am Wed, Apr 19, 2023 at 06:18:33AM +0000 schrieb Kyle Andrews:
> I'm curious what other options might be. The main bashism I use is default environment variables ${FOO:-bar}. On a native Guix system I could just add /usr/bin/env back in. However, that doesn't seem to be possible with only Guix shell. Maybe I am wrong here?

as an exception to the rule, a Guix system contains /bin/sh, which on my
machine looks like this:

$ ll /bin
lrwxrwxrwx 1 root root  2  5. Nov 2019  bash -> sh
lrwxrwxrwx 1 root root 61 17. Apr 09:51 sh -> /gnu/store/d99ykvj3axzzidygsmdmzxah4lvxd6hw-bash-5.1.8/bin/sh

$ ll /gnu/store/d99ykvj3axzzidygsmdmzxah4lvxd6hw-bash-5.1.8/bin
-r-xr-xr-x 2 root root 951344  1. Jan 1970  bash
-r-xr-xr-x 1 root root   6791  1. Jan 1970  bashbug
lrwxrwxrwx 1 root root      4  1. Jan 1970  sh -> bash

So everything is bash in the end, but I do not know whether bash behaves
differently if it is called under the name "sh". You could give it a try.

Andreas

Re: Alternatives to --emulate-fhs on foreign distros

[Martin Castillo]

Hi,

> 
> So everything is bash in the end, but I do not know whether bash behaves
> differently if it is called under the name "sh". You could give it a try.
> 

Yes. Search for \bsh\b in the manpage. It includes things like

If  bash is invoked with the name sh, it tries to mimic the startup be‐
havior of historical versions of sh as closely as possible, while  con‐
forming  to the POSIX standard as well.

Martin

Re: Alternatives to --emulate-fhs on foreign distros

[(]

Kyle Andrews <kyle@posteo.net> writes:
> The (many) shell scripts in my software depend on bashisms. In the code right
> now I typically use /usr/bin/env bash since I had read that was the most
> portable way of referencing a shell. However, I really don't think I have a good
> handle on what my choices are. Maybe the tedious solution of just typing `bash
> script.sh' in all of the system calls referencing this scripts might be the
> right approach in the long run?

``/usr/bin/env bash'' will automatically be substituted for the path to
an available ``/bin/bash'' in the store.  (If that's what you're talking
about here; not sure what the context is exactly.)

Re: Alternatives to --emulate-fhs on foreign distros

[(]

"(" <paren@disroot.org> writes:
> ``/usr/bin/env bash'' will automatically be substituted for the path to
> an available ``/bin/bash'' in the store.  (If that's what you're talking
> about here; not sure what the context is exactly.)

...never mind, i thought you were talking about software that you want
to package.

Re: Alternatives to --emulate-fhs on foreign distros

[Kyle]

The context is that my R package calls a bunch of shell scripts stored inside it. I am getting errors about /usr/bin/env not being available (e.g. bad interpreter: no such file or directory), so was thinking that their shebangs were not getting patched. It's not clear to me how thorough Guix is being at finding these. 

On April 19, 2023 11:10:44 AM EDT, "(" <paren@disroot.org> wrote:
>
>Kyle Andrews <kyle@posteo.net> writes:
>> The (many) shell scripts in my software depend on bashisms. In the code right
>> now I typically use /usr/bin/env bash since I had read that was the most
>> portable way of referencing a shell. However, I really don't think I have a good
>> handle on what my choices are. Maybe the tedious solution of just typing `bash
>> script.sh' in all of the system calls referencing this scripts might be the
>> right approach in the long run?
>
>``/usr/bin/env bash'' will automatically be substituted for the path to
>an available ``/bin/bash'' in the store.  (If that's what you're talking
>about here; not sure what the context is exactly.)

Re: Alternatives to --emulate-fhs on foreign distros

[(]

Kyle <kyle@posteo.net> writes:
> The context is that my R package calls a bunch of shell scripts stored inside it. I am getting errors about
> /usr/bin/env not being available (e.g. bad interpreter: no such file or directory), so was thinking that their
> shebangs were not getting patched. It's not clear to me how thorough Guix is being at finding these. 

Hmm.  Can you show me an example of one of these scripts?  (The
R-BUILD-SYSTEM *does* seem to include 'PATCH-SHEBANGS...)

Re: Alternatives to --emulate-fhs on foreign distros

[Kyle Andrews]

"(" <paren@disroot.org> writes:

> Kyle <kyle@posteo.net> writes:
>> The context is that my R package calls a bunch of shell scripts stored inside it. I am getting errors about
>> /usr/bin/env not being available (e.g. bad interpreter: no such file or directory), so was thinking that their
>> shebangs were not getting patched. It's not clear to me how thorough Guix is being at finding these. 
>
> Hmm.  Can you show me an example of one of these scripts?  (The
> R-BUILD-SYSTEM *does* seem to include 'PATCH-SHEBANGS...)

You are right about this. I created a minimal reproducible R package with a basic shell script included and the shebangs were patched. I'm still trying to figure out the actual cause of my shell errors.

Re: Alternatives to --emulate-fhs on foreign distros

[(]

Kyle Andrews <kyle@posteo.net> writes:
> You are right about this. I created a minimal reproducible R package with a basic shell script included and the shebangs were patched. I'm still trying to figure out the actual cause of my shell errors.

Perhaps try using ``shellcheck''.

Re: Alternatives to --emulate-fhs on foreign distros

[Kyle]

Its kind of hard to test. The easiest I have managed has been to see that I don't get the error with --emulate-fhs in my call, and the script that fails starts with:

#!/usr/bin/env txr

To simplify things, I made a self contained txr script and wrapped that up in my test R package. Sure enough, after building the package I checked its store directory. Guix did not patch the path to the path to the env utility in that case.

Re: Alternatives to --emulate-fhs on foreign distros

[Kyle]

Note that the easiest standalone TXR script I could think of is:

```
#!/usr/bin/env txr
#(output)
Hi
@(end)
```

I am betting you probably could include that in a package to reproduce the issue.

On April 21, 2023 2:35:30 PM EDT, Kyle <kyle@posteo.net> wrote:
>Its kind of hard to test. The easiest I have managed has been to see that I don't get the error with --emulate-fhs in my call, and the script that fails starts with:
>
>#!/usr/bin/env txr
>
>To simplify things, I made a self contained txr script and wrapped that up in my test R package. Sure enough, after building the package I checked its store directory. Guix did not patch the path to the path to the env utility in that case.
>

Re: Alternatives to --emulate-fhs on foreign distros

[Philip McGrath]

[-- Attachment #1: Type: text/plain, Size: 915 bytes --]

On Friday, April 21, 2023 2:35:30 PM EDT Kyle wrote:
> Its kind of hard to test. The easiest I have managed has been to see that I
> don't get the error with --emulate-fhs in my call, and the script that
> fails starts with:
> 
> #!/usr/bin/env txr
> 
> To simplify things, I made a self contained txr script and wrapped that up
> in my test R package. Sure enough, after building the package I checked its
> store directory. Guix did not patch the path to the path to the env utility
> in that case.

Did you include txr as an input to your package? The 'patch-shebangs doesn't 
patch in an absolute path to an env utility: it rewrites shebangs to use the 
absolute path to the actual interpreter, in this case txr, and it only does so 
if the interpreter is actually present at build time. (A package source might 
contain scripts that aren't actually used, and they shouldn't create artificial 
dependencies.)



[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Alternatives to --emulate-fhs on foreign distros

[Maxim Cournoyer]

Hi Kyle,

Kyle Andrews <kyle@posteo.net> writes:

> Dear Guix,
>
> The (many) shell scripts in my software depend on bashisms. In the
> code right now I typically use /usr/bin/env bash since I had read that
> was the most portable way of referencing a shell. However, I really
> don't think I have a good handle on what my choices are. Maybe the
> tedious solution of just typing `bash script.sh' in all of the system
> calls referencing this scripts might be the right approach in the long
> run?

Use /usr/bin/env.  It's available on all mainstream distributions, and
it's also available on Guix System, so it should work everywhere.

-- 
Thanks,
Maxim

Re: Alternatives to --emulate-fhs on foreign distros

[Kyle]

Dear Maxim, 

Unfortunately, I don't see /usr/bin/env when I look inside the file system resulting from:

guix shell coreutils --container

I don't understand how to apply "system" configuration to a guix shell container other than crudely through --emulate-fhs. Since noone has needed it so far, maybe that's not that helpful if /bin/bash was made available?


On April 20, 2023 8:56:47 AM EDT, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
>Hi Kyle,
>
>Kyle Andrews <kyle@posteo.net> writes:
>
>> Dear Guix,
>>
>> The (many) shell scripts in my software depend on bashisms. In the
>> code right now I typically use /usr/bin/env bash since I had read that
>> was the most portable way of referencing a shell. However, I really
>> don't think I have a good handle on what my choices are. Maybe the
>> tedious solution of just typing `bash script.sh' in all of the system
>> calls referencing this scripts might be the right approach in the long
>> run?
>
>Use /usr/bin/env.  It's available on all mainstream distributions, and
>it's also available on Guix System, so it should work everywhere.
>
>-- 
>Thanks,
>Maxim

Re: Alternatives to --emulate-fhs on foreign distros

[Maxim Cournoyer]

Hi Kyle,

Kyle <kyle@posteo.net> writes:

> Dear Maxim, 
>
> Unfortunately, I don't see /usr/bin/env when I look inside the file system resulting from:
>
> guix shell coreutils --container
>

For this use case, I use the --symlink option, like so:
'--symlink=/usr/bin/env=bin/env'.

-- 
Thanks,
Maxim

Re: Alternatives to --emulate-fhs on foreign distros

[Kyle]

Thanks so much! That is perfect.

On April 22, 2023 4:36:08 PM EDT, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
>Hi Kyle,
>
>Kyle <kyle@posteo.net> writes:
>
>> Dear Maxim, 
>>
>> Unfortunately, I don't see /usr/bin/env when I look inside the file system resulting from:
>>
>> guix shell coreutils --container
>>
>
>For this use case, I use the --symlink option, like so:
>'--symlink=/usr/bin/env=bin/env'.
>
>-- 
>Thanks,
>Maxim

Re: Alternatives to --emulate-fhs on foreign distros

[wolf]

[-- Attachment #1: Type: text/plain, Size: 505 bytes --]

Hey :)

While slightly of topic, I have to wonder about:

On 2023-04-19 06:18:33 +0000, Kyle Andrews wrote:
> The main bashism I use is default environment variables ${FOO:-bar}.

I'm pretty sure that is just regular POSIX compatible syntax no? At least if I
read [0] correctly.

0: https://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_06_02

W.

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]