bug#25258: Bashisms when building 0.11.0

bug#25258: Bashisms when building 0.11.0

[pelzflorian (Florian Pelz)]

[-- Attachment #1: Type: text/plain, Size: 884 bytes --]

Hello,

Guix' testsuite uses the non-POSIX bashism `type -P guile` in
tests/guix-packages.sh which causes a test failure when using
Dash to provide /bin/sh instead of Bash. Additionally gnu/local.mk uses
brace expansion which is not part of POSIX and does not work on Dash.

When building Guix on Parabola GNU/Linux-libre modified to use Dash as
/bin/sh,[1] this causes the Guix build to fail. For building, the guix
0.11.0 package from AUR[2] was used. The attached “patch” allows for a
successful build but is not what we want.

Guix should probably not use bashisms in its Makefile includes and
either not use bashisms in its test suite or make sure the test suite is
executed explicitly with Bash and not /bin/sh.

Regards,
Florian Pelz

[1] https://wiki.archlinux.org/index.php/Dash#Use_DASH_as_.2Fbin.2Fsh
[2] https://aur.archlinux.org/packages/guix/


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: guix-remove-bashisms.patch --]
[-- Type: text/x-patch; name="guix-remove-bashisms.patch", Size: 1140 bytes --]

diff -aur guix-0.11.0.pristine/gnu/local.mk guix-0.11.0.new/gnu/local.mk
--- guix-0.11.0.pristine/gnu/local.mk	2016-12-23 16:24:51.501094452 +0100
+++ guix-0.11.0.new/gnu/local.mk	2016-12-23 17:26:55.651112436 +0100
@@ -903,7 +903,10 @@
 # Those files must remain executable, so they remain executable once
 # imported into the store.
 set-bootstrap-executable-permissions:
-	chmod +x $(DESTDIR)$(bootstrapdir)/*/{bash,mkdir,tar,xz}
+	chmod +x $(DESTDIR)$(bootstrapdir)/*/bash
+	chmod +x $(DESTDIR)$(bootstrapdir)/*/mkdir
+	chmod +x $(DESTDIR)$(bootstrapdir)/*/tar
+	chmod +x $(DESTDIR)$(bootstrapdir)/*/xz
 
 DISTCLEANFILES =				\
   $(nodist_bootstrap_x86_64_linux_DATA)		\
diff -aur guix-0.11.0.pristine/Makefile.am guix-0.11.0.new/Makefile.am
--- guix-0.11.0.pristine/Makefile.am	2016-12-23 16:24:49.007702057 +0100
+++ guix-0.11.0.new/Makefile.am	2016-12-23 16:25:39.372217348 +0100
@@ -275,7 +275,6 @@
   tests/guix-download.sh			\
   tests/guix-gc.sh				\
   tests/guix-hash.sh				\
-  tests/guix-package.sh				\
   tests/guix-package-net.sh			\
   tests/guix-system.sh				\
   tests/guix-archive.sh				\

bug#25258: Bashisms when building 0.11.0

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 917 bytes --]

On Sat, Dec 24, 2016 at 08:26:27AM +0100, pelzflorian (Florian Pelz) wrote:
> Guix' testsuite uses the non-POSIX bashism `type -P guile` in
> tests/guix-packages.sh which causes a test failure when using
> Dash to provide /bin/sh instead of Bash. Additionally gnu/local.mk uses
> brace expansion which is not part of POSIX and does not work on Dash.
> 
> When building Guix on Parabola GNU/Linux-libre modified to use Dash as
> /bin/sh,[1] this causes the Guix build to fail. For building, the guix
> 0.11.0 package from AUR[2] was used. The attached “patch” allows for a
> successful build but is not what we want.
> 
> Guix should probably not use bashisms in its Makefile includes and
> either not use bashisms in its test suite or make sure the test suite is
> executed explicitly with Bash and not /bin/sh.

About two years have passed on this bug report without comment. Is it a
"wontfix"?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

bug#25258: Bashisms when building 0.11.0

[Leo Famulari]

On Tue, Feb 12, 2019 at 08:46:30PM -0500, Leo Famulari wrote:
> About two years have passed on this bug report without comment. Is it a
> "wontfix"?

Based on the lack of interest I've closed this bug.

bug#25258: Bashisms in make rules

[elaexuotee--- via Bug reports for GNU Guix]

[-- Attachment #1.1: Type: text/plain, Size: 3242 bytes --]

Resurrecting this, since it cause me significant grief, and is a simple
workaround which I believe is worth putting in the documentation. First, here
is a short summary of the problem:

Currently, some of our make rules contain bashisms. As of commit 408ae72c, this
ends up generating broken texinfo translations under doc/*.texi, eventually
failing with "@ref reference to nonexistent node..." errors.

From a pristine repository (e.g. after running `git clean -xfd'), running make
shows a bunch of errors from the broken rules, but since they happen to be
multi-line scripts, the errors fail to propogate up to make. Thus make leaves
around the broken texinfo files. The real kicker is that subsequent runs pick
up the broken files and make fails just as above, but without encountering or
showing the errors from the broken rules. This makes tracking down the error
significantly more painful and non-obvious.

Anyway, the source of the problem is a combination of our use of bashisms and
autoconf trying really hard to use the least-common-denominator shell.
Autoconf's configure script sets make's SHELL by first looking for /bin/sh and
only finding sh on PATH as a last resort. Thus, even when running under `guix
environment --pure guix' make will run with /bin/sh instead of
$GUIX_ENVIRONMENT/bin/sh.

On a Guix System this ends up running make with the sh of the system profile
which, at the moment, resolves to bash. However, on a foreign distribution,
/bin/sh often resolves to a non-bash shell; in particular, it is common these
days for it to be a symlink to dash. Hence, lots of pain.

Once we know the source of the error, the fix is simple. We can explicitly tell
make to run it's rules with bash:

    $ make SHELL=$(command -v bash)

In a guix environment, this should pick up the bash in $GUIX_ENVIRONMENT.
Better yet, we can do this once, by telling *configure* what shell to use:

    $ CONFIG_SHELL=$(command -v bash) ./configure

and all subsequent runs of make will Just Work from within the environment.

Does this make sense? Am I missing something obvious? If not, would it make
sense to include the above ./configure invocation in our documentation?

Ideally, we could modify Makefile.am or configure.ac in some way to
automatically fix the problem; however, if we want to support build exterior to
a guix build environment, I suspect the "most correct" course of action is to
simply remove our bashisms.

That said, we could tell automake to *not* explicity set make's SHELL by a
simple one-liner in `configure.ac':

    AM_SUBST_NOTMAKE([SHELL])

This will cause make to use it's baked-in default instead of the one autoconf
detects. Luckily, for guix builds make correctly references the profile's sh:

    $ strings $GUIX_ENVIRONMENT/bin/make | grep bin/sh
    /gnu/store/<hash>-bash-<version>/bin/sh

and non-guix make is likely to point to something relevant for that
distribution anyway. That said, I am sure automake and autoconf set make's
SHELL for a reason, so perhaps this solution is too heavy of a hammer.
Thoughts?

Anyway, this turned into a lot longer of an email than I anticipated. Thanks
for taking the time to read through it.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

bug#25258: Bashisms in make rules

[Vincent Legoll]

Hello,

Having done a few bashismectomies lately (in guix-install.sh), I
can sympathetize with you.

I don't know guix team's stance on that kind ow work (I still have not
polished my work enough for review). But it looks like you stumbled on
some painful items, that ought to be resolved one way or the other.

It would be nice to hear that those are actionable items that will
receive positive feedback before doing substantial work. (Looks like
I should have asked that earlier, before falling into the rabbit
hole...)

I personally think we should try to support using guix as widely as
possible/practical on foreign distros.

So, WDGCT (What Do Guix Core Think) ?

Thanks

-- 
Vincent Legoll

bug#25258: Bashisms in make rules

[Tobias Geerinckx-Rice via Bug reports for GNU Guix]

[-- Attachment #1: Type: text/plain, Size: 993 bytes --]

Vincent,

Vincent Legoll 写道：
> I don't know guix team's stance on that kind ow work (I still 
> have not
> polished my work enough for review). But it looks like you 
> stumbled on
> some painful items, that ought to be resolved one way or the 
> other.

The only reason for Guix to have any shell code at all is to run 
in places where Guile isn't (yet) available.  Therefore it should 
be portable between GNU systems, or it might as well be Guile.  I 
don't believe GNU implies bash.

> It would be nice to hear that those are actionable items that 
> will
> receive positive feedback before doing substantial work. (Looks 
> like
> I should have asked that earlier, before falling into the rabbit
> hole...)

Big upz from me.  Thanks for being proactive :-)

The challenge is keeping new bashisms out, so sudo please try to 
include some automated tests.  Maybe you can use shellcheck; our 
test suite could use a GHC dependency.

Kind regards,

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]