Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1867 bytes --]

Small clarification to old discussion:

Hello,

Ludovic Courtès <ludo@gnu.org> writes:

> Hi!
>
> Maxime Devos <maximedevos@telenet.be> skribis:
>
>> Ludovic Courtès schreef op vr 01-04-2022 om 10:58 [+0200]:
>>> As a first milestone, maybe we could start running ‘autoreconf’
more
>>> often, for packages higher in the graph.  We could change the
>>> ‘bootstrap’ build phase to do that unless it’s explicitly turned
off.
>>> It may turn out to be a Sisyphean task though…
>>>
>>> Thoughts?
>>
>> Changing all pre-existing packages, maybe.  But doing this for new
>> packages (reducing review effort) and perhaps when a package is
updated
>> (for purity) should be feasible I think?  Then gradually things
would
>> improve and eventually(TM) doing the switch in the bootstrap phase
may
>> become feasible ...
>
> Yes, we could do that as a first step (in fact it’s already happening
as
> some projects no longer distribute tarballs).
>
> What do maintainers think of that policy?

No strong opinion, but I agree that having a complete development
environment capable of building from the bare sources (e.g. a git tree)
is useful in general.  On the other hand, using tarballs is often
more efficient and practical (it's made to be built by downstream
users,
rather than by developers, so it includes everything needed).  Release
tarballs are also often signed by the projects, which is neat.  So
perhaps we can leave some flexibility there and not make it a hard
rule, but a case of best judgment?

We can both use tarballs _and_ unbundle/regenerate, no need to switch
from tarball to git (though that's one way to do things)!  E.g., adjust
the bootstrap phase to always "autoreconf -vif" and/or add snippets or
post-unpack phases to remove generated or bundled Autotools files.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1661 bytes --]

Hi guix,

Quite some packages in Guix use the Autotools system.  In this system,
a 'configure.ac' and 'Makefile.am' script / makefile is written, from
which 'autoconf' & 'automake' generate a very long bash script and a
Makefile.in.  Depending on the maintainer of the upstream package, this
'configure' and 'Makefile.in' are sometimes included in release
tarballs.

This seems in conflict with:

  * (guix)Submitting patches: ‘Make sure the package does not use
    bundled copies of software already available as separate
    packages.’

    Autotools packages have a 'config.guess' and 'config.sub' script
    that need to be updated whenever there's a new architecture.  As
    such, for some packages, these need to be replaced for aarch64,
    powerpc or risv64.  There are also some packages with very old
    configure scripts that don't support --build/--host/--target,
    which could gain --build/--host/--target support by just
    regenerating them.

    This also makes ensuring a package does not contain any malware
    much harder, because the configure script (and related files)
    needs to be read in their entirity.

 * When an upstream tarball contains .so, .dll, .a, etc. binaries,
   they are removed downstream in a snippet.  Why would the Autotools
   be an exception?

For some ‘early’ packages (gcc, glibc, binutils, ...), there's a
circularity problem, so building 'configure' and 'Makefile.in' from
source might not always be possible, but WDYT of building 'configure' &
'Makefile.in' from source for packages where it does not result in
bootstrapping problems?

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Zhu Zihao]

[-- Attachment #1: Type: text/plain, Size: 978 bytes --]


We can make GNU build system tries autoreconf or autogen.sh first if we
encounter some build issues. But I don't think it's a good idea to
remove configure script in the snippet of origin. Because you can't
track all files that generated by Autotools. It is also burden for
packager to pick the build-time dependencies like libtool,
autoconf-archive, gnulib etc. if we force all autotools baseed packages
use autoreconf

For software developer, they prefer to bundle a generated configure
script because it will be a bootstrap problem sometimes.

A simple examples,the m4 macro GUILE_PKG defined in guile.m4 helps
autotools to check guile installation. If there's no generated configure
script. Software user should have guile.m4 to generate configure script.
A macro that detects guile installation requires a guile installation to
work. Catch 22 :)


-- 
Retrieve my PGP public key:

  gpg --recv-keys D47A9C8B2AE3905B563D9135BE42B352A9F6821F

Zihao

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 255 bytes --]

Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 843 bytes --]

Zhu Zihao schreef op wo 30-03-2022 om 22:31 [+0800]:
> It is also burden for
> packager to pick the build-time dependencies like libtool,
> autoconf-archive, gnulib etc. if we force all autotools baseed
> packages use autoreconf

Reading 12037 lines of 'configure' and 7896 lines of 'Makefile.in'
seems way more of a burden to me than adding two lines

  (native-inputs
    (list libtool autoconf automake gettext-minimal autoconf-archive))

Would be nice to package 'gnulib' in Guix though, currently it's a bit
ad-hoc ...

I don't see any force here, at least not more than _not_ using
autoreconf would force package reviewers to review the 'configure' etc
for malware and people using new architectures to report bugs & patches
to regenerate the Autotools things to recognise the new architecture.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 487 bytes --]

Zhu Zihao schreef op wo 30-03-2022 om 22:31 [+0800]:
> For software developer, they prefer to bundle a generated configure
> script because it will be a bootstrap problem sometimes.

Sometimes that's convenient for upstream, but how is this relevant to
downstream (Guix)?  Downstream does not have to follow upstream on all
matters, and for the early packages where there would be bootstrap
problems (GCC, binutils, glibc ...), an exception could be made.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 520 bytes --]

Zhu Zihao schreef op wo 30-03-2022 om 22:31 [+0800]:
> A simple examples,the m4 macro GUILE_PKG defined in guile.m4 helps
> autotools to check guile installation. If there's no generated
> configure
> script. Software user should have guile.m4 to generate configure
> script.
> A macro that detects guile installation requires a guile installation
> to
> work. Catch 22 :)

Guix has a 'guile' package, which seems unlikely to disappear, so this
does not seem to be a problem for Guix.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 689 bytes --]

Zhu Zihao schreef op wo 30-03-2022 om 22:31 [+0800]:
> Because you can't track all files that generated by Autotools.

You can.  Just look for files named 'configure', 'Makefile.in',
'Makefile.in.in', and to reduce false positives, look inside the files
for the "Generated by ..." string.

Also look for files named 'config.guess', 'config.rpath', 'config.sub',
'ar-lib', 'install-sh', 'missing', 'mdate-sh', 'test-dirver,' 'test-
driver.scm', 'texinfo.tex', 'insert-header.sin', 'boldquot.sed'.

Perhaps this misses *some* of the Autotools files, but it should be
most of them, which eliminates *most* of the extra review work
generated by Autotools.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Liliana Marie Prikler]

Am Mittwoch, dem 30.03.2022 um 14:04 +0200 schrieb Maxime Devos:
> Hi guix,
> 
> Quite some packages in Guix use the Autotools system.  In this
> system, a 'configure.ac' and 'Makefile.am' script / makefile is
> written, from which 'autoconf' & 'automake' generate a very long bash
> script and a Makefile.in.  Depending on the maintainer of the
> upstream package, this 'configure' and 'Makefile.in' are sometimes
> included in release tarballs.
> 
> This seems in conflict with:
> 
>   * (guix)Submitting patches: ‘Make sure the package does not use
>     bundled copies of software already available as separate
>     packages.’
> 
>     Autotools packages have a 'config.guess' and 'config.sub' script
>     that need to be updated whenever there's a new architecture.  As
>     such, for some packages, these need to be replaced for aarch64,
>     powerpc or risv64.  There are also some packages with very old
>     configure scripts that don't support --build/--host/--target,
>     which could gain --build/--host/--target support by just
>     regenerating them.
> 
>     This also makes ensuring a package does not contain any malware
>     much harder, because the configure script (and related files)
>     needs to be read in their entirity.
> 
>  * When an upstream tarball contains .so, .dll, .a, etc. binaries,
>    they are removed downstream in a snippet.  Why would the Autotools
>    be an exception?
Note that many autotools-based packages already require the addition of
autoconf and friends due to being pulled from git.  That being said,
it's somewhat hard to argue for completely dropping them, because
a. simply matching files via ".in" suffix would be error-prone
b. autoreconf should regenerate these files regardless
Therefore, my counter-proposal would be to just simply always run the
bootstrap script or autoreconf, even if the respective files are
tarballed, as well as adding autoconf and automake to the implicit
native inputs of gnu build system.

> For some ‘early’ packages (gcc, glibc, binutils, ...), there's a
> circularity problem
The obvious solution to which would be to implement m4 in mes :)

> [B]uilding 'configure' and 'Makefile.in' from source might not always
> be possible, but WDYT of building 'configure' & 'Makefile.in' from
> source for packages where it does not result in bootstrapping
> problems?
See above, but to reiterate, I'm generally in favor.

Regarding tooling support, I think autotools should have an option to
build a non-bootstrapped dist tarball.  If more upstreams produced such
stripped tarballs, we wouldn't even be having that debate.

Cheers

Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1845 bytes --]

Liliana Marie Prikler schreef op wo 30-03-2022 om 20:55 [+0200]:
> Note that many autotools-based packages already require the addition of
> autoconf and friends due to being pulled from git.  That being said,
> it's somewhat hard to argue for completely dropping them, because
> a. simply matching files via ".in" suffix would be error-prone
> b. autoreconf should regenerate these files regardless
> Therefore, my counter-proposal would be to just simply always run the
> bootstrap script or autoreconf, even if the respective files are
> tarballed, as well as adding autoconf and automake to the implicit
> native inputs of gnu build system.

It should be possible to look for ‘# Generated by GNU Autoconf’ and ‘#
Makefile.in generated by automake’ lines in some 'find-generated-
autotools-fies’or something.

Adding autoconf & automake seems a bit much to me (gnu-build-system is
not necessarily autotools-build-system, the exact required version of
autoconf & automake can vary), but otherwise your proposal seems good
to me.

> > For some ‘early’ packages (gcc, glibc, binutils, ...), there's a
> > circularity problem
> The obvious solution to which would be to implement m4 in mes :)
> 
> > [B]uilding 'configure' and 'Makefile.in' from source might not always
> > be possible, but WDYT of building 'configure' & 'Makefile.in' from
> > source for packages where it does not result in bootstrapping
> > problems?
> See above, but to reiterate, I'm generally in favor.
> 
> Regarding tooling support, I think autotools should have an option to
> build a non-bootstrapped dist tarball.  If more upstreams produced such
> stripped tarballs, we wouldn't even be having that debate.

Yes, would be nice if upstreams could choose to opt-out.  Or maybe opt-
in instead.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Liliana Marie Prikler]

Am Mittwoch, dem 30.03.2022 um 21:24 +0200 schrieb Maxime Devos:
> Liliana Marie Prikler schreef op wo 30-03-2022 om 20:55 [+0200]:
> > Note that many autotools-based packages already require the
> > addition of autoconf and friends due to being pulled from git. 
> > That being said, it's somewhat hard to argue for completely
> > dropping them, because
> > a. simply matching files via ".in" suffix would be error-prone
> > b. autoreconf should regenerate these files regardless
> > Therefore, my counter-proposal would be to just simply always run
> > the bootstrap script or autoreconf, even if the respective files
> > are tarballed, as well as adding autoconf and automake to the
> > implicit native inputs of gnu build system.
> 
> It should be possible to look for ‘# Generated by GNU Autoconf’ and
> ‘# Makefile.in generated by automake’ lines in some 'find-generated-
> autotools-fies’or something.
I don't think that will work correctly.  Case in point: pre-inst-env in
the Guix source tree.

> Adding autoconf & automake seems a bit much to me (gnu-build-system
> is not necessarily autotools-build-system, the exact required version
> of autoconf & automake can vary), but otherwise your proposal seems
> good to me.
We could drop both from the deriving system or simply define a constant
%implicit-gnu-inputs-no-autotools (name subject to bikeshedding).

Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1069 bytes --]

Liliana Marie Prikler schreef op do 31-03-2022 om 06:22 [+0200]:
> > It should be possible to look for ‘# Generated by GNU Autoconf’ and
> > ‘# Makefile.in generated by automake’ lines in some 'find-
> > generated-autotools-fies’or something.
> I don't think that will work correctly.  Case in point: pre-inst-env
> in the Guix source tree.

I don't see the problem.  "guix build guix" would basically do:

1. unpack the tarball / copy the git checkout (which, if I'm not
   mistaken, does not contain pre-inst-env).
2. (if not already done in a snippet): Find the 'Makefile.in' and
   'configure', by looking at the file name + "Generated by ..." lines.
   The build-aux/pre-inst-env.in is left intact, as it is not named
   'configure' or 'Makefile.in' (and additionally, it does not contain
   the "Generated by" string.
3. Run ./bootstrap
4. Run ./configure --localstatedir=/var --prefix=...
   (which creates 'pre-inst-env')
5. Run "make" and "make install"
6. Success!

What would not work correctly here?

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Liliana Marie Prikler]

Am Donnerstag, dem 31.03.2022 um 13:10 +0200 schrieb Maxime Devos:
> Liliana Marie Prikler schreef op do 31-03-2022 om 06:22 [+0200]:
> > > It should be possible to look for ‘# Generated by GNU Autoconf’
> > > and
> > > ‘# Makefile.in generated by automake’ lines in some 'find-
> > > generated-autotools-fies’or something.
> > I don't think that will work correctly.  Case in point: pre-inst-
> > env
> > in the Guix source tree.
> 
> I don't see the problem.  "guix build guix" would basically do:
> 
> 1. unpack the tarball / copy the git checkout (which, if I'm not
>    mistaken, does not contain pre-inst-env).
> 2. (if not already done in a snippet): Find the 'Makefile.in' and
>    'configure', by looking at the file name + "Generated by ..."
> lines.
>    The build-aux/pre-inst-env.in is left intact, as it is not named
>    'configure' or 'Makefile.in' (and additionally, it does not
> contain
>    the "Generated by" string.
> 3. Run ./bootstrap
> 4. Run ./configure --localstatedir=/var --prefix=...
>    (which creates 'pre-inst-env')
> 5. Run "make" and "make install"
> 6. Success!
> 
> What would not work correctly here?
grep "[Gg]enerated" pre-inst-env

Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 515 bytes --]

Liliana Marie Prikler schreef op do 31-03-2022 om 20:24 [+0200]:
> > What would not work correctly here?
> grep "[Gg]enerated" pre-inst-env

I suggest:

  1. check that the file name is "configure" or "Makefile.in"
  2. and search for "[Gg]enerated by (automake|GNU Autoconf)"
  3. If (1) and (2) match, consider it to be a generated file

pre-inst-env doesn't match this, so no false positive in this case.

Just searching for "generated" is indeed too general a search string.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Liliana Marie Prikler]

Am Donnerstag, dem 31.03.2022 um 20:31 +0200 schrieb Maxime Devos:
> Liliana Marie Prikler schreef op do 31-03-2022 om 20:24 [+0200]:
> > > What would not work correctly here?
> > grep "[Gg]enerated" pre-inst-env
> 
> I suggest:
> 
>   1. check that the file name is "configure" or "Makefile.in"
>   2. and search for "[Gg]enerated by (automake|GNU Autoconf)"
>   3. If (1) and (2) match, consider it to be a generated file
> 
> pre-inst-env doesn't match this, so no false positive in this case.
I wasn't arguing for a false positive, but a false negative under the
assumption that we want to remove *all* files generated by Autotools,
not just configure or Makefile.in.

> Just searching for "generated" is indeed too general a search string.
See above, the point I'm making is that no such blurb shows up in files
that don't match (1).

If we want to approximate from below, simply deleting configure should
suffice to rebuild build files (assuming the tarballs was indeed
created by autotools and contains no other build files magically
referenced despite not being bootstrapped), and I don't think space
savings are large enough to be worth it outside of a puritan
perspective (which again would be better served by improving Autotools
so only old bootstrapping tarballs remain affected).

Cheers

Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Ludovic Courtès]

Hi!

Maxime Devos <maximedevos@telenet.be> skribis:

> Quite some packages in Guix use the Autotools system.  In this system,
> a 'configure.ac' and 'Makefile.am' script / makefile is written, from
> which 'autoconf' & 'automake' generate a very long bash script and a
> Makefile.in.  Depending on the maintainer of the upstream package, this
> 'configure' and 'Makefile.in' are sometimes included in release
> tarballs.

Clearly, this is an exception to the rule in Guix, some might say it’s
the elephant in the room.

Debian has long considered that these are not source and so it
regenerates all these files as a first step (IIRC).

This is conceptually the “right thing”, but it raises a significant
bootstrapping problem.  Our bootstrap path currently focuses on being
able to run shell scripts, in particular ‘configure’ scripts, before we
can even build Bash, thanks to Gash and Gash-Utils.  If we were to build
the autotools machinery systematically, we’d need Gash-Utils to provide
a Perl and an M4 implementation.  (Now, Timothy has written an Awk
interpreter, so a Perl interpreter can’t be so hard, right?  :-))
The other option is to write an Autoconf and Automake implementation…

As a first milestone, maybe we could start running ‘autoreconf’ more
often, for packages higher in the graph.  We could change the
‘bootstrap’ build phase to do that unless it’s explicitly turned off.
It may turn out to be a Sisyphean task though…

Thoughts?

Ludo’.

Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 680 bytes --]

Ludovic Courtès schreef op vr 01-04-2022 om 10:58 [+0200]:
> As a first milestone, maybe we could start running ‘autoreconf’ more
> often, for packages higher in the graph.  We could change the
> ‘bootstrap’ build phase to do that unless it’s explicitly turned off.
> It may turn out to be a Sisyphean task though…
>
> Thoughts?

Changing all pre-existing packages, maybe.  But doing this for new
packages (reducing review effort) and perhaps when a package is updated
(for purity) should be feasible I think?  Then gradually things would
improve and eventually(TM) doing the switch in the bootstrap phase may
become feasible ...

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Ludovic Courtès]

Hi!

Maxime Devos <maximedevos@telenet.be> skribis:

> Ludovic Courtès schreef op vr 01-04-2022 om 10:58 [+0200]:
>> As a first milestone, maybe we could start running ‘autoreconf’ more
>> often, for packages higher in the graph.  We could change the
>> ‘bootstrap’ build phase to do that unless it’s explicitly turned off.
>> It may turn out to be a Sisyphean task though…
>>
>> Thoughts?
>
> Changing all pre-existing packages, maybe.  But doing this for new
> packages (reducing review effort) and perhaps when a package is updated
> (for purity) should be feasible I think?  Then gradually things would
> improve and eventually(TM) doing the switch in the bootstrap phase may
> become feasible ...

Yes, we could do that as a first step (in fact it’s already happening as
some projects no longer distribute tarballs).

What do maintainers think of that policy?

Thanks,
Ludo’.

Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Maxim Cournoyer]

Hello,

Ludovic Courtès <ludo@gnu.org> writes:

> Hi!
>
> Maxime Devos <maximedevos@telenet.be> skribis:
>
>> Ludovic Courtès schreef op vr 01-04-2022 om 10:58 [+0200]:
>>> As a first milestone, maybe we could start running ‘autoreconf’ more
>>> often, for packages higher in the graph.  We could change the
>>> ‘bootstrap’ build phase to do that unless it’s explicitly turned off.
>>> It may turn out to be a Sisyphean task though…
>>>
>>> Thoughts?
>>
>> Changing all pre-existing packages, maybe.  But doing this for new
>> packages (reducing review effort) and perhaps when a package is updated
>> (for purity) should be feasible I think?  Then gradually things would
>> improve and eventually(TM) doing the switch in the bootstrap phase may
>> become feasible ...
>
> Yes, we could do that as a first step (in fact it’s already happening as
> some projects no longer distribute tarballs).
>
> What do maintainers think of that policy?

No strong opinion, but I agree that having a complete development
environment capable of building from the bare sources (e.g. a git tree)
is useful in general.  On the other hand, using tarballs is often more
efficient and practical (it's made to be built by downstream users,
rather than by developers, so it includes everything needed).  Release
tarballs are also often signed by the projects, which is neat.  So
perhaps we can leave some flexibility there and not make it a hard rule,
but a case of best judgment?

Thanks,

Maxim

Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Ludovic Courtès]

Hi,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

[...]

>>> Changing all pre-existing packages, maybe.  But doing this for new
>>> packages (reducing review effort) and perhaps when a package is updated
>>> (for purity) should be feasible I think?  Then gradually things would
>>> improve and eventually(TM) doing the switch in the bootstrap phase may
>>> become feasible ...
>>
>> Yes, we could do that as a first step (in fact it’s already happening as
>> some projects no longer distribute tarballs).
>>
>> What do maintainers think of that policy?
>
> No strong opinion, but I agree that having a complete development
> environment capable of building from the bare sources (e.g. a git tree)
> is useful in general.  On the other hand, using tarballs is often more
> efficient and practical (it's made to be built by downstream users,
> rather than by developers, so it includes everything needed).  Release
> tarballs are also often signed by the projects, which is neat.

Right, authentication can be a good reason to keep using tarballs.

Projects that publish signed tarballs are likely to use signed tags as
well though.  We just need to update ‘guix refresh’ to handle signed
tags.

(Another argument, this time against tarballs, is archival, though that
part if now partly addressed thanks to Disarchive.)

> So perhaps we can leave some flexibility there and not make it a hard
> rule, but a case of best judgment?

Yeah.

Thanks,
Ludo’.

Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[zimoun]

Hi,

On Tue, 05 Apr 2022 at 14:06, Ludovic Courtès <ludo@gnu.org> wrote:

>> Changing all pre-existing packages, maybe.  But doing this for new
>> packages (reducing review effort) and perhaps when a package is updated
>> (for purity) should be feasible I think?  Then gradually things would
>> improve and eventually(TM) doing the switch in the bootstrap phase may
>> become feasible ...
>
> Yes, we could do that as a first step (in fact it’s already happening as
> some projects no longer distribute tarballs).

Moreover, what appears to me unexpected is that the ’gnu-build-system’
run ’autoreconf’ but Autotools are not part of the build inputs by
default and the user has to explicitly add them.

--8<---------------cut here---------------start------------->8---
(define* (bootstrap #:key (bootstrap-scripts %bootstrap-scripts)
                    #:allow-other-keys)
  "If the code uses Autotools and \"configure\" is missing, run
\"autoreconf\".  Otherwise do nothing."

[...]

  (if (not (script-exists? "configure"))

[...]
                (invoke "autoreconf" "-vif")
[...]
--8<---------------cut here---------------end--------------->8---

For sure, it leads to some issues for the deep nodes in the bootstrap
chain.  And it also leads to some other issues as fetching tarballs for
saving bandwidth or for checking signature, etc.

However, IMHO, the logic should be reverted:

 - include by default Autotools and run ’autoreconf’ by default
 - for some cases, use the provided ’./configure’ script


Cheers,
simon

Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Jonathan McHugh]

Hi Ludo,

April 1, 2022 11:01 AM, "Ludovic Courtès" <ludo@gnu.org> wrote:

> 
> This is conceptually the “right thing”, but it raises a significant
> bootstrapping problem. Our bootstrap path currently focuses on being
> able to run shell scripts, in particular ‘configure’ scripts, before we
> can even build Bash, thanks to Gash and Gash-Utils. If we were to build
> the autotools machinery systematically, we’d need Gash-Utils to provide
> a Perl and an M4 implementation. (Now, Timothy has written an Awk
> interpreter, so a Perl interpreter can’t be so hard, right? :-))
> The other option is to write an Autoconf and Automake implementation…
> 

A huge Awk fan, have you got a reference for Timothy's interpreter?

> As a first milestone, maybe we could start running ‘autoreconf’ more
> often, for packages higher in the graph. We could change the
> ‘bootstrap’ build phase to do that unless it’s explicitly turned off.
> It may turn out to be a Sisyphean task though…
> 
> Thoughts?
> 
> Ludo’.



====================
Jonathan McHugh
indieterminacy@libre.brussels