Annoyingly cautious make rules

Annoyingly cautious make rules

[Richard Stallman]

I ran config.status, hoping to avoid the 10 minutes it takes to run
configure on my machine, since I did that yesterday.  But
config.status ran configure.

This is correct in some rigid theoretical sense, but it is nasty to
disregard what the user asks for.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Annoyingly cautious make rules

[Paul Eggert]

On 11/30/11 13:48, Richard Stallman wrote:
> I ran config.status, hoping to avoid the 10 minutes it takes to run
> configure on my machine, since I did that yesterday.  But
> config.status ran configure.

That doesn't happen with me (see below).
From the Subject: line, it sounds like you're saying
that a Makefile rule contains a dependency that it shouldn't,
but it's not clear which dependency you're referring to.
Currently we have:

config.status: configure lisp/version.el # This one actually runs 'configure'.
configure: configure.in aclocal.m4

Are any of these dependencies the one that caused your problem?


$ ./config.status
config.status: creating Makefile
config.status: creating lib/Makefile
config.status: creating lib-src/Makefile
config.status: creating oldXMenu/Makefile
config.status: creating doc/emacs/Makefile
config.status: creating doc/misc/Makefile
config.status: creating doc/lispintro/Makefile
config.status: creating doc/lispref/Makefile
config.status: creating src/Makefile
config.status: creating lwlib/Makefile
config.status: creating lisp/Makefile
config.status: creating leim/Makefile
config.status: creating test/automated/Makefile
config.status: creating src/config.h
config.status: src/config.h is unchanged
config.status: executing depfiles commands
config.status: executing mkdirs commands
config.status: executing epaths commands
creating src/epaths.h
[ -r "/home/eggert/src/gnu/emacs/static-checking/src/config.in" ] || ( cd /home/eggert/src/gnu/emacs/static-checking && autoheader )
config.status: executing gdbinit commands
$ 

Re: Annoyingly cautious make rules

[Eli Zaretskii]

> Date: Wed, 30 Nov 2011 16:48:54 -0500
> From: Richard Stallman <rms@gnu.org>
> 
> I ran config.status, hoping to avoid the 10 minutes it takes to run
> configure on my machine, since I did that yesterday.

If you want to avoid running configure as much as possible, but still
run it when necessary (to avoid breaking the build), always run
"make".  It will run configure for you when needed (by the virtue of
rules in GNUMakefile), or run config.status, or none of them at all,
getting directly to compilation and linking.

Re: Annoyingly cautious make rules

[Andreas Schwab]

Richard Stallman <rms@gnu.org> writes:

> I ran config.status, hoping to avoid the 10 minutes it takes to run
> configure on my machine, since I did that yesterday.  But
> config.status ran configure.

If lisp/version.el changes this is correct.

> This is correct in some rigid theoretical sense, but it is nasty to
> disregard what the user asks for.

If you want to speed up configure you should enable the cache.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Annoyingly cautious make rules

[Richard Stallman]

    Are any of these dependencies the one that caused your problem?

    configure: configure.in aclocal.m4

It might have been that one: configure.in was changed.

Thanks.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Annoyingly cautious make rules

[Stefan Monnier]

>     Are any of these dependencies the one that caused your problem?
>     configure: configure.in aclocal.m4
> It might have been that one: configure.in was changed.

Over the last few months, configure.in has changed many more times than
in the "previous past", I think.


        Stefan

Re: Annoyingly cautious make rules

[Paul Eggert]

On 12/01/11 13:54, Richard Stallman wrote:
>     Are any of these dependencies the one that caused your problem?
> 
>     configure: configure.in aclocal.m4
> 
> It might have been that one: configure.in was changed.

To avoid this problem, you can run 'configure' without
specifying the --enable-maintainer-mode option.  That's
the option that generates the dependency that gave
you the problem.  I guess the idea is that maintainers
typically want to be more cautious.

"make bootstrap" enables maintainer mode if it is run in
a freshly checked-out source tree, so if you did that you'll
also have the problem.  I don't know why "make bootstrap"
sometimes (but not always) enables maintainer mode; this
was introduced in bzr 88802 dated 2008-06-20 but the ChangeLog
doesn't say why the maintainer-mode enabling was added.

Perhaps "make bootstrap" should not enable maintainer mode.
That might prevent the kind of problem that you ran into.

Re: Annoyingly cautious make rules

[Richard Stallman]

    If you want to speed up configure you should enable the cache.

The cache used to be enabled automatically.  How do you enable it now?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Annoyingly cautious make rules

[Andy Wingo]

On Fri 02 Dec 2011 13:05, Richard Stallman <rms@gnu.org> writes:

>     If you want to speed up configure you should enable the cache.
>
> The cache used to be enabled automatically.  How do you enable it now?

Pass the -C option to ./configure.  That will create config.cache when
configure is run.  If config.cache is present it will be used.

Regards,

Andy
-- 
http://wingolog.org/

Re: Annoyingly cautious make rules

[Andreas Schwab]

Richard Stallman <rms@gnu.org> writes:

> The cache used to be enabled automatically.  How do you enable it now?

$ ./configure --help
[...]
      --cache-file=FILE   cache test results in FILE [disabled]
  -C, --config-cache      alias for `--cache-file=config.cache'
[...]

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Annoyingly cautious make rules

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Date: Fri, 02 Dec 2011 13:13:32 +0100
> Cc: emacs-devel@gnu.org
> 
> Richard Stallman <rms@gnu.org> writes:
> 
> > The cache used to be enabled automatically.  How do you enable it now?
> 
> $ ./configure --help
> [...]
>       --cache-file=FILE   cache test results in FILE [disabled]
>   -C, --config-cache      alias for `--cache-file=config.cache'
> [...]

Why is the default OFF?

Re: Annoyingly cautious make rules

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andreas Schwab <schwab@linux-m68k.org>
>> Date: Fri, 02 Dec 2011 13:13:32 +0100
>> Cc: emacs-devel@gnu.org
>> 
>> Richard Stallman <rms@gnu.org> writes:
>> 
>> > The cache used to be enabled automatically.  How do you enable it now?
>> 
>> $ ./configure --help
>> [...]
>>       --cache-file=FILE   cache test results in FILE [disabled]
>>   -C, --config-cache      alias for `--cache-file=config.cache'
>> [...]
>
> Why is the default OFF?

Did you mean to send this mail to autoconf@gnu.org?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Annoyingly cautious make rules

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Date: Fri, 02 Dec 2011 16:05:20 +0100
> Cc: rms@gnu.org, emacs-devel@gnu.org
> 
> >> $ ./configure --help
> >> [...]
> >>       --cache-file=FILE   cache test results in FILE [disabled]
> >>   -C, --config-cache      alias for `--cache-file=config.cache'
> >> [...]
> >
> > Why is the default OFF?
> 
> Did you mean to send this mail to autoconf@gnu.org?

No, I thought there were nice people here that could help.

Re: Annoyingly cautious make rules

[Stefan Monnier]

>> > The cache used to be enabled automatically.  How do you enable it now?
>> $ ./configure --help
>> [...]
>> --cache-file=FILE   cache test results in FILE [disabled]
>> -C, --config-cache      alias for `--cache-file=config.cache'
>> [...]

> Why is the default OFF?

IIUC it's because the expectation is that it's not run very often and
that current machines are fast enough to make it bearable, the advantage
being that when you "aptitude upgrade" you won't waste your time
wondering why configure does not notice that your libjpeg62 doesn't
exist any more and how to coerce configure into doing its job
(i.e. find out that you need to remove the config.cache that you'd
never heard of).
I.e. the problem of autoconf's cache is that there's no automatic way to
discover when it's stale.


        Stefan

Re: Annoyingly cautious make rules

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Date: Fri, 02 Dec 2011 11:14:03 -0500
> Cc: Andreas Schwab <schwab@linux-m68k.org>, rms@gnu.org, emacs-devel@gnu.org
> 
> >> > The cache used to be enabled automatically.  How do you enable it now?
> >> $ ./configure --help
> >> [...]
> >> --cache-file=FILE   cache test results in FILE [disabled]
> >> -C, --config-cache      alias for `--cache-file=config.cache'
> >> [...]
> 
> > Why is the default OFF?
> 
> IIUC it's because the expectation is that it's not run very often and
> that current machines are fast enough to make it bearable, the advantage
> being that when you "aptitude upgrade" you won't waste your time
> wondering why configure does not notice that your libjpeg62 doesn't
> exist any more and how to coerce configure into doing its job
> (i.e. find out that you need to remove the config.cache that you'd
> never heard of).
> I.e. the problem of autoconf's cache is that there's no automatic way to
> discover when it's stale.

Thanks.

Re: Annoyingly cautious make rules

[Paul Eggert]

On 12/02/11 08:14, Stefan Monnier wrote:
> IIUC it's because the expectation is that it's not run very often and
> that current machines are fast enough to make it bearable,

Both assumptions are false for me.  I run 'configure' often
(either directly, or indirectly because of 'make'), I have a
current desktop (purchased within the past year), and 'configure'
is not parallelized and takes 48 seconds real time.
That's waaaayy too slow and detracts from getting real work done.
It sounds like Richard has a similar problem, except that his
machine is slower so his problem is even worse.

Some of this problem comes back the 2011-03-20 patch that added
the autogen subdir.  That patch also made --enable-maintainer-mode
the default, which wasn't an integral part of the autogen stuff
and which (given the above issues) seems like it may not have been
such a good idea.

Another part of the problem is that even if I use "configure -C",
"make" sometimes invokes "configure" without -C, which is reeeeally
slow.

And another part of this problem is that "configure -C" is poorly
publicized.  (Perhaps -C should be the default, but that's a bigger
change.)

Here's a patch that addresses the above issues, without making
-C the default.

=== modified file 'ChangeLog'
--- ChangeLog	2011-11-27 18:33:17 +0000
+++ ChangeLog	2011-12-02 18:13:04 +0000
@@ -1,3 +1,17 @@
+2011-12-02  Paul Eggert  <eggert@cs.ucla.edu>
+
+	configure: Disable maintainer mode by default and suggest -C.
+	This causes make rules to be less annoyingly-cautious; see
+	<http://lists.gnu.org/archive/html/emacs-devel/2011-11/msg00547.html>.
+	* INSTALL.BZR, autogen.sh: Suggest "configure -C" instead of plain
+	"configure".
+	* Makefile.in (CONFIGUREFLAGS): New macro.
+	(config.status, bootstrap): Use it.  This preserves -C and
+	--enable-maintainer-mode in subsidiary calls to "configure".
+	* configure.in (maintainer-mode): Change the default back to "no".
+	This was changed to 'yes' on 2011-03-20 as part of the autogen patch,
+	but the dependencies it adds cause problems in practice.
+
 2011-11-27  Jan Djärv  <jan.h.d@swipnet.se>
 
 	* configure.in: Check for gtk_window_set_has_resize_grip.

=== modified file 'INSTALL.BZR'
--- INSTALL.BZR	2011-06-25 17:51:03 +0000
+++ INSTALL.BZR	2011-12-02 17:54:11 +0000
@@ -26,7 +26,7 @@
 You can then configure your build (use `./configure --help' to see
 options you can set):
 
-  $ ./configure
+  $ ./configure -C
 
 Some of the files that are included in the Emacs tarball, such as
 byte-compiled Lisp files, are not stored in Bazaar.  Therefore, to

=== modified file 'Makefile.in'
--- Makefile.in	2011-11-22 01:56:49 +0000
+++ Makefile.in	2011-12-02 18:11:50 +0000
@@ -66,6 +66,8 @@
 
 # ==================== Things `configure' Might Edit ====================
 
+CONFIGUREFLAGS=--cache-file=@cache_file@@MAINT@ --enable-maintainer-mode
+
 CC=@CC@
 CFLAGS=@CFLAGS@
 LDFLAGS=@LDFLAGS@
@@ -404,7 +406,7 @@
 	if [ -x ./config.status ]; then	\
 	    ./config.status --recheck;	\
 	else				\
-	    ./configure;		\
+	    ./configure $(CONFIGUREFLAGS); \
 	fi
 
 AUTOCONF_INPUTS = @MAINT@ $(srcdir)/configure.in $(srcdir)/aclocal.m4
@@ -936,7 +938,7 @@
 	if [ -x ./config.status ]; then           \
 	    ./config.status;                      \
 	else                                      \
-	    ./configure --enable-maintainer-mode; \
+	    ./configure $(CONFIGUREFLAGS);        \
 	fi
 	$(MAKE) $(MFLAGS) info all
 

=== modified file 'autogen.sh'
--- autogen.sh	2011-03-31 04:24:03 +0000
+++ autogen.sh	2011-12-02 17:51:37 +0000
@@ -209,7 +209,7 @@
 ## Let autoreconf figure out what, if anything, needs doing.
 autoreconf -i -I m4 || exit $?
 
-echo "You can now run \`./configure'."
+echo "You can now run \`./configure -C'."
 
 exit 0
 

=== modified file 'configure.in'
--- configure.in	2011-11-30 16:23:05 +0000
+++ configure.in	2011-12-02 18:11:11 +0000
@@ -182,6 +182,9 @@
 dnl http://lists.gnu.org/archive/html/emacs-devel/2008-04/msg01844.html
 OPTION_DEFAULT_ON([makeinfo],[don't require makeinfo for building manuals])
 
+## Makefile.in needs the cache file name.
+AC_SUBST(cache_file)
+
 ## This is an option because I do not know if all info/man support
 ## compressed files, nor how to test if they do so.
 OPTION_DEFAULT_ON([compress-info],[don't compress the installed Info pages])
@@ -232,11 +235,11 @@
       USE_XASSERTS=no)
 
 AC_ARG_ENABLE(maintainer-mode,
-[AS_HELP_STRING([--disable-maintainer-mode],
-                [disable make rules and dependencies not useful (and sometimes
+[AS_HELP_STRING([--enable-maintainer-mode],
+                [enable make rules and dependencies not useful (and sometimes
 		 confusing) to the casual installer])],
       USE_MAINTAINER_MODE=$enableval,
-      USE_MAINTAINER_MODE=yes)
+      USE_MAINTAINER_MODE=no)
 if test $USE_MAINTAINER_MODE = yes; then
   MAINT=
 else

Re: Annoyingly cautious make rules

[Glenn Morris]

Paul Eggert wrote:

> +	configure: Disable maintainer mode by default

I would suggest not doing this.

I believe the default build rules ought to be what is most correct, not
what is mostly correct but fast, since the latter can lead to confusing
errors for people who are not familiar with all the details. So I
suggest that those of you who find configure too slow and who know what
you are doing disable maintainer mode in your personal copies, rather
than disabling it for everyone.

Ref eg

http://lists.gnu.org/archive/html/emacs-devel/2011-03/msg00862.html

and the cited automake documentation.

Re: Annoyingly cautious make rules

[Paul Eggert]

On 12/02/2011 10:39 AM, Glenn Morris wrote:
> I suggest that those of you who find configure too slow and who know what
> you are doing disable maintainer mode in your personal copies

Even if it is a good idea to enable these problematic dependencies
by default, surely there's no question that "maintainer mode" (whatever
we decide it to be) should not be the default.  Maintainer mode should
cater to experts, not to casual and unskilled builders, and it's
pretty confusing to say (as we do now) that you should disable maintainer
mode only if you're an expert and you know what you're doing.

In other words, if we stick with the 2011-03-20 change to enable
the dependencies by default, --enable-maintainer-mode should
*disable* those dependencies.

> I believe the default build rules ought to be what is most correct, not
> what is mostly correct but fast, since the latter can lead to confusing
> errors for people who are not familiar with all the details.

We agree about this, but the disagreement is over whether these problematic
dependencies are more "correct".  In an environment where Autoconf isn't
installed, or is the wrong version (or similarly for m4, Automake, etc.),
these problematic dependencies are more likely to cause problems than to
cure them.  For example, there are plausible use cases where a naive builder copies
files around and then gets stuck because an unnecessary autoconf invocation
fails.  So there's a good case to be made that it's more "correct"
(for the casual, unskilled builder) to omit these dependencies, as we did
before 2011-03-20.

Re: Annoyingly cautious make rules

[Stefan Monnier]

> Some of this problem comes back the 2011-03-20 patch that added
> the autogen subdir.  That patch also made --enable-maintainer-mode
> the default, which wasn't an integral part of the autogen stuff

Actually, I think it is an integral part: the old setup did not need
maintainer-mode because the configure file was auto-updated by "cvs/bzr
update", but since this file is not under revisions control any more,
maintainer-mode is needed to keep the file up to date.

> Another part of the problem is that even if I use "configure -C",
> "make" sometimes invokes "configure" without -C, which is
> reeeeally slow.

That needs fixing, then.


        Stefan "whose office desktop is a Fit-PC2"

Re: Annoyingly cautious make rules

[Paul Eggert]

On 12/02/11 12:36, Stefan Monnier wrote:
> the old setup did not need
> maintainer-mode because the configure file was auto-updated by "cvs/bzr
> update", but since this file is not under revisions control any more,
> maintainer-mode is needed to keep the file up to date.

Sorry, I don't follow.  The 2011-03-20 change put a copy of 'configure'
into autogen/configure, and autogen/copy_autogen creates an up-to-date
copy of 'configure' from 'autogen/configure'.  So I don't see why
the problematic dependencies are needed after autogen/copy_autogen is run.  It's true
that 'configure' is not under revision control, but the problematic dependencies
aren't needed to keep it 'configure' up to date if one uses copy_autogen.

Or perhaps what you're saying is that the dependences are needed if one wants to
edit 'configure.in', then do a 'make' and then run autogen/update_autogen?
That's true -- but only maintainers do that, so it's OK to require
maintainer-mode for that, and a maintainer should invoke the
pre-2011-03-20-style 'configure --enable-maintainer-mode'
to enable those dependencies before doing something heavy-duty like that.

Re: Annoyingly cautious make rules

[Stefan Monnier]

>> the old setup did not need
>> maintainer-mode because the configure file was auto-updated by "cvs/bzr
>> update", but since this file is not under revisions control any more,
>> maintainer-mode is needed to keep the file up to date.

> Sorry, I don't follow.  The 2011-03-20 change put a copy of
> 'configure' into autogen/configure, and autogen/copy_autogen creates
> an up-to-date copy of 'configure' from 'autogen/configure'.
> So I don't see why the problematic dependencies are needed after
> autogen/copy_autogen is run.  It's true that 'configure' is not under
> revision control, but the problematic dependencies aren't needed to
> keep it 'configure' up to date if one uses copy_autogen.

Those changes were all about building configure locally from
configure.in rather than getting it from the VCS.  copy-autogen is only
a fallback case that's expected to be used only by those very few people
who don't have an autoconf that's sufficiently up to date.


        Stefan

Re: Annoyingly cautious make rules

[Andreas Schwab]

Paul Eggert <eggert@cs.ucla.edu> writes:

> Sorry, I don't follow.  The 2011-03-20 change put a copy of 'configure'
> into autogen/configure, and autogen/copy_autogen creates an up-to-date
> copy of 'configure' from 'autogen/configure'.  So I don't see why
> the problematic dependencies are needed after autogen/copy_autogen is run.  It's true
> that 'configure' is not under revision control, but the problematic dependencies
> aren't needed to keep it 'configure' up to date if one uses copy_autogen.

That wouldn't help for this case since you must rerun configure anyway
when it changes, not matter how you update it.  This has nothing to do
with maintainer mode.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Annoyingly cautious make rules

[Paul Eggert]

On 12/02/11 16:34, Andreas Schwab wrote:
> you must rerun configure anyway when it changes, not matter how you
> update it.  This has nothing to do with maintainer mode.

Yes, and that was the point I was trying to make (evidently I was
not clear enough): the main 2011-03-20 changes are independent of
maintainer mode, and these changes do not motivate making
maintainer mode the default for ordinary builds.


On 12/02/11 15:26, Stefan Monnier wrote:
> Those [2011-03-20] changes were all about building configure locally from
> configure.in rather than getting it from the VCS.  copy-autogen is only
> a fallback case that's expected to be used only by those very few people
> who don't have an autoconf that's sufficiently up to date.

Yes, but regardless of whether one uses autogen.sh or
copy_autogen to build 'configure' locally, 'configure' is up to
date afterwards.  So either way, the maintainer-mode dependencies
do not help and are not needed.

I suspect the reasoning was that, if you run autogen.sh, then
you're a maintainer and you most likely want maintainer mode.
That makes sense.  And we can implement that by having autogen.sh
run 'configure' in maintainer mode, or having it recommend running
'configure' that way.  But that doesn't mean that maintainer mode
should be the default for ordinary builds.

Re: Annoyingly cautious make rules

[Stefan Monnier]

> date afterwards.  So either way, the maintainer-mode dependencies
> do not help and are not needed.

I don't think I can agree with this statement until it gets refined to
more specific dependencies.


        Stefan

Re: Annoyingly cautious make rules

[Paul Eggert]

On 12/02/11 19:42, Stefan Monnier wrote:
>> date afterwards.  So either way, the maintainer-mode dependencies
>> do not help and are not needed.
> 
> I don't think I can agree with this statement until it gets refined to
> more specific dependencies.

Here's the complete list of dependencies enabled by maintainer-mode:

  configure: configure.in aclocal.m4
  aclocal.m4: m4/gl-comp.m4
  lib/Makefile.in: aclocal.m4 lib/Makefile.am lib/gnulib.mk

autogen.sh runs autoreconf, which updates aclocal.m4,
configure, and lib/Makefile.in (in that order).  So,
after autogen.sh is run, the dependencies are already
satisfied, and "make" does not need them and they do
not help the build process.

The autogen/copy_autogen fallback also updates these three
files in the same order, so it works like autogen.sh in
this respect.

Re: Annoyingly cautious make rules

[Stephen J. Turnbull]

Eli Zaretskii writes:
 > > From: Andreas Schwab <schwab@linux-m68k.org>
 > > Date: Fri, 02 Dec 2011 13:13:32 +0100
 > > Cc: emacs-devel@gnu.org
 > > 
 > > Richard Stallman <rms@gnu.org> writes:
 > > 
 > > > The cache used to be enabled automatically.  How do you enable it now?

 > Why is the default OFF?

Because *persistently* caching information about system configurations
is a very bad idea, *especially* a GNU system.  (I don't think I've
ever had an Emacs build break due to a Linux upgrade, so I'll leave
Linux out of this. :-)  I would guess I have to rebuild my stable
Emacsen about once a month because some library "just disappears" (ie,
the SO version gets bumped and the old library gets removed because my
local Emacs builds don't have dependencies recorded in portage) in a
Gentoo upgrade, and they won't run at all.  I would imagine subtle
changes that should be reconfirmed by configure are happening almost
daily.

This is especially useful in the case of occasional beta testers, who
(like me) tend to upgrade system utilities a lot and get bit by such
issues all the time.  Emacs evidently has a more complex process and
needs the bootstrap stage which is more costly than XEmacs's "start
from scratch" process, but in XEmacs the biggest improvement we made
in the process for testers to get a clean build first time after
pulling new code into their source tree was disabling the cache.

This can be a bit annoying for developers who don't work on configure
(of course those of us who do work on configure need to run it often) 
but developers know how to type "./configure --help" and/or send mail
to the dev list, so we shouldn't need to worry about them too much.

It's a different matter if you do a lot of recursive configuration, of
course.  Then the cache is the way you communicate that checks have
already been done to configure subprocesses.  XEmacs does very little
so disabling the cache is obviously TRT.  I don't know about Emacs (I
just run "./configure && make" and go teach class :-).

Re: Annoyingly cautious make rules

[Stephen J. Turnbull]

Paul Eggert writes:
 > On 12/02/11 08:14, Stefan Monnier wrote:
 > > IIUC it's because the expectation is that it's not run very often and
 > > that current machines are fast enough to make it bearable,
 > 
 > Both assumptions are false for me.

But the assumption that you can find "configure -C" easily is true,
no?  Let's optimize the default for the people for whom doing the
wrong thing can be quite expensive.  It costs you and Richard minutes
waiting for configure, and from a few seconds to a day to learn about
"-C", but it can cost an occasional tester days if their build breaks
and they have no idea that it's a configure bug rather than a real
build issue.

 > And another part of this problem is that "configure -C" is poorly
 > publicized.  (Perhaps -C should be the default, but that's a bigger
 > change.)

It used to be default.  But for reasons given elsewhere, that really
sucked for people who should *never* need to worry about autoconf's
foot-shooting proclivities.  It took years to get that changed.

 > Here's a patch that addresses the above issues, without making
 > -C the default.

Thank you!

 > --- INSTALL.BZR	2011-06-25 17:51:03 +0000
 > +++ INSTALL.BZR	2011-12-02 17:54:11 +0000
 > @@ -26,7 +26,7 @@
 >  You can then configure your build (use `./configure --help' to see
 >  options you can set):
 >  
 > -  $ ./configure
 > +  $ ./configure -C

Please, no.  "-C" is an heuristic, not an optimization.  Write this
instead:

    You can then configure your build (use `./configure --help' to see
    options you can set):

    $ ./configure

    (If you run configure often, it can be annoyingly slow.  You can
    use "./configure -C" to enable a cache, which does speed things up
    dramatically, but at the cost of correctness: your build *will*
    break occasionally over a period of months if you use "-C".)

BTW, in

$ ./configure -C
$ ./configure

does the second configure step read the cache?  Does it leave an
existing cache behind?

 > -echo "You can now run \`./configure'."
 > +echo "You can now run \`./configure -C'."

Please, no.  "-C" is an heuristic, not an optimization.  It should be
available to those who need it, but advertising it is inviting trouble
for the testers and the folks who help testers on this list and others.

Re: Annoyingly cautious make rules

[Stefan Monnier]

>>> date afterwards.  So either way, the maintainer-mode dependencies
>>> do not help and are not needed.
>> I don't think I can agree with this statement until it gets refined to
>> more specific dependencies.
> Here's the complete list of dependencies enabled by maintainer-mode:

>   configure: configure.in aclocal.m4
>   aclocal.m4: m4/gl-comp.m4
>   lib/Makefile.in: aclocal.m4 lib/Makefile.am lib/gnulib.mk

> autogen.sh runs autoreconf, which updates aclocal.m4,

autogen.sh is expected to be run once after checking out a new tree.
After that, "make" should take care of rebuilding configure if/when
needed, so the dependencies all seem right.

> configure, and lib/Makefile.in (in that order).  So,
> after autogen.sh is run, the dependencies are already
> satisfied, and "make" does not need them and they do
> not help the build process.

Indeed in this situation they don't help but neither do they hurt.
And they do help when you run "make" after configure.in was modified.


        Stefan

Re: Annoyingly cautious make rules

[Paul Eggert]

On 12/02/11 21:48, Stefan Monnier wrote:
> After that, "make" should take care of rebuilding configure if/when
> needed

There's no practical way for "make" to do that, and Emacs's
current make rules do not do that.  For example, if I manually
edit Makefile.in so that the rule for 'configure' looks like this:

	$(srcdir)/configure: $(AUTOCONF_INPUTS)
		cd ${srcdir} && autoconf
		echo '#foo' >>$@ # <-- I added this line.

then "make" should rebuild "configure".  But "make" does not,
as "Makefile.in" is not a dependency for "configure".

We could fix this particular problem by adding "Makefile.in" as a
dependency for every file that gets built in every directory.
But (1) even that would not suffice in general, and (2) that would
cause more builds to run unnecessarily and annoyingly slowly.

Emacs makefiles, like all practical makefiles, are an
engineering compromise between correctness and speed.  There's
no practical way to get 100% correctness with "make" after an
arbitrary maintainer change to source files.  Instead, we strive for
correctness a high percentage of the time, in such a way that
maintainers should have a good idea of all the issues that might cause
incorrectness and how to avoid these issues.

One possible rule of thumb, for example, is the idea that
"make" runs fairly quickly and works unless you have edited
the build procedure, and in the latter case you should run
"make bootstrap" which is much safer but also much slower.
That is an easy guideline to remember, and surely we can get this
to work a high percentage of the time.  Perhaps you prefer
a different guideline, and if so we can try to get that
other guideline to work.  But the rule cannot be that
"make" always works unaided: that's not practical.

Re: Annoyingly cautious make rules

[Eli Zaretskii]

> Date: Fri, 02 Dec 2011 13:29:21 -0800
> From: Paul Eggert <eggert@cs.ucla.edu>
> CC: Eli Zaretskii <eliz@gnu.org>, Andreas Schwab <schwab@linux-m68k.org>, 
>  rms@gnu.org, emacs-devel@gnu.org
> 
> On 12/02/11 12:36, Stefan Monnier wrote:
> > the old setup did not need
> > maintainer-mode because the configure file was auto-updated by "cvs/bzr
> > update", but since this file is not under revisions control any more,
> > maintainer-mode is needed to keep the file up to date.
> 
> Sorry, I don't follow.  The 2011-03-20 change put a copy of 'configure'
> into autogen/configure, and autogen/copy_autogen creates an up-to-date
> copy of 'configure' from 'autogen/configure'.  So I don't see why
> the problematic dependencies are needed after autogen/copy_autogen is run.

Don't forget that 'autogen/configure' could be outdated, because the
job that commits it runs only once in a while.  During that time
window, running 'autogen/configure' will not DTRT.

Re: Annoyingly cautious make rules

[Andreas Schwab]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 12/02/11 16:34, Andreas Schwab wrote:
>> you must rerun configure anyway when it changes, not matter how you
>> update it.  This has nothing to do with maintainer mode.
>
> Yes, and that was the point I was trying to make (evidently I was
> not clear enough): the main 2011-03-20 changes are independent of
> maintainer mode, and these changes do not motivate making
> maintainer mode the default for ordinary builds.

I don't understand.  It is vital that configure is up-to-date
wrt. configure.in.  That's what the rule does, nothing more, nothing
less.  Before the change, the committer of configure.in had to commit
configure as well, to keep it updated.  Now this is done by make the
next time it is run.  The outcome is exactly the same, except that now
you are guaranteed that configure is up-to-date wrt. configure.in,
whereas before that change you were dependent on the one who changed
configure.in (who often didn't update configure as well, and also caused
a lot of version churn do to different autoconf versions).  So the
current situation is a overall win, but it didn't really change in a
substantial way.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Annoyingly cautious make rules

[Richard Stallman]

    > Why is the default OFF?

    Did you mean to send this mail to autoconf@gnu.org?

No.  I don't know why the cache is off, or which package the decision
is in.


-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Annoyingly cautious make rules

[Richard Stallman]

    IIUC it's because the expectation is that it's not run very often and
    that current machines are fast enough to make it bearable, the advantage
    being that when you "aptitude upgrade" you won't waste your time
    wondering why configure does not notice that your libjpeg62 doesn't
    exist any more and how to coerce configure into doing its job

I see.  Thanks.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Annoyingly cautious make rules

[Paul Eggert]

On 12/03/11 00:49, Andreas Schwab wrote:
> It is vital that configure is up-to-date wrt. configure.in.

It's vital sometimes, but usually not.  For example, if you look
at the last 10 changes to configure.in (in bzrs 106255 through 106559),
only one (bzr 106532) has any relevance to any of the platforms that we
routinely develop Emacs on.  The others are trivial spelling fixes,
or patches for obscure platforms hardly anybody uses, or updating
version numbers that don't matter when developing.  Each of these
unimportant changes causes significant unnecessary delays in development
builds, delays that we didn't have before the 2011-03-20 change.
It'd be better if we didn't have those delays.

Re: Annoyingly cautious make rules

[Paul Eggert]

On 12/02/11 12:36, Stefan Monnier wrote:
>> > Another part of the problem is that even if I use "configure -C",
>> > "make" sometimes invokes "configure" without -C, which is
>> > reeeeally slow.
> That needs fixing, then.

OK, I did that as follows in bzr 106594.

Propagate configure flags to sub-configures.
* Makefile.in (cache_file, MAINTAINER_MODE_FLAG, CONFIGUREFLAGS):
New macros.
(config.status, bootstrap): Use them to propagate configure flags
to sub-configures.
* configure.in (cache_file): AC_SUBST this, for Makefile.in.
=== modified file 'Makefile.in'
--- Makefile.in	2011-11-22 01:56:49 +0000
+++ Makefile.in	2011-12-03 20:17:56 +0000
@@ -66,6 +66,11 @@

 # ==================== Things `configure' Might Edit ====================

+MAINTAINER_MODE_FLAG = --disable-maintainer-mode
+@MAINT@MAINTAINER_MODE_FLAG = --enable-maintainer-mode
+cache_file = @cache_file@
+CONFIGURE_FLAGS = --cache-file=$(cache_file) $(MAINTAINER_MODE_FLAG)
+
 CC=@CC@
 CFLAGS=@CFLAGS@
 LDFLAGS=@LDFLAGS@
@@ -404,7 +409,7 @@
 	if [ -x ./config.status ]; then	\
 	    ./config.status --recheck;	\
 	else				\
-	    ./configure;		\
+	    ./configure $(CONFIGURE_FLAGS); \
 	fi

 AUTOCONF_INPUTS = @MAINT@ $(srcdir)/configure.in $(srcdir)/aclocal.m4
@@ -936,7 +941,7 @@
 	if [ -x ./config.status ]; then           \
 	    ./config.status;                      \
 	else                                      \
-	    ./configure --enable-maintainer-mode; \
+	    ./configure $(CONFIGURE_FLAGS);       \
 	fi
 	$(MAKE) $(MFLAGS) info all


=== modified file 'configure.in'
--- configure.in	2011-11-30 16:23:05 +0000
+++ configure.in	2011-12-03 20:17:56 +0000
@@ -182,6 +182,9 @@
 dnl http://lists.gnu.org/archive/html/emacs-devel/2008-04/msg01844.html
 OPTION_DEFAULT_ON([makeinfo],[don't require makeinfo for building manuals])

+## Makefile.in needs the cache file name.
+AC_SUBST(cache_file)
+
 ## This is an option because I do not know if all info/man support
 ## compressed files, nor how to test if they do so.
 OPTION_DEFAULT_ON([compress-info],[don't compress the installed Info pages])

Re: Annoyingly cautious make rules

[Andreas Schwab]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 12/03/11 00:49, Andreas Schwab wrote:
>> It is vital that configure is up-to-date wrt. configure.in.
>
> It's vital sometimes, but usually not.

That is completely beside the point.  Fact is that nothing substantial
has changed in the way configure is updated.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Annoyingly cautious make rules

[Paul Eggert]

On 12/02/11 20:52, Stephen J. Turnbull wrote:

> Please, no.  "-C" is an heuristic, not an optimization.

Thanks for this and for your other comments.
I pushed the following much-more-conservative patch,
to try to briefly document this long thread.

* INSTALL.BZR: Mention configure -C, --disable-maintainer-mode.
=== modified file 'INSTALL.BZR'
--- INSTALL.BZR	2011-06-25 17:51:03 +0000
+++ INSTALL.BZR	2011-12-03 20:26:26 +0000
@@ -28,6 +28,10 @@

   $ ./configure

+If you want later builds to go faster, at the expense of sometimes
+doing the wrong thing if you update the build procedure, you can
+invoke "./configure -C --disable-maintainer-mode" instead.
+
 Some of the files that are included in the Emacs tarball, such as
 byte-compiled Lisp files, are not stored in Bazaar.  Therefore, to
 build from Bazaar you must run "make bootstrap" instead of just "make":

Re: Annoyingly cautious make rules

[Glenn Morris]

Paul Eggert wrote:

> It's vital sometimes, but usually not.  For example, if you look
> at the last 10 changes to configure.in (in bzrs 106255 through 106559),
> only one (bzr 106532) has any relevance to any of the platforms that we
> routinely develop Emacs on. 

That's a call you're qualified to make, but not everyone who builds
Emacs from bzr is, or will expect to have to. So, again, my opinion is
that if you find these delays annoying, you should
disable-maintainer-mode for your own use.

I agree that the term "maintainer-mode" is now confusing, and that it
would be good to either remove the option altogether, or rename
--disable-maintainer-mode to --enable-loose-dependency-checking or
somesuch. But such a change doesn't seem suitable for 24.1.

Re: Annoyingly cautious make rules

[Richard Stallman]

    I agree that the term "maintainer-mode" is now confusing, and that it
    would be good to either remove the option altogether, or rename
    --disable-maintainer-mode to --enable-loose-dependency-checking or
    somesuch. But such a change doesn't seem suitable for 24.1.

The existing option name is not merely misleading, it is backwards.
I think that qualifies as a bug.


-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Annoyingly cautious make rules

[Stefan Monnier]

>     I agree that the term "maintainer-mode" is now confusing, and that it
>     would be good to either remove the option altogether, or rename
>     --disable-maintainer-mode to --enable-loose-dependency-checking or
>     somesuch. But such a change doesn't seem suitable for 24.1.

> The existing option name is not merely misleading, it is backwards.
> I think that qualifies as a bug.

In defense of whoever came up with it, it's really designed in the
context of "user downloads a tarball and compiles it" vs "maintainer
keeps a tree for long periods of time".


        Stefan

Re: Annoyingly cautious make rules

[Paul Eggert]

On 12/04/2011 08:55 AM, Stefan Monnier wrote:
> In defense of whoever came up with it, it's really designed in the
> context of "user downloads a tarball and compiles it" vs "maintainer
> keeps a tree for long periods of time".

Unfortunately it isn't being used that way in the latest
pretest tarball, and that's another symptom of the problem.

admin/make-tarball.txt step 5 says that when making a
pretest or release tarball, one should edit configure.in
so that maintainer-mode is off by default.  But this step
was not taken when generating the 24.0.92 tarball, so this
tarball builds in maintainer mode by default.

The whole thing is kind of a mess (and this brings up a
related issue: the process of building a release tarball
should be more automated and less error-prone), but at the
very least the actual practice in cutting tarballs should
match what's in admin/make-tarball.txt.

Re: Annoyingly cautious make rules

[Glenn Morris]

Paul Eggert wrote:

> admin/make-tarball.txt step 5 says that when making a pretest or
> release tarball, one should edit configure.in so that maintainer-mode
> is off by default.

It means, or was meant, to say, _maybe_ one should do that. The issue
had not been decided at the time that was written, so that note was
really intended as a reminder to think about the issue before the next
release. Based on the cited discussion from March, personally I think it
is more correct to leave it on even in release tarfiles, so that note
could just be deleted now.