bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Eli Zaretskii]

Building the pretest recompiles many Lisp files, which it shouldn't IMO.

Looking at the Makefile's, I think the problem is in this snippet from
src/Makefile.in (near the end):

  /* Since the .el.elc rule cannot specify an extra dependency, we do it here.  */
  ${lisp} ${SOME_MACHINE_LISP}: $(BOOTSTRAPEMACS)

  ${lispsource}loaddefs.el: $(BOOTSTRAPEMACS)
	  cd ../lisp; $(MAKE) $(MFLAGS) autoloads EMACS=../src/bootstrap-emacs${EXEEXT}

  /* Dump an Emacs executable named bootstrap-emacs containing the
     files from loadup.el in source form.  */

  bootstrap-emacs${EXEEXT}: temacs${EXEEXT}
	  cd ../lisp; $(MAKE) $(MFLAGS) update-subdirs
  #ifdef CANNOT_DUMP
	  ln -f temacs${EXEEXT} bootstrap-emacs${EXEEXT}
  #else
	  $(RUN_TEMACS) --batch --load loadup bootstrap
	  mv -f emacs${EXEEXT} bootstrap-emacs${EXEEXT}
  #endif /* ! defined (CANNOT_DUMP) */
	  @: Compile some files earlier to speed up further compilation.
	  cd ../lisp; $(MAKE) $(MFLAGS) compile-first EMACS=../src/bootstrap-emacs${EXEEXT}

It goes like this: since temacs is built, the last rule says to
produce bootstrap-emacs${EXEEXT}.  The new bootstrap-emacs then
triggers the 2 rules before it, which rebuild loaddefs.el and
recompile the Lisp files in ${lisp} and ${SOME_MACHINE_LISP}.


In GNU Emacs 23.0.90.1 (x86_64-unknown-linux-gnu)
 of 2009-02-02 on fencepost
configured using `configure  '--without-x''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: nil
  value of $XMODIFIERS: nil
  locale-coding-system: nil
  default-enable-multibyte-characters: t

Major mode: Fundamental

Minor modes in effect:
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  global-auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
ESC O D ESC O D ESC O D DEL ESC O B ESC O D a t SPC 
s t a r t u p SPC w h e n SPC y o u r SPC ~ / . e m 
a c s SPC s e t s SPC t h a t SPC v a r i a b ESC DEL 
o p t i o n s DEL . ESC b ESC b ESC b ESC d c u s t 
o m i z e s ESC b ESC b ESC b ESC b ESC b ESC f ESC 
d SPC i f ESC q ESC O B ESC O B ESC O B ESC O A ESC 
O B ESC O B DEL ESC O A ESC O A ESC O B ESC O B ESC 
O A ESC M-q M-k ESC O A ESC O B ESC O B C-x M-m C-x 
o C-x o ESC x f i n d - f i l e - l i t TAB RET m a 
i l . n e w RET C-s c y d ESC [ 6 ~ ESC [ 6 ~ ESC O 
A ESC O B ESC O B ESC O C ESC O C ESC O C ESC O C ESC 
O C ESC O C ESC O C ESC O D C-@ C-e ESC w ESC [ 6 ~ 
C-x b RET C-x b RET C-x b RET C-c C-c y e s RET C-x 
4 f e m a c s . c v TAB e m a TAB ESC DEL p r e TAB 
e m a TAB 3 TAB / t y TAB RET C-x o ESC x ESC O A ESC 
O A RET

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Mark saved where search started
Quit
Making completion list... [3 times]
Mark saved where search started
Auto-saving...done
Auto-saving...done
Mark saved where search started
Mark set
Sending...done

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Stefan Monnier]

> Building the pretest recompiles many Lisp files, which it shouldn't IMO.
> Looking at the Makefile's, I think the problem is in this snippet from
> src/Makefile.in (near the end):

>   /* Since the .el.elc rule cannot specify an extra dependency, we do it here.  */
>   ${lisp} ${SOME_MACHINE_LISP}: $(BOOTSTRAPEMACS)

>   ${lispsource}loaddefs.el: $(BOOTSTRAPEMACS)
> 	  cd ../lisp; $(MAKE) $(MFLAGS) autoloads EMACS=../src/bootstrap-emacs${EXEEXT}

>   /* Dump an Emacs executable named bootstrap-emacs containing the
>      files from loadup.el in source form.  */

>   bootstrap-emacs${EXEEXT}: temacs${EXEEXT}
> 	  cd ../lisp; $(MAKE) $(MFLAGS) update-subdirs
>   #ifdef CANNOT_DUMP
> 	  ln -f temacs${EXEEXT} bootstrap-emacs${EXEEXT}
>   #else
> 	  $(RUN_TEMACS) --batch --load loadup bootstrap
> 	  mv -f emacs${EXEEXT} bootstrap-emacs${EXEEXT}
>   #endif /* ! defined (CANNOT_DUMP) */
> 	  @: Compile some files earlier to speed up further compilation.
> 	  cd ../lisp; $(MAKE) $(MFLAGS) compile-first EMACS=../src/bootstrap-emacs${EXEEXT}

> It goes like this: since temacs is built, the last rule says to
> produce bootstrap-emacs${EXEEXT}.  The new bootstrap-emacs then
> triggers the 2 rules before it, which rebuild loaddefs.el and
> recompile the Lisp files in ${lisp} and ${SOME_MACHINE_LISP}.

I think we have to live with this for now.
Getting make to understand the nature of the dependencies here is pretty
tricky, so you can get it to work right for the tarball or you can get
it to work right for the "cvs update" case, but it's pretty painful
to get it to work right in both cases.

I'd be tempted to remove the .elc files that get rebuilt anyway.
This will bring the two cases closer to each other, which can only help
us waste less time on this.


        Stefan


PS: Part of the problem, as far as I understand it, is that we need
the .elc files to depend on bootstrap-emacs so as to tell `make' to
build bootstrap-emacs when needed, but we don't actually need the .elc
files to be newer than bootstrap-emacs (in some rare cases a newer
bootstrap-emacs will compile the .el differently, but in general it's
not the case).  But `make' doesn't have such a concept of a dependency
that "needs to exist, but doesn't need to be older".

PPS: I guess we could get it to work better if the toplevel Makefile.in
were changed so as to call "cd src; make bootstrap-emacs" first and then
"cd src; make" as a separate step.  E.g. along the lines of the barely
tested patch below.


=== modified file 'Makefile.in'
--- Makefile.in	2009-01-09 16:56:04 +0000
+++ Makefile.in	2009-02-02 16:01:10 +0000
@@ -320,6 +320,12 @@
 # We need to build `emacs' in `src' to compile the *.elc files in `lisp'.
 lisp: src
 
+# These targets should be "${SUBDIR} without `src'".
+lib-src lisp: Makefile FRC
+	cd $@; $(MAKE) all $(MFLAGS)                           \
+	  CC='${CC}' CFLAGS='${CFLAGS}' CPPFLAGS='${CPPFLAGS}' \
+	  LDFLAGS='${LDFLAGS}' MAKE='${MAKE}'
+
 # Pass to src/Makefile.in an additional BOOTSTRAPEMACS variable which
 # is either set to bootstrap-emacs (in case bootstrap-emacs has not been
 # constructed yet) or the empty string (otherwise).
@@ -329,12 +335,16 @@
 # file src/foo.c forces dumping a new bootstrap-emacs, then re-byte-compiling
 # all preloaded elisp files, and only then dump the actual src/emacs, which
 # is not wrong, but is overkill in 99.99% of the cases.
-${SUBDIR}: Makefile FRC
+src: Makefile FRC
 	boot=bootstrap-emacs$(EXEEXT);                         \
-	if [ -x "src/$$boot" ]; then boot=""; fi;                   \
+	if [ ! -x "src/$$boot" ]; then                                     \
+	    cd $@; $(MAKE) all $(MFLAGS)                                   \
+	      CC='${CC}' CFLAGS='${CFLAGS}' CPPFLAGS='${CPPFLAGS}'         \
+	      LDFLAGS='${LDFLAGS}' MAKE='${MAKE}' BOOTSTRAPEMACS="$$boot"; \
+	fi;                                                                \
 	cd $@; $(MAKE) all $(MFLAGS)                           \
 	  CC='${CC}' CFLAGS='${CFLAGS}' CPPFLAGS='${CPPFLAGS}' \
-	  LDFLAGS='${LDFLAGS}' MAKE='${MAKE}' BOOTSTRAPEMACS="$$boot"
+	  LDFLAGS='${LDFLAGS}' MAKE='${MAKE}' BOOTSTRAPEMACS=""
 
 blessmail: Makefile src FRC
 	cd lib-src; $(MAKE) maybe-blessmail $(MFLAGS) \

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: 2151@emacsbugs.donarmstrong.com,  emacs-pretest-bug@gnu.org
> Date: Mon, 02 Feb 2009 11:02:49 -0500
> 
> > It goes like this: since temacs is built, the last rule says to
> > produce bootstrap-emacs${EXEEXT}.  The new bootstrap-emacs then
> > triggers the 2 rules before it, which rebuild loaddefs.el and
> > recompile the Lisp files in ${lisp} and ${SOME_MACHINE_LISP}.
> 
> I think we have to live with this for now.

I feared you'd say that.  All I can say is that I think it's
fundamentally wrong to have Lisp files compile as part of the build
(Yes, I know we compile Leim files, presumably to conserve space in
the tarball, but I think that's wrong, too.)  The result is that a
successful build becomes less predictable, and we can no longer depend
on having the same good .elc files on all platforms.

(It is also a major headache for the DOS port, since lisp/Makefile
needs a Unixy shell, and I always avoided requiring that for building
an official release.)

> Getting make to understand the nature of the dependencies here is pretty
> tricky, so you can get it to work right for the tarball or you can get
> it to work right for the "cvs update" case, but it's pretty painful
> to get it to work right in both cases.

I think it shouldn't be too hard, and the ideas you suggested further
in your mail are my evidence.

> PS: Part of the problem, as far as I understand it, is that we need
> the .elc files to depend on bootstrap-emacs so as to tell `make' to
> build bootstrap-emacs when needed, but we don't actually need the .elc
> files to be newer than bootstrap-emacs (in some rare cases a newer
> bootstrap-emacs will compile the .el differently, but in general it's
> not the case).  But `make' doesn't have such a concept of a dependency
> that "needs to exist, but doesn't need to be older".

I don't think we need a bootstrap-emacs in a released version at all.
We could add some file to the tarball, generated at make-dist time, to
signal that bootstrap-emacs is not needed.  That file could actually
be named `bootstrap-emacs', which should resolve the problem nicely
(assuming we manage to have it older than the oldest .elc file).

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Stefan Monnier]

> I feared you'd say that.  All I can say is that I think it's
> fundamentally wrong to have Lisp files compile as part of the build
> (Yes, I know we compile Leim files, presumably to conserve space in
> the tarball, but I think that's wrong, too.)  The result is that a
> successful build becomes less predictable, and we can no longer depend
> on having the same good .elc files on all platforms.

OTOH, I feel like it's fundamentally wrong to provide pre-built files in
our tarball.  It's also fundamentally wrong to have such very different
build-"rules" between the "checkout from CVS" and "untarred" cases.
It makes maintenance more difficult.

> (It is also a major headache for the DOS port, since lisp/Makefile
> needs a Unixy shell, and I always avoided requiring that for building
> an official release.)

Good point.  So removing the .elc files would require more work.

>> Getting make to understand the nature of the dependencies here is pretty
>> tricky, so you can get it to work right for the tarball or you can get
>> it to work right for the "cvs update" case, but it's pretty painful
>> to get it to work right in both cases.

> I think it shouldn't be too hard, and the ideas you suggested further
> in your mail are my evidence.

Have you tried it?  It seems to work OK for the "checkout from CVS"
case, so maybe it's a good solution.

> I don't think we need a bootstrap-emacs in a released version at all.
> We could add some file to the tarball, generated at make-dist time, to
> signal that bootstrap-emacs is not needed.  That file could actually
> be named `bootstrap-emacs', which should resolve the problem nicely
> (assuming we manage to have it older than the oldest .elc file).

Maybe we can get that to work, but it sounds terribly hackish.
Also, I'd like to make sure that if some wants to change some .el files
and then recompile, it still works correctly.


        Stefan

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: 2151@emacsbugs.donarmstrong.com
> Date: Tue, 03 Feb 2009 16:24:54 -0500
> 
> >> Getting make to understand the nature of the dependencies here is pretty
> >> tricky, so you can get it to work right for the tarball or you can get
> >> it to work right for the "cvs update" case, but it's pretty painful
> >> to get it to work right in both cases.
> 
> > I think it shouldn't be too hard, and the ideas you suggested further
> > in your mail are my evidence.
> 
> Have you tried it?  It seems to work OK for the "checkout from CVS"
> case, so maybe it's a good solution.

Which one? you suggested more than one, I think.

> > I don't think we need a bootstrap-emacs in a released version at all.
> > We could add some file to the tarball, generated at make-dist time, to
> > signal that bootstrap-emacs is not needed.  That file could actually
> > be named `bootstrap-emacs', which should resolve the problem nicely
> > (assuming we manage to have it older than the oldest .elc file).
> 
> Maybe we can get that to work, but it sounds terribly hackish.

Why hackish? many distributions have or create such time-stamp files,
for Make's sake.

> Also, I'd like to make sure that if some wants to change some .el files
> and then recompile, it still works correctly.

It will, I think, because we can copy emacs to bootstrap-emacs at the
end of the build.

I will give this a try when I have time.

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Stefan Monnier]

>> Have you tried it?  It seems to work OK for the "checkout from CVS"
>> case, so maybe it's a good solution.
> Which one? you suggested more than one, I think.

There was only one patch in my message.

>> > I don't think we need a bootstrap-emacs in a released version at all.
>> > We could add some file to the tarball, generated at make-dist time, to
>> > signal that bootstrap-emacs is not needed.  That file could actually
>> > be named `bootstrap-emacs', which should resolve the problem nicely
>> > (assuming we manage to have it older than the oldest .elc file).
>> Maybe we can get that to work, but it sounds terribly hackish.
> Why hackish? many distributions have or create such time-stamp files,
> for Make's sake.

I guess it depends on the details.

>> Also, I'd like to make sure that if some wants to change some .el files
>> and then recompile, it still works correctly.
> It will, I think, because we can copy emacs to bootstrap-emacs at the
> end of the build.

What if they edit .el files before building a first `emacs'?


        Stefan

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: 2151@emacsbugs.donarmstrong.com
> Date: Wed, 04 Feb 2009 14:51:49 -0500
> 
> >> > We could add some file to the tarball, generated at make-dist time, to
> >> > signal that bootstrap-emacs is not needed.  That file could actually
> >> > be named `bootstrap-emacs', which should resolve the problem nicely
> >> > (assuming we manage to have it older than the oldest .elc file).
> >> Maybe we can get that to work, but it sounds terribly hackish.
> > Why hackish? many distributions have or create such time-stamp files,
> > for Make's sake.
> 
> I guess it depends on the details.

After thinking some more about this, I don't see any easy and safe
solution besides adding a "bootstrap-emacs" file to the tarball.
Maybe in future releases we will be able to find a better way, but for
now this is the only one that's safe.

I will do that if you agree.

In any case, it sounds wrong to me to have the distribution biased in
favor of people who build Emacs out of CVS.  Those who do that
generally know more about the build procedure than people who build
the release tarball.  We should favor the latter, not the former.

> >> Also, I'd like to make sure that if some wants to change some .el files
> >> and then recompile, it still works correctly.
> > It will, I think, because we can copy emacs to bootstrap-emacs at the
> > end of the build.
> 
> What if they edit .el files before building a first `emacs'?

We could arrange for byte-recompile-directory at the end of the build,
if that is a real-life use-case.  Note that in previous releases, the
user needed to manually compile the modified .el files and re-dump
Emacs in this case, so we are not introducing any new problems, even
if we do nothing about it.

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Stefan Monnier]

>> >> > We could add some file to the tarball, generated at make-dist time, to
>> >> > signal that bootstrap-emacs is not needed.  That file could actually
>> >> > be named `bootstrap-emacs', which should resolve the problem nicely
>> >> > (assuming we manage to have it older than the oldest .elc file).
>> >> Maybe we can get that to work, but it sounds terribly hackish.
>> > Why hackish? many distributions have or create such time-stamp files,
>> > for Make's sake.
>> 
>> I guess it depends on the details.

> After thinking some more about this, I don't see any easy and safe
> solution besides adding a "bootstrap-emacs" file to the tarball.
> Maybe in future releases we will be able to find a better way, but for
> now this is the only one that's safe.

> I will do that if you agree.

I'd rather not if the patch I suggested works.  Does it?

> In any case, it sounds wrong to me to have the distribution biased in
> favor of people who build Emacs out of CVS.

My guess is that many more people build from CVS than from the release
tarball.  Most users of the release will use a precompiled package.
So I care much more about building from CVS than building from
a tarball.  Also I care more about the build being reliable and
maintainable, than about it being optimal in all circumstances, so if
building from the tarball does a bit of extra work, I don't really mind
as long as it makes maintenance easier.
I do think the patch I proposed fixes the problem and doesn't make the
code significantly more hairy, so I'm willing to install it.  I've been
using it ever since I sent it and it seems to work fine (tho
I basically never boostrap anyway), but I'd like confirmation that it
fixes your problem before installing it.

>> >> Also, I'd like to make sure that if some wants to change some .el files
>> >> and then recompile, it still works correctly.
>> > It will, I think, because we can copy emacs to bootstrap-emacs at the
>> > end of the build.
>> What if they edit .el files before building a first `emacs'?
> We could arrange for byte-recompile-directory at the end of the build,
> if that is a real-life use-case.

That wouldn't help: the problem is that such a change would trigger
recompilation of the .elc file, which in turn will run your dummy
bootstrap-emacs, at which point the build will presumably fail.


        Stefan

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: 2151@emacsbugs.donarmstrong.com
> Date: Sat, 21 Feb 2009 15:54:01 -0500
> 
> > After thinking some more about this, I don't see any easy and safe
> > solution besides adding a "bootstrap-emacs" file to the tarball.
> > Maybe in future releases we will be able to find a better way, but for
> > now this is the only one that's safe.
> 
> > I will do that if you agree.
> 
> I'd rather not if the patch I suggested works.  Does it?

Maybe it does, but it isn't meant to solve the problem at hand, as it
builds bootstrap-emacs unconditionally.

> > In any case, it sounds wrong to me to have the distribution biased in
> > favor of people who build Emacs out of CVS.
> 
> My guess is that many more people build from CVS than from the release
> tarball.  Most users of the release will use a precompiled package.

Then perhaps we should stop producing releases.

> I do think the patch I proposed fixes the problem and doesn't make the
> code significantly more hairy, so I'm willing to install it.

Go ahead, and let's see what it does to the next pretest.

> >> >> Also, I'd like to make sure that if some wants to change some .el files
> >> >> and then recompile, it still works correctly.
> >> > It will, I think, because we can copy emacs to bootstrap-emacs at the
> >> > end of the build.
> >> What if they edit .el files before building a first `emacs'?
> > We could arrange for byte-recompile-directory at the end of the build,
> > if that is a real-life use-case.
> 
> That wouldn't help: the problem is that such a change would trigger
> recompilation of the .elc file, which in turn will run your dummy
> bootstrap-emacs, at which point the build will presumably fail.

You are assuming too much about what I meant.
byte-recompile-directory could use the just-built Emacs binary, and
the dummy bootstrap-emacs could be a shell script that just invoked
that same binary.

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Stefan Monnier]

>> > After thinking some more about this, I don't see any easy and safe
>> > solution besides adding a "bootstrap-emacs" file to the tarball.
>> > Maybe in future releases we will be able to find a better way, but for
>> > now this is the only one that's safe.
>> 
>> > I will do that if you agree.
>> 
>> I'd rather not if the patch I suggested works.  Does it?

> Maybe it does, but it isn't meant to solve the problem at hand, as it
> builds bootstrap-emacs unconditionally.

Not sure what you mean.  It solves the title problem "pretest recompiles
Lisp files".  AFAIK the only thing it does unnecessarily is to dump
Emacs twice.

>> My guess is that many more people build from CVS than from the release
>> tarball.  Most users of the release will use a precompiled package.
> Then perhaps we should stop producing releases.

I don't see the relationship.  The precompiled packages built from the
tarball are probably used by many more people than either of the tarball
or the CVS.

>> > We could arrange for byte-recompile-directory at the end of the build,
>> > if that is a real-life use-case.
>> That wouldn't help: the problem is that such a change would trigger
>> recompilation of the .elc file, which in turn will run your dummy
>> bootstrap-emacs, at which point the build will presumably fail.
> You are assuming too much about what I meant.
> byte-recompile-directory could use the just-built Emacs binary, and
> the dummy bootstrap-emacs could be a shell script that just invoked
> that same binary.

Without seeing the details, it's hard to know indeed.
But the scenario I imagine is someone changing lisp/startup.el, which
will trigger recompilation before Emacs is dumped.
But yes, maybe you can come up with a clever bootstrap-emacs script
which will cause a real build of a proper bootstrap-emacs.
Still, I think this whole discussion is making us waste more time than
users will spent waiting for the extra bit of compilation time due to
having to dump Emacs twice and/or rebuild the .el files.


        Stefan

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: 2151@emacsbugs.donarmstrong.com
> Date: Sat, 21 Feb 2009 17:49:16 -0500
> 
> >> I'd rather not if the patch I suggested works.  Does it?
> 
> > Maybe it does, but it isn't meant to solve the problem at hand, as it
> > builds bootstrap-emacs unconditionally.
> 
> Not sure what you mean.  It solves the title problem "pretest recompiles
> Lisp files".  AFAIK the only thing it does unnecessarily is to dump
> Emacs twice.

Yes.  And the Right Solution (IMO) is to invoke src/Makefile with
`boot' argument empty, which would only dump once and not recompile
anything.

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Eli Zaretskii]

> Date: Sun, 22 Feb 2009 06:07:58 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 2151@emacsbugs.donarmstrong.com
> 
> > From: Stefan Monnier <monnier@iro.umontreal.ca>
> > Cc: 2151@emacsbugs.donarmstrong.com
> > Date: Sat, 21 Feb 2009 17:49:16 -0500
> > 
> > >> I'd rather not if the patch I suggested works.  Does it?
> > 
> > > Maybe it does, but it isn't meant to solve the problem at hand, as it
> > > builds bootstrap-emacs unconditionally.
> > 
> > Not sure what you mean.  It solves the title problem "pretest recompiles
> > Lisp files".  AFAIK the only thing it does unnecessarily is to dump
> > Emacs twice.
> 
> Yes.  And the Right Solution (IMO) is to invoke src/Makefile with
> `boot' argument empty, which would only dump once and not recompile
> anything.

A new idea: can we avoid the `boot=bootstrap-emacs$(EXEEXT)' thing in
top-level Makefile.in by testing for some file that is only present
when Emacs is built out of CVS?  For example, the `admin' directory is
not in the pretest/release tarballs; can we test for its existence and
invoke sub-Make's with `boot=""' if `admin' is not there?

There's one other reason for compiling Lisp files during the build, it
is this fragment from the top-level Makefile.in:

    # Subdirectories to make recursively.  `lisp' is not included
    # because the compiled lisp files are part of the distribution.
    # leim is not included because it needs special handling.
    # 
    # Actually, we now include `lisp' as well, since the compiled files
    # are not included any more in case of bootstrap or in case Emacs was
    # checked out from a VCS.
    SUBDIR = lib-src src lisp

The second part gives the rationale for adding `lisp' to the list, but
that rationale is not valid for when Emacs is built from a pretest or
release tarball.  Is it okay to avoid recursing into `lisp' in that
case, again by testing the existence of `admin' or some such?

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Eli Zaretskii]

> Date: Fri, 27 Feb 2009 17:02:38 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 
> 
> There's one other reason for compiling Lisp files during the build, it
> is this fragment from the top-level Makefile.in:
> 
>     # Subdirectories to make recursively.  `lisp' is not included
>     # because the compiled lisp files are part of the distribution.
>     # leim is not included because it needs special handling.
>     # 
>     # Actually, we now include `lisp' as well, since the compiled files
>     # are not included any more in case of bootstrap or in case Emacs was
>     # checked out from a VCS.
>     SUBDIR = lib-src src lisp

The reason that recursing into `lisp' causes more recompilation is
that `make-dist' does this:

  echo "Recompiling Lisp files"
  $EMACS -batch -f batch-byte-recompile-directory lisp leim

while the top-level Makefile.in does this:

  ${SUBDIR}: Makefile FRC
	  boot=bootstrap-emacs$(EXEEXT);                         \
	  if [ -x "src/$$boot" ]; then boot=""; fi;                   \
	  cd $@; $(MAKE) all $(MFLAGS)                           \
	    CC='${CC}' CFLAGS='${CFLAGS}' CPPFLAGS='${CPPFLAGS}' \
	    LDFLAGS='${LDFLAGS}' MAKE='${MAKE}' BOOTSTRAPEMACS="$$boot"

IOW, Makefile.in runs "make all" in `lisp'.  Recompilation happens
because "make all" observes dependencies between some *.elc files,
while batch-byte-recompile-directory does not.  So among the *.elc
files generated by make-dist it could well happen that a .elc file
that is prerequisite for some other .elc file ends up being newer,
which will cause "make all" to want to recompile that particular
dependency.

I think we should replace batch-byte-recompile-directory in make-dist
with a call to "make -C lisp all".  Any objections?

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Glenn Morris]

Eli Zaretskii wrote:

> I think we should replace batch-byte-recompile-directory in make-dist
> with a call to "make -C lisp all".  Any objections?


I'd be surprised if this makes a difference. I thought the pretests
were built using the instructions in admin/make-tarball.txt. This says
to bootstrap, then run `make-dist --snapshot'. The latter doesn't do
any building AFAIK. It does not run batch-byte-recompile-directory.

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: 2151@emacsbugs.donarmstrong.com
> Date: Fri, 27 Feb 2009 18:21:42 -0500
> 
> Eli Zaretskii wrote:
> 
> > I think we should replace batch-byte-recompile-directory in make-dist
> > with a call to "make -C lisp all".  Any objections?
> 
> 
> I'd be surprised if this makes a difference. I thought the pretests
> were built using the instructions in admin/make-tarball.txt. This says
> to bootstrap, then run `make-dist --snapshot'. The latter doesn't do
> any building AFAIK. It does not run batch-byte-recompile-directory.

That's true, but then how can we explain the contradiction between
these 2 facts:

  . bootstrap runs "cd lisp; make all", which should compile
    progmodes/cc-*.el and mh-e/*.el according to the dependencies at
    the end of lisp/Makefile.in

  . in the 23.0.91 pretest tarball, the *.elc files have time stamps
    in the alphabetical order, which hints that they were built by
    batch-byte-recompile-directory; in particular, mh-e/*.elc files
    are NOT in the order dictated by their dependencies

What am I missing?

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Stefan Monnier]

> A new idea: can we avoid the `boot=bootstrap-emacs$(EXEEXT)' thing in
> top-level Makefile.in by testing for some file that is only present
> when Emacs is built out of CVS?  For example, the `admin' directory is
> not in the pretest/release tarballs; can we test for its existence and
> invoke sub-Make's with `boot=""' if `admin' is not there?

I am not interested in adding hacks specifically to speed up building
from the tarball.  I just installed the patch that prevents recompiling
all the .elc files just because the src/bootstrap-emacs isn't present.
This should bring down the unnecessary work to just an extra redundant
dump, while still being "correct" in the sense that it should correctly
react to most kinds of file modifications.  Also it may occasionally be
useful for people using the CVS code.


        Stefan

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: 2151@emacsbugs.donarmstrong.com
> Date: Fri, 27 Feb 2009 23:48:51 -0500
> 
> I am not interested in adding hacks specifically to speed up building
> from the tarball.

Thanks a lot for your kind words about my repeated attempts to find an
agreeable solution of this problem.

> I just installed the patch that prevents recompiling
> all the .elc files just because the src/bootstrap-emacs isn't present.

Well, it doesn't seem to work.  Here's what I did to test it (on a
GNU/Linux machine):

  . unpack emacs-23.0.91 pretest tarball
  . replace the top-level Makefile.in with the one from today's CVS
  . verify by `diff' that Makefile.in indeed has your changes
  . ./configure --without-x && make

Excerpts from the resulting build session are below (the omitted parts
are marked with "[...]"); they still show that many Lisp files are
recompiled, after building bootstrap-emacs.

Did I perhaps need to copy more files from CVS?

Also note this error message:

  /bin/sh: line 6: cd: src: No such file or directory

It probably means that something else is wrong with your changes.

-----------------------------------------------------------
Script started on Sat Feb 28 05:03:12 2009
manpath: warning: $MANPATH set, ignoring /etc/manpath.config
eliz@fencepost:~/emacs.cvs/pretest/emacs-23.0.91$./configure --without-x
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu

[...]

Configured for `x86_64-unknown-linux-gnu'.

  Where should the build process find the source code?    /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91
  What operating system and machine description files should Emacs use?
        `s/gnu-linux.h' and `m/amdx86-64.h'
  What compiler should emacs be built with?               gcc -g -O2 -Wno-pointer-sign 
  Should Emacs use the GNU version of malloc?             yes
      (Using Doug Lea's new malloc from the GNU C Library.)
  Should Emacs use a relocating allocator for buffers?    yes
  Should Emacs use mmap(2) for buffer allocation?         no
  What window system should Emacs use?                    none
  What toolkit should Emacs use?                          none
  Where do we find X Windows header files?                NONE
  Where do we find X Windows libraries?                   NONE
  Does Emacs use -lXaw3d?                                 no
  Does Emacs use -lXpm?                                   no
  Does Emacs use -ljpeg?                                  no
  Does Emacs use -ltiff?                                  no
  Does Emacs use a gif library?                           no 
  Does Emacs use -lpng?                                   no
  Does Emacs use -lrsvg-2?                                no
  Does Emacs use -lgpm?                                   yes
  Does Emacs use -ldbus?                                  no
  Does Emacs use -lfreetype?                              no
  Does Emacs use -lm17n-flt?                              no
  Does Emacs use -lotf?                                   no
  Does Emacs use -lxft?                                   no
  Does Emacs use toolkit scroll bars?                     no


configure: creating ./config.status
config.status: creating Makefile
config.status: creating lib-src/Makefile.c
config.status: creating oldXMenu/Makefile
config.status: creating doc/emacs/Makefile
config.status: creating doc/misc/Makefile
config.status: creating doc/lispintro/Makefile
config.status: creating doc/lispref/Makefile
config.status: creating src/Makefile.c
config.status: creating lwlib/Makefile
config.status: creating lisp/Makefile
config.status: creating leim/Makefile
config.status: creating src/config.h
config.status: executing default commands
creating src/epaths.h
creating lib-src/Makefile
creating src/Makefile
eliz@fencepost:~/emacs.cvs/pretest/emacs-23.0.91$ make
cd lib-src; make all                            \
	  CC='gcc' CFLAGS='-g -O2 -Wno-pointer-sign ' CPPFLAGS='-D_BSD_SOURCE  ' \
	  LDFLAGS='-Wl,-znocombreloc' MAKE='make'
make[1]: Entering directory `/srv/data/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lib-src'

[...]

gcc -D_BSD_SOURCE -DHAVE_CONFIG_H -I. -I../src -I/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lib-src -I/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lib-src/../src -Wl,-znocombreloc -D_BSD_SOURCE   -g -O2 -Wno-pointer-sign  -DVERSION="\"23.0.91\"" /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lib-src/ebrowse.c   -o ebrowse
make[1]: Leaving directory `/srv/data/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lib-src'
boot=bootstrap-emacs;                         \
	if [ ! -x "src/$boot" ]; then                                     \
	    cd src; make all                                    \
	      CC='gcc' CFLAGS='-g -O2 -Wno-pointer-sign ' CPPFLAGS='-D_BSD_SOURCE  '         \
	      LDFLAGS='-Wl,-znocombreloc' MAKE='make' BOOTSTRAPEMACS="$boot"; \
	fi;                                                                \
	cd src; make all                            \
	  CC='gcc' CFLAGS='-g -O2 -Wno-pointer-sign ' CPPFLAGS='-D_BSD_SOURCE  ' \
	  LDFLAGS='-Wl,-znocombreloc' MAKE='make' BOOTSTRAPEMACS=""
make[1]: Entering directory `/srv/data/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/src'
gcc -c -D_BSD_SOURCE   -Demacs -DHAVE_CONFIG_H  -I. -I/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/src -D_BSD_SOURCE    -g -O2 -Wno-pointer-sign  pre-crt0.c
touch stamp-oldxmenu
gcc -c -D_BSD_SOURCE   -Demacs -DHAVE_CONFIG_H  -I. -I/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/src -D_BSD_SOURCE    -g -O2 -Wno-pointer-sign  dispnew.c

[...]

gcc -nostdlib `./prefix-args -Xlinker  -z nocombreloc` -Wl,-znocombreloc -o temacs pre-crt0.o /usr/lib64/crt1.o /usr/lib64/crti.o dispnew.o frame.o scroll.o xdisp.o menu.o  window.o charset.o coding.o category.o ccl.o character.o chartab.o cm.o term.o terminal.o xfaces.o    emacs.o keyboard.o macros.o keymap.o sysdep.o buffer.o filelock.o insdel.o marker.o minibuf.o fileio.o dired.o filemode.o cmds.o casetab.o casefiddle.o indent.o search.o regex.o undo.o alloc.o data.o doc.o editfns.o callint.o eval.o floatfns.o fns.o font.o print.o lread.o syntax.o unexelf.o bytecode.o process.o callproc.o region-cache.o sound.o atimer.o doprnt.o strftime.o intervals.o textprop.o composite.o md5.o     terminfo.o lastfile.o   vm-limit.o         -lgpm -lncurses   -lm -lgcc -lc -lgcc /usr/lib64/crtn.o 
cd ../lisp; make -w update-subdirs
make[2]: Entering directory `/srv/data/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp'
wd=/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp; subdirs=`(cd $wd; find . -type d -print)`; for file in $subdirs; do case $file in */Old | */RCS | */CVS | */CVS/* | */.* | */.*/* | */=* ) ;; *) wins="$wins $wd/$file" ;; esac; done; \
	for file in $wins; do \
	   /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/../update-subdirs $file; \
	done;
make[2]: Leaving directory `/srv/data/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp'
`/bin/pwd`/temacs --batch --load loadup bootstrap
Loading loadup.el (source)...
Using load-path (/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/emacs-lisp /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/language /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/international /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/textmodes)
Loading emacs-lisp/byte-run...
Loading emacs-lisp/backquote...
Loading subr...
Loading version.el (source)...

[...]

Finding pointers to doc strings...
Finding pointers to doc strings...done
Dumping under the name emacs
101634 pure bytes used
mv -f emacs bootstrap-emacs
cd ../lisp; make -w compile-first EMACS=../src/bootstrap-emacs
make[2]: Entering directory `/srv/data/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp'
make[2]: Nothing to be done for `compile-first'.
make[2]: Leaving directory `/srv/data/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp'
cd ../lisp; make -w autoloads EMACS=../src/bootstrap-emacs
make[2]: Entering directory `/srv/data/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp'
chmod +w /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/ps-print.el /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/emulation/tpu-edt.el \
	  /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/emacs-lisp/cl-loaddefs.el
wd=/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp; subdirs=`(cd $wd; find . -type d -print)`; for file in $subdirs; do case $file in */Old | */RCS | */CVS | */CVS/* | */.* | */.*/* | */=* | */obsolete | */term ) ;; *) wins="$wins $wd/$file" ;; esac; done; \
	echo Directories: $wins; \
	EMACSLOADPATH=/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp LC_ALL=C ../src/bootstrap-emacs -batch --no-site-file --multibyte -l autoload --eval '(setq generated-autoload-file "/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/loaddefs.el")' -f batch-update-autoloads $wins
Directories: /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/. /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/./url /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/./textmodes /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/./progmodes /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/./play /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/./org /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/./nxml /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/./net /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/./mh-e /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/./mail /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/./language /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/./international /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/./eshell /home/e/eliz/emacs.cvs/pretes
 t/emacs-23.0.91/lisp/./emulation /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/./emacs-lisp /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/./calendar /home/e/eliz/emacs.cvs/pretest/e!
 macs-23.0
Saving file /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/loaddefs.el...
Wrote /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/loaddefs.el
make[2]: Leaving directory `/srv/data/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp'
make[2]: Entering directory `/srv/data/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp'
Compiling /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/src/../lisp/abbrev.el
Wrote /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/abbrev.elc
make[2]: Leaving directory `/srv/data/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp'
make[2]: Entering directory `/srv/data/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp'
Compiling /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/src/../lisp/buff-menu.el
Wrote /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/buff-menu.elc
make[2]: Leaving directory `/srv/data/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp'

[...]

make[2]: Entering directory `/srv/data/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp'
Compiling ../lisp/term/ns-win.el
Wrote /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/term/ns-win.elc
make[2]: Leaving directory `/srv/data/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp'
rm -f ../etc/DOC
../lib-src/make-docfile -d /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/src dosfns.o msdos.o xterm.o xfns.o xmenu.o xselect.o xrdb.o xsmfns.o fringe.o image.o fontset.o nsterm.o nsfns.o nsmenu.o nsselect.o nsimage.o nsfont.o w32.o w32console.o w32fns.o w32heap.o w32inevt.o w32menu.o w32proc.o w32reg.o w32select.o w32term.o w32xfns.o  dispnew.o frame.o scroll.o xdisp.o menu.o  window.o charset.o coding.o category.o ccl.o character.o chartab.o cm.o term.o terminal.o xfaces.o    emacs.o keyboard.o macros.o keymap.o sysdep.o buffer.o filelock.o insdel.o marker.o minibuf.o fileio.o dired.o filemode.o cmds.o casetab.o casefiddle.o indent.o search.o regex.o undo.o alloc.o data.o doc.o editfns.o callint.o eval.o floatfns.o fns.o font.o print.o lread.o syntax.o unexelf.o bytecode.o process.o callpr
 oc.o region-cache.o sound.o atimer.o doprnt.o strftime.o intervals.o textprop.o composite.o md5.o     > ../etc/DOC
../lib-src/make-docfile -a ../etc/DOC -d /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/src ../lisp/mouse.elc ../lisp/select.elc ../lisp/scroll-bar.elc ../lisp/ls-lisp.elc ../lisp/dos-fns.elc ../lisp/w32-fns.elc ../lisp/dos-w32.elc ../lisp/disp-table.elc ../lisp/dos-vars.elc ../lisp/tooltip.elc ../lisp/image.elc ../lisp/fringe.elc ../lisp/dnd.elc ../lisp/mwheel.elc ../lisp/tool-bar.elc ../lisp/x-dnd.elc ../lisp/international/ccl.elc ../lisp/international/fontset.elc ../lisp/mouse.elc ../lisp/term/x-win.elc ../lisp/term/pc-win.elc ../lisp/term/ns-win.elc ../lisp/loaddefs.el ../lisp/abbrev.elc ../lisp/buff-menu.elc ../lisp/button.elc ../lisp/emacs-lisp/byte-run.elc ../lisp/composite.elc ../lisp/cus-face.elc ../lisp/cus-start.elc ../lisp/custom.elc ../lisp/emacs-lisp/backquote.elc ../lisp/emacs
 -lisp/lisp-mode.elc ../lisp/emacs-lisp/lisp.elc ../lisp/facemenu.elc ../lisp/faces.elc ../lisp/files.elc ../lisp/emacs-lisp/float-sup.elc ../lisp/format.elc ../lisp/frame.elc ../lisp/help.e!
 lc ../lis
LC_ALL=C `/bin/pwd`/temacs -batch -l loadup dump
Loading loadup.el (source)...
Using load-path (/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp)
Loading emacs-lisp/byte-run...
Loading emacs-lisp/backquote...
Loading subr...
Loading version.el (source)...
Loading widget...
[...]
Finding pointers to doc strings...
Finding pointers to doc strings...done
Dumping under the name emacs
1877507 pure bytes used
Adding name emacs-23.0.91.1
ln -f emacs bootstrap-emacs
./emacs -q -batch -f list-load-path-shadows
make[1]: Leaving directory `/srv/data/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/src'
/bin/sh: line 6: cd: src: No such file or directory
make[1]: Entering directory `/srv/data/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/src'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/srv/data/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/src'
cd lisp; make all                            \
	  CC='gcc' CFLAGS='-g -O2 -Wno-pointer-sign ' CPPFLAGS='-D_BSD_SOURCE  ' \
	  LDFLAGS='-Wl,-znocombreloc' MAKE='make'
make[1]: Entering directory `/srv/data/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp'
Compiling /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/mh-e/mh-e.el
Wrote /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/mh-e/mh-e.elc
Compiling /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/mh-e/mh-alias.el
nmh 1.1 installed as MH variant
Wrote /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/mh-e/mh-alias.elc
Compiling /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/mh-e/mh-scan.el
nmh 1.1 installed as MH variant

[...]

Compiling /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/progmodes/cc-subword.el
Wrote /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp/progmodes/cc-subword.elc
make -w compile-last EMACS=../src/emacs
make[2]: Entering directory `/srv/data/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp'
make[2]: Leaving directory `/srv/data/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp'
make[1]: Leaving directory `/srv/data/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/lisp'
(export PARALLEL; PARALLEL=0; cd leim; make all  \
	  CC='gcc' CFLAGS='-g -O2 -Wno-pointer-sign ' CPPFLAGS='-D_BSD_SOURCE  ' \
	  LDFLAGS='-Wl,-znocombreloc' MAKE='make')
make[1]: Entering directory `/srv/data/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/leim'
EMACSLOADPATH=/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/leim/../lisp LC_ALL=C ../src/emacs -batch --no-init-file --no-site-file --multibyte -l /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/leim/../lisp/international/titdic-cnv \
	  -f batch-titdic-convert -dir quail /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/leim/CXTERM-DIC; \
	  echo "changed" > changed.tit
Converting all tit files in the directory /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/leim/CXTERM-DIC

[...]

Saving file /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/leim/leim-list.el...
Wrote /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/leim/leim-list.el
Updating /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/leim/leim-list.el ... done
sed -n '/^[^;]/ p' < /home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/leim/leim-ext.el >> leim-list.el
make[1]: Leaving directory `/srv/data/home/e/eliz/emacs.cvs/pretest/emacs-23.0.91/leim'
eliz@fencepost:~/emacs.cvs/pretest/emacs-23.0.91$ exit

Script done on Sat Feb 28 05:07:47 2009

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Stefan Monnier]

>> I am not interested in adding hacks specifically to speed up building
>> from the tarball.
> Thanks a lot for your kind words about my repeated attempts to find an
> agreeable solution of this problem.

Since you keep insisting on finding tarball-only solutions, I felt the
need to make painfully clear that I'm not interested in such solutions.

>> I just installed the patch that prevents recompiling
>> all the .elc files just because the src/bootstrap-emacs isn't present.

> Well, it doesn't seem to work.  Here's what I did to test it (on a
> GNU/Linux machine):

>   . unpack emacs-23.0.91 pretest tarball
>   . replace the top-level Makefile.in with the one from today's CVS
>   . verify by `diff' that Makefile.in indeed has your changes
>   . ./configure --without-x && make

> Excerpts from the resulting build session are below (the omitted parts
> are marked with "[...]"); they still show that many Lisp files are
> recompiled, after building bootstrap-emacs.

> Did I perhaps need to copy more files from CVS?

No idea.

> Also note this error message:

>   /bin/sh: line 6: cd: src: No such file or directory

> It probably means that something else is wrong with your changes.

Indeed.  We'll have to dig deeper.


        Stefan

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: 2151@emacsbugs.donarmstrong.com
> Date: Sat, 28 Feb 2009 17:06:53 -0500
> 
> >> I am not interested in adding hacks specifically to speed up building
> >> from the tarball.
> > Thanks a lot for your kind words about my repeated attempts to find an
> > agreeable solution of this problem.
> 
> Since you keep insisting on finding tarball-only solutions, I felt the
> need to make painfully clear that I'm not interested in such solutions.

And painful it was, indeed.

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Eli Zaretskii]

This bug is still present when building 23.0.93 on GNU/Linux.

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Lars Magne Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> This bug is still present when building 23.0.93 on GNU/Linux.

Is this bug still relevant?  I mean, these days probably most people
builds from bzr and not the tarballs...

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Chong Yidong]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> This bug is still present when building 23.0.93 on GNU/Linux.
>
> Is this bug still relevant?  I mean, these days probably most people
> builds from bzr and not the tarballs...

I don't know if that is true, and anyway the actual releases are
distributed as tarballs.  So yes, this is still relevent.

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Glenn Morris]

Version: 24.5

Fixed thanks to GNU make.

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Date: Thu, 26 Jun 2014 20:42:51 -0400
> 
> Fixed thanks to GNU make.

At last!  Thank you.

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Kevin Rodgers]

Stefan Monnier wrote:
> My guess is that many more people build from CVS than from the release
> tarball.  Most users of the release will use a precompiled package.
> So I care much more about building from CVS than building from
> a tarball.  Also I care more about the build being reliable and
> maintainable, than about it being optimal in all circumstances, so if
> building from the tarball does a bit of extra work, I don't really mind
> as long as it makes maintenance easier.

Is there a reliable way to count CVS users vs. tarball users?

-- 
Kevin Rodgers
Denver, Colorado, USA

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Magnus Henoch]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> PS: Part of the problem, as far as I understand it, is that we need
> the .elc files to depend on bootstrap-emacs so as to tell `make' to
> build bootstrap-emacs when needed, but we don't actually need the .elc
> files to be newer than bootstrap-emacs (in some rare cases a newer
> bootstrap-emacs will compile the .el differently, but in general it's
> not the case).  But `make' doesn't have such a concept of a dependency
> that "needs to exist, but doesn't need to be older".

I think (info "(make)Prerequisite Types") provides just that.  (And the
Features section at least doesn't explicitly say that this is unique for
GNU make.)

I.e. something like:

%.elc: %.el | bootstrap-emacs
       cp $< $@

Apologies if I'm missing the point; I just wanted to point out this
rarely-used make feature.

Magnus

bug#2151: 23.0.90; Building the 23.0.90 pretest recompiles Lisp files

[Eli Zaretskii]

> From: Magnus Henoch <mange@freemail.hu>
> Date: Mon, 02 Feb 2009 21:25:59 +0000
> Cc: 2151@emacsbugs.donarmstrong.com
> 
> I think (info "(make)Prerequisite Types") provides just that.  (And the
> Features section at least doesn't explicitly say that this is unique for
> GNU make.)
> 
> I.e. something like:
> 
> %.elc: %.el | bootstrap-emacs
>        cp $< $@

I'm quite sure this is definitely GNU Make specific, both the static
pattern rules using % and the | thing.  Even if other Unix Make
flavors support something like that, not all of them do.

I think providing an empty bootstrap-emacs file in the tarball should
solve the problem in a much more portable way.