Current CVS doesn't bootstrap

Current CVS doesn't bootstrap

[Eli Zaretskii]

With today's CVS, I get this error message from "make bootstrap":

    Compiling /home/e/eliz/emacs.cvs/emacs/lisp/./printing.el

    In toplevel form:
    printing.el:2474:1:Error: Malformed menu in easy-menu: (keymap)
    make[1]: *** [compile] Error 1
    make[1]: Leaving directory `/home/e/eliz/emacs.cvs/emacs/lisp'
    make: *** [bootstrap] Error 2

This is a freshly-checkedout tree, so I don't think stale files can be
the culprit.

TIA

Re: Current CVS doesn't bootstrap

[Andreas Schwab]

"Eli Zaretskii" <eliz@gnu.org> writes:

> With today's CVS, I get this error message from "make bootstrap":
>
>     Compiling /home/e/eliz/emacs.cvs/emacs/lisp/./printing.el
>
>     In toplevel form:
>     printing.el:2474:1:Error: Malformed menu in easy-menu: (keymap)
>     make[1]: *** [compile] Error 1
>     make[1]: Leaving directory `/home/e/eliz/emacs.cvs/emacs/lisp'
>     make: *** [bootstrap] Error 2
>
> This is a freshly-checkedout tree, so I don't think stale files can be
> the culprit.

It works again when reverting the last change to easymenu.el.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux AG, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Current CVS doesn't bootstrap

[Andreas Schwab]

"Eli Zaretskii" <eliz@gnu.org> writes:

> With today's CVS, I get this error message from "make bootstrap":
>
>     Compiling /home/e/eliz/emacs.cvs/emacs/lisp/./printing.el
>
>     In toplevel form:
>     printing.el:2474:1:Error: Malformed menu in easy-menu: (keymap)
>     make[1]: *** [compile] Error 1
>     make[1]: Leaving directory `/home/e/eliz/emacs.cvs/emacs/lisp'
>     make: *** [bootstrap] Error 2

Should work again.

Stefan, I have made up a change log entry from your commit message.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux AG, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Current CVS doesn't bootstrap

[Eli Zaretskii]

> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel@gnu.org
> From: Andreas Schwab <schwab@suse.de>
> Date: Sat, 06 Nov 2004 15:40:23 +0100
> 
> >     Compiling /home/e/eliz/emacs.cvs/emacs/lisp/./printing.el
> >
> >     In toplevel form:
> >     printing.el:2474:1:Error: Malformed menu in easy-menu: (keymap)
> >     make[1]: *** [compile] Error 1
> >     make[1]: Leaving directory `/home/e/eliz/emacs.cvs/emacs/lisp'
> >     make: *** [bootstrap] Error 2
> 
> Should work again.

Thanks, it does.

Btw, when I did a "cvs up" and tried to continue the failed bootstrap,
it failed.  The commands I typed after "cvs up" were:

   ./configure
   make bootstrap.

The last portion of the Make output is attached below.  I worked
around the problem by doing a "make maintainer-clean" and then
"make bootstrap".

Any ideas, anyone?

    Finding pointers to doc strings...
    Finding pointers to doc strings...done
    Dumping under names emacs and emacs-21.3.50
    67252 pure bytes used
    mv -f emacs bootstrap-emacs
    make[1]: Leaving directory `/home/e/eliz/emacs.cvs/emacs/src'
    (cd lisp; make  bootstrap EMACS=../src/bootstrap-emacs)
    make[1]: Entering directory `/home/e/eliz/emacs.cvs/emacs/lisp'
    wd=/home/e/eliz/emacs.cvs/emacs/lisp; subdirs=`(cd $wd; find . -type d -print)`; for file in $subdirs; do case $file in */Old | */RCS | */CVS | */CVS/* | */.* | */.*/* | */=* ) ;; *) wins="$wins $wd/$file" ;; esac; done; \
    for file in $wins; do \
       /home/e/eliz/emacs.cvs/emacs/lisp/../update-subdirs $file; \
    done;
    wd=/home/e/eliz/emacs.cvs/emacs/lisp; subdirs=`(cd $wd; find . -type d -print)`; for file in $subdirs; do case $file in */Old | */RCS | */CVS | */CVS/* | */.* | */.*/* | */=* ) ;; *) wins="$wins $wd/$file" ;; esac; done; \
    echo Directories: $wins; \
    ../src/bootstrap-emacs -batch --no-site-file --multibyte -l autoload --eval '(setq generated-autoload-file "/home/e/eliz/emacs.cvs/emacs/lisp/loaddefs.el")' -f batch-update-autoloads $wins
    Directories: /home/e/eliz/emacs.cvs/emacs/lisp/. /home/e/eliz/emacs.cvs/emacs/lisp/./calc /home/e/eliz/emacs.cvs/emacs/lisp/./gnus /home/e/eliz/emacs.cvs/emacs/lisp/./calendar /home/e/eliz/emacs.cvs/emacs/lisp/./emacs-lisp /home/e/eliz/emacs.cvs/emacs/lisp/./emulation /home/e/eliz/emacs.cvs/emacs/lisp/./eshell /home/e/eliz/emacs.cvs/emacs/lisp/./international /home/e/eliz/emacs.cvs/emacs/lisp/./language /home/e/eliz/emacs.cvs/emacs/lisp/./mail /home/e/eliz/emacs.cvs/emacs/lisp/./mh-e /home/e/eliz/emacs.cvs/emacs/lisp/./net /home/e/eliz/emacs.cvs/emacs/lisp/./obsolete /home/e/eliz/emacs.cvs/emacs/lisp/./play /home/e/eliz/emacs.cvs/emacs/lisp/./progmodes /home/e/eliz/emacs.cvs/emacs/lisp/./term /home/e/eliz/emacs.cvs/emacs/lisp/./textmodes /home/e/eliz/emacs.cvs/emacs/lisp/./toolbar /home/e/eliz/emacs.cvs/emacs/lisp/./url
    Cannot open load file: cl-macs
    make[1]: *** [autoloads] Error 255
    make[1]: Leaving directory `/home/e/eliz/emacs.cvs/emacs/lisp'
    make: *** [bootstrap] Error 2

Re: Current CVS doesn't bootstrap

[Luc Teirlinck]

Eli Zaretskii wrote:

   Btw, when I did a "cvs up" and tried to continue the failed bootstrap,
   it failed.  The commands I typed after "cvs up" were:

      ./configure
      make bootstrap.

   The last portion of the Make output is attached below.  I worked
   around the problem by doing a "make maintainer-clean" and then
   "make bootstrap".

   Any ideas, anyone?

I believe we discussed this before.  We either have to document in
INSTALL.CVS that you have to do `make maintainer-clean' before `make
bootstrap' or we have to make sure that `make bootstrap' removes the
.elc files automatically, as it used to do.  In the previous
discussion, I believe I understood that we decided on the latter and
that Kim planned to implement it, but I might have misunderstood.

Note that the problems with failing to do `make maintainer-clean' are
not just that bootstrapping may fail.  Sometimes one is able to
bootstrap without any problems, but the compiled files that Emacs is
using are not the same as the source files.  This is actually worse,
because one usually will not notice this and it can lead to plenty of
confusion when trying to debug stuff.  That is why I always run `make
maintainer-clean' before `make bootstrap'.  I ran into problems
continuously before I started doing so.

Sincerely,

Luc.

Re: Current CVS doesn't bootstrap

[Andreas Schwab]

Luc Teirlinck <teirllm@dms.auburn.edu> writes:

> I believe we discussed this before.  We either have to document in
> INSTALL.CVS that you have to do `make maintainer-clean' before `make
> bootstrap' or we have to make sure that `make bootstrap' removes the
> .elc files automatically, as it used to do.

Alternatively, we could implement an option that tells load to ignore
*.elc files that are out of date and load the *.el file instead.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux AG, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Current CVS doesn't bootstrap

[Luc Teirlinck]

Andreas Schwab wrote:

   Alternatively, we could implement an option that tells load to ignore
   *.elc files that are out of date and load the *.el file instead.

This would actually be a change that would be far more drastic and
general than the two alternatives.  Its effect would not be limited to
bootstrapping.

I believe we discussed this before and the reasons for not doing so
still remain valid.  If I make changes to a .el file, then at a given
moment I have to save these to disk.  But that does not mean that the
file is ready for use.  It only is when I compile and take a look at
what the compiler has to say.

Also, it could lead to a slowdown in the functions in the file, which
might be confusing to the user, who might not know that the .el file
is being used.

Moreover, if a .el file is newer than the .elc file, then I _believe_
that the present version of make bootstrap _already_ recompiles
anyway.  (It is a long time ago that I did a `make bootstrap' without
a prior `make maintainer-clean', however.) 

I believe that your proposed solution would not help with some of the
most common problems that arise with the current version of `make
bootstrap' (without prior `make maintainer-clean').  One of these is
changes in byte compilation.

Another problem (that can lead to very confusing results) is if you
make changes to a bunch of files, revert the changes and forget to
manually recompile some of the reverted files.  Now the out of date
.elc files are _newer_ than the source files and thus are not out of
date in as far as the computer is concerned.

Sincerely,

Luc.

Re: Current CVS doesn't bootstrap

[Andreas Schwab]

Luc Teirlinck <teirllm@dms.auburn.edu> writes:

> Andreas Schwab wrote:
>
>    Alternatively, we could implement an option that tells load to ignore
>    *.elc files that are out of date and load the *.el file instead.
>
> This would actually be a change that would be far more drastic and
> general than the two alternatives.  Its effect would not be limited to
> bootstrapping.

It's only an option, maybe even hidden.  I never proposed to enable that
by default.

> Moreover, if a .el file is newer than the .elc file, then I _believe_
> that the present version of make bootstrap _already_ recompiles
> anyway.  (It is a long time ago that I did a `make bootstrap' without
> a prior `make maintainer-clean', however.) 

But the problem is that the file to be compiled may require other files
with a out-of-date elc files.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux AG, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Current CVS doesn't bootstrap

[Satyaki Das]

Andreas Schwab <schwab@suse.de> writes:

> > Moreover, if a .el file is newer than the .elc file, then I _believe_
> > that the present version of make bootstrap _already_ recompiles
> > anyway.  (It is a long time ago that I did a `make bootstrap' without
> > a prior `make maintainer-clean', however.) 
> 
> But the problem is that the file to be compiled may require other files
> with a out-of-date elc files.

Yes, this is exactly the problem, especially if the code that you
are compiling have macros.  Then the old macros are used in the
new .elc files and all hell breaks loose.

We came across this in MH-E.  The solution that we came up with
was to add an advice to require to load the uncompiled file
instead of the .elc file which might be stale.  The code is in
lisp/mh-e/mh-acros.el.  For general use during emacs compilation,
I would vote for something a little more general where we check
the time-stamps of the .el and .elc file and load the .el if the
.elc is older.

Satyaki

Re: Current CVS doesn't bootstrap

[Luc Teirlinck]

As I already pointed out, the "loading .el if newer than .elc"
solution will solve _some_ problems caused by the change in `make
bootstrap' but not all.  Removing all .elc files during `make
bootstrap', as used to be the case, will solve not only the MH-E type
problems but all other problems created by the unfortunate change to
`make bootstrap' as well.  It would be a lot more solid.

Sincerely,

Luc.

Re: Current CVS doesn't bootstrap

[Luc Teirlinck]

Andreas Schwab wrote:

   Alternatively, we could implement an option that tells load to ignore
   *.elc files that are out of date and load the *.el file instead.

I believe that I forgot to read the word "option".  But that would
mean that we would still have to choose one of the two other
alternatives for users who did not want the option.  Also, as I
already pointed out, I do not believe that the option would solve the
most common present problems with `make bootstrap' without `make
maintainer-clean', even for those users who might choose to use the
option.

Sincerely,

Luc.

Re: Current CVS doesn't bootstrap

[Andreas Schwab]

Luc Teirlinck <teirllm@dms.auburn.edu> writes:

> Andreas Schwab wrote:
>
>    Alternatively, we could implement an option that tells load to ignore
>    *.elc files that are out of date and load the *.el file instead.
>
> I believe that I forgot to read the word "option".  But that would
> mean that we would still have to choose one of the two other
> alternatives for users who did not want the option.

I believe that enabling that option unconditionally during "make
bootstrap" will prevent all future occurences of this problem.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux AG, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Current CVS doesn't bootstrap

[Richard Stallman]

    > I believe we discussed this before.  We either have to document in
    > INSTALL.CVS that you have to do `make maintainer-clean' before `make
    > bootstrap' or we have to make sure that `make bootstrap' removes the
    > .elc files automatically, as it used to do.

What was the motive for changing it not to do that?

Re: Current CVS doesn't bootstrap

[Luc Teirlinck]

Richard Stallman wrote:

       > I believe we discussed this before.  We either have to document in
       > INSTALL.CVS that you have to do `make maintainer-clean' before `make
       > bootstrap' or we have to make sure that `make bootstrap' removes the
       > .elc files automatically, as it used to do.

   What was the motive for changing it not to do that?

To make bootstrapping run faster.  We discussed this before and
decided to have `make bootstrap' remove the .elc files once again, but
provide some alternate version bootstrap-build that does not do that.

Essentially:

http://lists.gnu.org/archive/html/emacs-devel/2004-09/msg00720.html

Again, this was never implemented.  I am not familiar enough with the
Makefiles to do this myself.

Sincerely,

Luc.

Re: Current CVS doesn't bootstrap

[Luc Teirlinck]

>From my previous message:

   We discussed this before and
   decided to have `make bootstrap' remove the .elc files once again, but
   provide some alternate version bootstrap-build that does not do that.

   Essentially:

   http://lists.gnu.org/archive/html/emacs-devel/2004-09/msg00720.html

   Again, this was never implemented.  I am not familiar enough with the
   Makefiles to do this myself.

I forgot that you asked me before not to send you urls.  Here is the full
message referred to:

    From: Stefan Monnier
    Subject: Re: bootstrap fails on tty-supports-face-attributes-p
    Date: Tue, 21 Sep 2004 16:09:28 -0400
    User-agent: Gnus/5.1006 (Gnus v5.10.6) Emacs/21.3.50 (gnu/linux)

    > Let me repeat my proposal that 'make bootstrap' should do
    > maintainer-clean.

    Mostly agreed, with the following caveat: don't just change
    `bootstrap'.  Instead, rename the current `bootstrap' to
    `bootstrap-build' and than create a new `bootstrap' which first does
    `bootstrapclean' and then `bootstrap-build'.


	    Stefan

Re: Current CVS doesn't bootstrap

[Richard Stallman]

	Mostly agreed, with the following caveat: don't just change
	`bootstrap'.  Instead, rename the current `bootstrap' to
	`bootstrap-build' and than create a new `bootstrap' which first does
	`bootstrapclean' and then `bootstrap-build'.

I just implemented this, but I see someone beat me to it.
Thanks.

Re: Current CVS doesn't bootstrap

[Kim F. Storm]

Luc Teirlinck <teirllm@dms.auburn.edu> writes:

> Richard Stallman wrote:
>
>        > I believe we discussed this before.  We either have to document in
>        > INSTALL.CVS that you have to do `make maintainer-clean' before `make
>        > bootstrap' or we have to make sure that `make bootstrap' removes the
>        > .elc files automatically, as it used to do.
>
>    What was the motive for changing it not to do that?
>
> To make bootstrapping run faster.  We discussed this before and
> decided to have `make bootstrap' remove the .elc files once again, but
> provide some alternate version bootstrap-build that does not do that.

I made the change.

So 'make bootstrap' now removes the .elc files.

There is a new 'make bootfast' target that doesn't do this (identical to
the old 'make bootstrap'.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Current CVS doesn't bootstrap

[Luc Teirlinck]

Kim Storm wrote:

   I made the change.

   So 'make bootstrap' now removes the .elc files.

   There is a new 'make bootfast' target that doesn't do this (identical to
   the old 'make bootstrap'.

I took another look at INSTALL.CVS and, after this change, we can
probably leave INSTALL.CVS unchanged.

Sincerely,

Luc.

Re: Current CVS doesn't bootstrap

[Eli Zaretskii]

> Date: Sun, 7 Nov 2004 17:45:07 -0600 (CST)
> From: Luc Teirlinck <teirllm@dms.auburn.edu>
> CC: schwab@suse.de, eliz@gnu.org, emacs-devel@gnu.org, rms@gnu.org,
>         monnier@iro.umontreal.ca
> 
> Kim Storm wrote:
> 
>    I made the change.
> 
>    So 'make bootstrap' now removes the .elc files.
> 
>    There is a new 'make bootfast' target that doesn't do this (identical to
>    the old 'make bootstrap'.
> 
> I took another look at INSTALL.CVS and, after this change, we can
> probably leave INSTALL.CVS unchanged.

I don't think so.  I think we should tell when the new bootfast target
should be used.  After all, it is there for a reason.

Could Stefan, or someone else who uses the fast bootstrap, please tell
how do they know when it is safe to use it?  Is there some method
besides trial-and-error?

Here's something about "make bootfast" that I wonder whether it could
cause trouble in specific situations: could the bootstrap-prepare and
bootstrap-clean-before targets that run before it remove some files
which will cause the rest of bootstrap fail in a way similar to what I
saw?

Re: Current CVS doesn't bootstrap

[Luc Teirlinck]

Eli Zaretskii wrote:

   I don't think so.  I think we should tell when the new bootfast target
   should be used.

If I understand Kim's change correctly, it is just the old `make bootstrap'
we all have been using for a while.  I believe it is a version of
`make bootstrap'  that is meant for people who know what they are
doing.  (Which is why making it the default was a mistake.)

Sincerely,

Luc.

Re: Current CVS doesn't bootstrap

[Eli Zaretskii]

> Date: Sat, 6 Nov 2004 16:48:14 -0600 (CST)
> From: Luc Teirlinck <teirllm@dms.auburn.edu>
> CC: schwab@suse.de, monnier@iro.umontreal.ca, emacs-devel@gnu.org
> 
> Eli Zaretskii wrote:
> 
>    Btw, when I did a "cvs up" and tried to continue the failed bootstrap,
>    it failed.  The commands I typed after "cvs up" were:
> 
>       ./configure
>       make bootstrap.
> 
>    The last portion of the Make output is attached below.  I worked
>    around the problem by doing a "make maintainer-clean" and then
>    "make bootstrap".
> 
>    Any ideas, anyone?
> 
> I believe we discussed this before.

Can you point me to these discussions?  The only one I found by
searching the archives was inconclusive (because the problem
disappeared and the OP couldn't reproduce it anymore).

> We either have to document in INSTALL.CVS that you have to do `make
> maintainer-clean' before `make bootstrap' or we have to make sure
> that `make bootstrap' removes the .elc files automatically, as it
> used to do.

I don't think this is the right solution.  First, either
maintainer-clean or removing *.elc files causes the next bootstrap to
run much longer, so we should avoid that as much as we could.
Ideally, "make maintainer-clean" should be used only if "make
bootstrap" fails (and that is something that we should add to
INSTALL.CVS _right_now_).  We should look for a way to allow the
bootstrap to continue from the point the last one failed, without the
need to recompile all Lisp files.

In any case, I don't believe that stale *.elc files were the reason
for the failure in my case, because all "cvs up" did was to update a
single Lisp file: easymenu.el.  Why should that cause a failure with
loading cl-macs, and how removing *.elc files should fix that is
something I need to understand before I agree that the solutions you
mentioned are reasonable.  After all, the only file that failed to
compile was printing.el, so after updating easymenu.el, I'd expect the
bootstrap to proceed right from that point and thus time succeed,
right?  And yet it fails.  Why?

Re: Current CVS doesn't bootstrap

[Luc Teirlinck]

Eli Zaretskii wrote:

   > We either have to document in INSTALL.CVS that you have to do `make
   > maintainer-clean' before `make bootstrap' or we have to make sure
   > that `make bootstrap' removes the .elc files automatically, as it
   > used to do.

   I don't think this is the right solution.  First, either
   maintainer-clean or removing *.elc files causes the next bootstrap to
   run much longer, so we should avoid that as much as we could.
   Ideally, "make maintainer-clean" should be used only if "make
   bootstrap" fails (and that is something that we should add to
   INSTALL.CVS _right_now_).

Most of the time the problems caused by omitting `make maintainer-clean'
are not that bootstrapping fails, but that various bugs occur while
running the bootstrapped Emacs.  Usually, people have not the slightest
idea that these bugs are caused by failure to run `make maintainer-clean'.
I remember having seen many examples of fake bug reports caused by this.

I have a 1.7 dual Xeon.  That was a pretty fast machine when I bought
it three years ago, but today, just about any computer I see
advertised is faster and you can easily get something twice as fast.
On my machine, the entire:

make maintainer-clean
./configure
make bootstrap

takes less than 15 minutes.  Omitting the `make maintainer-clean'
probably saves about 7 of those 15 minutes and (to me) that is not
worth worrying about the potential bugs it introduces.  It are 7
minutes of my computer's time, not of my time.  I noticed on several
sites that many people are _trying_ to do the above procedure, but are
using `make-distclean', mistakenly believing that it will do what
`make maintainer-clean' does.

I realize that on older machines, and especially on operating systems
that can only run one process at a time, the situation may be very
different.

I propose, for now, the following patch to INSTALL.CVS.  Andreas'
proposed change would eliminate one of the three problems listed in
that patch, but not the two others.  I do not know whether that list
is exhaustive.  (I would guess not).

If the last added paragraph would seem to long or technical, it could
be replaced by one line saying: "`make distclean' is not a valid
substitute for `make maintainer-clean'".

===File ~/INSTALL.CVS-diff==================================
*** INSTALL.CVS	02 Apr 2004 12:04:32 -0600	1.3
--- INSTALL.CVS	07 Nov 2004 10:44:55 -0600	
***************
*** 11,19 ****
  The bootstrap process makes sure all necessary files are rebuilt
  before it builds the final Emacs binary.
  
  Normally, it is not necessary to use "make bootstrap" after every CVS
! update.  Unless there are problems, we suggest the following
! procedure:
  
    $ ./configure
    $ make
--- 11,23 ----
  The bootstrap process makes sure all necessary files are rebuilt
  before it builds the final Emacs binary.
  
+ The best way to proceed after a CVS update depends on how fast or busy
+ your computer is, whether your operating system can run more than one
+ job at a time and on how badly you want to avoid potential bugs.
+ 
  Normally, it is not necessary to use "make bootstrap" after every CVS
! update.  Thus, if you have a slow computer, or if you can only run one
! process at a time, we suggest the following procedure:
  
    $ ./configure
    $ make
***************
*** 39,44 ****
--- 43,73 ----
  
  If either of above procedures fails, try "make bootstrap".
  
+ If this still fails, do:
+ 
+ make maintainer-clean
+ ./configure
+ make bootstrap
+ 
+ You may also want to do this if you report bugs that other people can
+ not reproduce.  In fact, if you have a reasonably fast computer that
+ can run more than one process at a time you may always want to do the
+ above straight away, as it is the only way to be totally sure that
+ your Emacs is completely up to date.  However, `make maintainer-clean'
+ can slow down `make bootstrap' considerably on slower computers.
+ 
+ `make maintainer-clean' removes all .elc files, so that the subsequent
+ `make bootstrap' will recompile them.  Note that just doing `make
+ distclean' does not do this.  Even without prior `make maintainer-clean',
+ `make bootstrap' will automatically recompile all files whose .elc
+ file is newer than their .el files.  But this does not prevent all
+ problems.  For instance, problems arise when a file requires an out of
+ date file that has not been recompiled yet, when changes to byte
+ compiling are made, when the file that should be loaded is actually
+ older than the wrong file and similar situations.  All C files are
+ automatically recompiled by all procedures listed above, so there is
+ no need to worry about them.
+ 
  Users of non-Posix systems (MS-Windows etc.) should run the
  platform-specific configuration scripts (nt/configure.bat, config.bat,
  etc.) before "make bootstrap" or "make"; the rest of the procedure is
============================================================

Re: Current CVS doesn't bootstrap

[Han Boetes]

Luc Teirlinck wrote:
> I noticed on several sites that many people are _trying_ to do
> the above procedure, but are using `make-distclean', mistakenly
> believing that it will do what `make maintainer-clean' does.

Would it be an idea to replace the distclean target with the
maintainer-clean target?



# Han

Re: Current CVS doesn't bootstrap

[Luc Teirlinck]

Han Boetes wrote:

   Would it be an idea to replace the distclean target with the
   maintainer-clean target?

I believe that the argument for not doing that is that the .elc files
are included with Emacs releases, and hence should not be removed by
`make distclean'.  .elc files are not included with CVS Emacs (hence
the problems and the confusion), but CVS Emacs is technically not a
"distribution".

Sincerely,

Luc.

Re: Current CVS doesn't bootstrap

[David Kastrup]

Luc Teirlinck <teirllm@dms.auburn.edu> writes:

> I have a 1.7 dual Xeon.  That was a pretty fast machine when I
> bought it three years ago, but today, just about any computer I see
> advertised is faster and you can easily get something twice as fast.

If you pay for it.  My laptop is a 600MHz single Pentium III, and my
desktop is a 200MHz AMD K6II.

Both are perfectly fine running Emacs/LaTeX and quite a few add-ons
(including the interactive preview-latex) for text processing.  There
is little point in acquiring more demanding hardware (and in the
laptop area, you nowadays have to choose between ridiculously small
and ridiculously heavy ones, either completely unsuitable for work or
for transportation).  And I am living in a country with quite more
than average income when compared with the rest of the world that
could be interested in running Emacs.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Current CVS doesn't bootstrap

[Luc Teirlinck]

David Kastrup wrote:

   If you pay for it.  My laptop is a 600MHz single Pentium III, and my
   desktop is a 200MHz AMD K6II.

There are two points I wanted to make:

1.  The main recommended procedure described in INSTALL.CVS does not
remove the .elc files.  It only recommends to do `make bootstrap' in
case of problems.  But the current version of `make bootstrap' is not
likely to solve these problems, most of which are caused by failing to
recompile .el files.

2.  The second point I was trying to make, which I guess is what you
reacted to, is that for people that _do_ have reasonably fast
machines, _always_ doing a `make maintainer-clean' might be better
than the procedure recommended in INSTALL.CVS.  It can save many
people a lot of time worrying about fake bugs.  If you do have
resources, you might as well take advantage of them.

INSTALL.CVS already describes a procedure for people with slower
computers to use.  The proposed `make bootstrap-build' would preserve
the current `make-bootstrap' behavior as a second alternative for slow
computers.

But currently people are mislead into believing that `make bootstrap'
is guaranteed to produce something completely equivalent to a fresh
checkout.  This very far from the case with the current version.

Sincerely,

Luc.

Re: Current CVS doesn't bootstrap

[David Kastrup]

Luc Teirlinck <teirllm@dms.auburn.edu> writes:

> David Kastrup wrote:
>
>    If you pay for it.  My laptop is a 600MHz single Pentium III, and my
>    desktop is a 200MHz AMD K6II.
>
> There are two points I wanted to make:
>
> 1.  The main recommended procedure described in INSTALL.CVS does not
> remove the .elc files.  It only recommends to do `make bootstrap' in
> case of problems.  But the current version of `make bootstrap' is
> not likely to solve these problems, most of which are caused by
> failing to recompile .el files.
>
> 2.  The second point I was trying to make, which I guess is what you
> reacted to, is that for people that _do_ have reasonably fast
> machines, _always_ doing a `make maintainer-clean' might be better
> than the procedure recommended in INSTALL.CVS.  It can save many
> people a lot of time worrying about fake bugs.  If you do have
> resources, you might as well take advantage of them.

Well, if that were the points you were trying to make, you watered
them down unnecessarily with introducing your hardware.  It is ok to
tell people what bargains in reliability versus performance they can
make.  It is not ok to assume that performance may never be an issue.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Current CVS doesn't bootstrap

[Luc Teirlinck]

David Kastrup wrote:

   Well, if that were the points you were trying to make, you watered
   them down unnecessarily with introducing your hardware.

I did that to give concrete idea of the times that were involved on
fast computers, with concrete numbers.

   It is ok to tell people what bargains in reliability versus
   performance they can make.  It is not ok to assume that performance
   may never be an issue.

I believe that in my proposed patch to INSTALL.CVS, I did the former
and not the latter.  Both in my patch and in my preceding remarks I
mentioned that the OS may only be able to run one job at a time, in
which case performance is obviously a big issue.

I believe that you read things in my message that were not in there.

Maybe part of my message implied a conjecture that the majority of new
computers bought _today_ would be fast enough.  Maybe that conjecture
was wrong, but anyway, it is irrelevant since people do not buy a new
computer every year.  The only conclusion I made out of that was that
it would be good to mention the _possibility_ of always using `make
maintainer-clean', since the number of people for which it makes sense
is already non-trivial and will be growing over time.

Sincerely,

Luc.

Re: Current CVS doesn't bootstrap

[Robert J. Chassell]

   I did that to give concrete idea of the times that were involved on
   fast computers, with concrete numbers.

I did a `make maintainer-clean' this morning and the full build took
me 1.5 hours.  I will spend the time when needed, but not otherwise.

      It is ok to tell people what bargains in reliability versus
      performance they can make.  It is not ok to assume that
      performance may never be an issue.

Yes, very true.

   ... it would be good to mention the _possibility_ of always using
   `make maintainer-clean' ...

Yes, that is reasonable for people with fast machines.  We should
encourage them.  But for others, we should be practical....

-- 
    Robert J. Chassell                         
    bob@rattlesnake.com                         GnuPG Key ID: 004B4AC8
    http://www.rattlesnake.com                  http://www.teak.cc

Re: Current CVS doesn't bootstrap

[Piet van Oostrum]

>>>>> Luc Teirlinck <teirllm@dms.auburn.edu> (LT) wrote:

LT> David Kastrup wrote:
LT>    If you pay for it.  My laptop is a 600MHz single Pentium III, and my
LT>    desktop is a 200MHz AMD K6II.

LT> There are two points I wanted to make:

LT> 1.  The main recommended procedure described in INSTALL.CVS does not
LT> remove the .elc files.  It only recommends to do `make bootstrap' in
LT> case of problems.  But the current version of `make bootstrap' is not
LT> likely to solve these problems, most of which are caused by failing to
LT> recompile .el files.

Why not include a `make -C lisp recompile' as the first step of
 make bootstrap? 
-- 
Piet van Oostrum <piet@cs.uu.nl>
URL: http://www.cs.uu.nl/~piet [PGP]
Private email: P.van.Oostrum@hccnet.nl

Re: Current CVS doesn't bootstrap

[Piet van Oostrum]

Sorry, I see you need to have a working emacs to do the 'make -C lisp
recompile'. An alternative would be to remove only those .elc files that
are older than their corresponding .el files.
-- 
Piet van Oostrum <piet@cs.uu.nl>
URL: http://www.cs.uu.nl/~piet [PGP]
Private email: P.van.Oostrum@hccnet.nl

Re: Current CVS doesn't bootstrap

[Luc Teirlinck]

Piet van Oostrum wrote:

   An alternative would be to remove only those .elc files that
   are older than their corresponding .el files.

That, or Andreas' suggestion, could actually be an improvement for the
proposed `make bootstrap-build', assuming our (apparent) consensus from a
previous thread got implemented.  But I believe that two problems
would remain.

It would seem that some changes in byte-compilation may require
recompiling everything.  I believe nothing can be done about that.

Before I started systematically using `make maintainer-clean', I
encountered several problems where the .elc file was wrong even though
_newer_ than the .el file.  For instance, `C-x v u' in VC produces
such situations.  I guess this problem could be avoided by manually
removing the .elc file each time after you do `C-x v u' or otherwise
cause this type of situation.  But this assume that most users are
aware of this danger.  It also is easy to forget, even if you know.
If you run Emacs from the place you built it, then you might want to
recompile instead of deleting the .elc file.  But then you also have
to also manually change the last modification time to ensure that the
.elc will not be considered newer than a changed .el at the next
update.  This starts getting tricky.

Sincerely,

Luc.

Re: Current CVS doesn't bootstrap

[Luc Teirlinck]

Piet van Oostrum wrote:

   An alternative would be to remove only those .elc files that
   are older than their corresponding .el files.

In addition to what I pointed out in my previous message, recompiling
files to take a look at compiler warnings might also produce falsely
"up to date" .elc files.  So this would be another situation where one
would have to be careful.  There probably are plenty of other similar
situations that I am just not thinking of right now.

Sincerely,

Luc.

Re: Current CVS doesn't bootstrap

[Eli Zaretskii]

> Date: Sun, 7 Nov 2004 11:43:41 -0600 (CST)
> From: Luc Teirlinck <teirllm@dms.auburn.edu>
> CC: emacs-devel@gnu.org
> 
> Most of the time the problems caused by omitting `make maintainer-clean'
> are not that bootstrapping fails, but that various bugs occur while
> running the bootstrapped Emacs.

The right way to solve this is to make a list of dependencies between
Lisp files.  Perhaps we could have a Lisp function that would glean
such a list and put them into lisp/Makefile.

> Usually, people have not the slightest
> idea that these bugs are caused by failure to run `make maintainer-clean'.
> I remember having seen many examples of fake bug reports caused by this.

Are you saying that a bootstrap should always be preceded by a "make
maintainer-clean"?  If so, why doesn't "make bootstrap" does the
equivalent of "make maintainer-clean" automatically?

> I have a 1.7 dual Xeon.  That was a pretty fast machine when I bought
> it three years ago, but today, just about any computer I see
> advertised is faster and you can easily get something twice as fast.
> On my machine, the entire:
> 
> make maintainer-clean
> ./configure
> make bootstrap
> 
> takes less than 15 minutes.  Omitting the `make maintainer-clean'
> probably saves about 7 of those 15 minutes and (to me) that is not
> worth worrying about the potential bugs it introduces.

Please don't try to convince me that maintainer-clean doesn't slow
down the bootstrap: it does, BIG TIME, at least in my experience.

> I propose, for now, the following patch to INSTALL.CVS.

I think your additions to INSTALL.CVS are too voluminous.  I'd simply
say something like

  Normally, it is not necessary to use "make bootstrap" after every CVS
  update.  Unless there are problems, we suggest the following
  procedure:
  
    $ ./configure
    $ make

  If this fails, do:
 
  make maintainer-clean
  ./configure
  make bootstrap

(Btw, I still didn't see any explanation why a "cvs up" that updates
easymenu.el causes a failure to load cl-macs, which was the specific
problem I reported.  Without seeing such an explanation, this entire
argument sounds a bit academic to me.)

Re: Current CVS doesn't bootstrap

[Luc Teirlinck]

Eli Zaretskii wrote:

   Are you saying that a bootstrap should always be preceded by a "make
   maintainer-clean"?

If you want to be _completely_ sure that your version of Emacs is
fully up to date.  I believe that there are two situations where
people usually run `make bootstrap': if they experience problems or if
they have a fast computer.  In case of problems, I believe it is
advisable to run `make maintainer-clean'.  If your computer is fast
enough, the entire purpose of running the `make bootstrap' sequence is
to be completely safe, so it would be inconsistent not to run `make
maintainer-clean'.  Apparently Stefan knows a counterexample to these
rules.

   If so, why doesn't "make bootstrap" does the equivalent of "make
   maintainer-clean" automatically?

It used to do so.  Then Stefan changed it to speed up bootstrapping.
But we have had plenty of bug reports because of this.  As already
mentioned, we had a discussion on this and I believe that Stefan
agreed that `make bootstrap' should be reverted to its old behavior,
and that a new target `make bootstrap-build' should do what the
current `make bootstrap' does.  Maybe some of the imperfect, but fast,
proposed partial solutions could be applied to this new target.  They
would not make it 100% safe, but much safer than it is now.

   (Btw, I still didn't see any explanation why a "cvs up" that updates
   easymenu.el causes a failure to load cl-macs, which was the specific
   problem I reported.  Without seeing such an explanation, this entire
   argument sounds a bit academic to me.)

It would be, if this were the only instance of the problem, but it is
just the latest in a long list.  It is very far from the first and if
nothing is changed it will be very far from the last.

Sincerely,

Luc.

Re: Current CVS doesn't bootstrap

[Lennart Borgman]

----- Original Message ----- 
From: "Eli Zaretskii" <eliz@gnu.org>

: The right way to solve this is to make a list of dependencies between
: Lisp files.  Perhaps we could have a Lisp function that would glean
: such a list and put them into lisp/Makefile.

What would in practice have to be checked? (require ...) and it cousins?
What about autoload.el - can it write a dependency list for the autoloaded
objects? Can everything be automatically checked or is an hand-written
supplement needed?

- Lennart

Re: Current CVS doesn't bootstrap

[Eli Zaretskii]

> From: "Lennart Borgman" <lennart.borgman.073@student.lu.se>
> Cc: <emacs-devel@gnu.org>
> Date: Mon, 8 Nov 2004 02:32:18 +0100
> 
> What would in practice have to be checked? (require ...) and it cousins?

Yes.

> What about autoload.el - can it write a dependency list for the autoloaded
> objects?

Autoloads could be picked up from loaddefs.el, yes.

> Can everything be automatically checked or is an hand-written
> supplement needed?

In principle, it could all be done automatically.

But this is probably not something to do right now, when we are trying
to start a pretest VSN.

Re: Current CVS doesn't bootstrap

[Lennart Borgman]

[-- Attachment #1: Type: text/plain, Size: 1102 bytes --]

----- Original Message ----- 
From: "Eli Zaretskii" <eliz@gnu.org>

> > What would in practice have to be checked? (require ...) and it cousins?
>
> Yes.
>
> > What about autoload.el - can it write a dependency list for the
autoloaded
> > objects?
>
> Autoloads could be picked up from loaddefs.el, yes.
>
> > Can everything be automatically checked or is an hand-written
> > supplement needed?
>
> In principle, it could all be done automatically.
>
> But this is probably not something to do right now, when we are trying
> to start a pretest VSN.

Though this may not be the write time I tried to write some routines that
checks the dependencies. It is not at all integrated with the build process,
I just wrote the routines to check dependencies. I am sending it here for a
review to see if the routines do what they are supposed to do (except for
any errors I might have done). Some more routines may also have to be
written to make it useful.

When the attached file is loaded it will go through the lisp files in the
Emacs tree and write two files lisp\dep.lst and lisp\dep.warnings.

- Lennart

[-- Attachment #2: elisp-dep.el --]
[-- Type: application/octet-stream, Size: 12242 bytes --]

;;; Bugs:
;;
;; - Does not handle file names in require.
;; - Does not handle abs file names in load.

;; From starup.el:
;; 	  (load (concat term-file-prefix
;; 			(symbol-name window-system)
;; 			"-win")

(defconst elisp-dep-autoloads-hash (make-hash-table :test 'equal :size 1500))
(defconst elisp-dep-files-hash     (make-hash-table :test 'equal :size 1500))
(defvar elisp-dep-warnings ())
;;(defvar elisp-dep-processing-file nil)
(defvar elisp-dep-dontcompile ())
  
(defun elisp-dep-warning(msg)
  (save-excursion
    (backward-sexp)
    (let* ((line-no (+ (count-lines (point-min) (point)) 1))
	   (line-no-str (format "line %s" line-no)))
      (message "WARNING (%s): %s" line-no-str msg)
      (add-to-list 'elisp-dep-warnings
		   (cons (cons elisp-dep-processing-module line-no-str) msg)))))

(defun elisp-dep-get-dontcompile()
  (message "Reading DONTCOMPILE in makefile")
  (save-excursion
    (let ((makefile (expand-file-name "../lisp/makefile" exec-directory))
	  (more t))
      (setq elisp-dep-dontcompile nil)
      (find-file makefile)
      (goto-char (point-min))
      (search-forward-regexp "^DONTCOMPILE =")
      (forward-line)
      (while more
	(if (search-forward-regexp "\\=\\s-\\$(lisp).*?/\\([^/]*?\\).el\\(?: \\|$\\)" nil t)
	    (progn
	      (add-to-list 'elisp-dep-dontcompile (match-string-no-properties 1))
	      (forward-line))
	  (setq more nil)))
      (kill-buffer (current-buffer)))))
;;(elisp-dep-get-dontcompile)
;;(member "version" elisp-dep-dontcompile)

(defun elisp-dep-get-autoloads()
  (message "Reading autoloads..")
  (setq elisp-dep-autoloads-hash (make-hash-table :test 'equal :size 1500))
  (find-file (expand-file-name "../lisp/loaddefs.el" exec-directory))
  (goto-char (point-min))
  (let ((sexp)
	(count 1))
    (while (forward-comment 1))
    (while (not (eobp))
      (setq sexp (read (current-buffer)))
      (when (equal (car sexp) 'autoload)
	(let ((objname (car (cdr sexp)))
	      (objmodule (nth 2 sexp)))
	  ;;(error "%s" objmodule)
	  (unless (equal (car objname) 'quote)
	    (error "expected quote"))
	  (setq objname (car (cdr objname)))
	  (puthash objname objmodule elisp-dep-autoloads-hash)
	  (setq count (+ count 1))
	  )
	)
      (while (forward-comment 1))
      )
    (message "Number of autoloads: %s" count)
    (sleep-for 1)
    (kill-buffer (current-buffer)) ) )



(defun elisp-dep-add-1-extra-dep(module extra-dep)
  (let ((depends (gethash module elisp-dep-files-hash)))
    (mapc (lambda(sym) (add-to-list 'depends sym)) extra-dep)
    (message "add-1-extra: %s %s" module depends)
    (puthash module depends elisp-dep-files-hash)))

(defun elisp-dep-add-extra-dep()
  "Add dependencies not found with the static search."
  ;; FIX ME: not completed.
  (message "Adding known dependencies that can not be found by elisp-dep")
  (elisp-dep-add-1-extra-dep "edt" '(edt-lk201 edt-pc edt-vt100))
  )

(defun elisp-dep-warn-load-but-not-string(sexp)
  "Avoid warning for known things."
  ;; FIX ME: not completed.
  (cond 

   ;;(("abbrev" . "line 174") . "load but not string: (load (if (and file (> (length file) 0)) file abbrev-file-name) nil quietly)")
   ((equal sexp
	   '(load (if (and file (> (length file) 0)) file abbrev-file-name) nil quietly)))

   ;;(("battery" . "line 244") . "load but not string: (load file-name nil t t)")
   ((equal sexp
	   '(load file-name nil t t)))

   ;;(("byte-opt" . "line 256") . "load but not string: (load (nth 1 fn))")
   ((equal sexp
	   '(load (nth 1 fn))))

   ;;(("bytecomp" . "line 1323") . "load but not string: (load target-file)")
   ((equal sexp
	   '(load target-file)))

   ;;(("cc-bytecomp" . "line 149") . "load but not string: (load cc-file nil t t)")
   ((equal sexp
	   '(load cc-file nil t t)))

   ;;(("cc-bytecomp" . "line 188") . "load but not string: (load (, cc-part) nil t nil)")
   ((equal sexp
	   '(load ,cc-part nil t nil)))

   ;;(("cl-macs" . "line 2429") . "load but not string: (load (nth 1 (symbol-function func)))")
   ((equal sexp
	   '(load (nth 1 (symbol-function func)))))

   ;;(("cus-edit" . "line 1816") . "load but not string: (load-library load)")
   ((equal sexp
	   '(load-library load)))

   ;;(("cus-edit" . "line 892") . "load but not string: (load file)")
   ((equal sexp
	   '(load file)))

   ;;(("desktop" . "line 565") . "load but not string: (load (expand-file-name desktop-basefilename desktop-dirname) t t t)")
   ((equal sexp
	   '(load (expand-file-name desktop-basefilename desktop-dirname) t t t)))

   ;;(("dired-aux" . "line 717") . "load but not string: (load file nil nil t)")
   ((equal sexp
	   '(load file nil nil t)))

   ;;(("disass" . "line 73") . "load but not string: (load (nth 1 obj))")
   ((equal sexp
	   '(load (nth 1 obj))))

   ;;(("edt" . "line 2141") . "load but not string: (load (concat edt- term) t t)")
   ((equal sexp
	   '(load (concat "edt-" term) t t)))

   ;;(("esh-mode" . "line 292") . "load but not string: (load module-shortname)")
   ((equal sexp
	   '(load module-shortname)))





   (t
    (elisp-dep-warning (format "load but not string: %s" sexp)))))



(defun elisp-dep-get-sexp-depends(sexp &optional let-state)
  ;;(message "get-sexp-depends %s %S" let-state sexp)
  (if (not (consp sexp))
      (progn
	;; This can only happen on top level!
	(elisp-dep-warning (format "top-level sexp is not list: %s" sexp))
	nil)
    (let ((dependon ())
	  (fun (car sexp))
	  (new-let-state))

      (when let-state
	;; Skip let variables:
	(when (= let-state 0)
	  (setq sexp (cdr sexp))
	  ;;(message "new sexp=%S" sexp)
	  )
	(setq new-let-state (- let-state 1))
	(when (< let-state 0)
	  (setq new-let-state nil)))

      (when (or (equal fun 'let)
		(equal fun 'let*))
	(setq new-let-state 1))
      
      (when (equal fun 'require)
	(let ((reqarg (car (cdr sexp))))
	  (if (listp reqarg)
	      (if (equal (car reqarg) 'quote)
		  (let ((mod (car (cdr reqarg))))
		    ;;(message "require.mod=%s" mod)
		    (add-to-list 'dependon mod))
		(elisp-dep-warning (format "require but not quote: %s" sexp)))
	    (elisp-dep-warning (format "require but not listp: %s" sexp))
	    )))
      ;;(message "get-s-d.dependon=%s" dependon)))
      (when (or (equal fun 'load)
		(equal fun 'load-library)
		)
	(let ((mod (car (cdr sexp))))
	  ;;(message "load %s" mod)
	  (if (stringp mod)
	      (cond
	       ((file-name-absolute-p mod)
		(elisp-dep-warning (format "skipping %s - absolute file name" mod)))
	       ;;((> (length (file-name-extension mod)) 0)
		;;(elisp-dep-warning (format "skipping %s - file name has extension" mod)))
	       (t
		(setq mod (file-name-sans-extension (file-name-nondirectory mod)))
		(add-to-list 'dependon mod)))
	    (elisp-dep-warn-load-but-not-string sexp))))

      ;; skip arguments for functions etc
      (let ((has-args (or (equal fun 'defun)
			  (equal fun 'defmacro)))
	    (ord 0)
	    )
	(while sexp
	  (setq ord (+ ord 1))
	  (let ((sym (if (consp sexp) (car sexp) sexp)))
	    (when sym
	      (when (atom sym)
		(let ((mod (gethash sym elisp-dep-autoloads-hash)))
		  (when mod
		    (add-to-list 'dependon mod)
		    ;;(message "sym=%s mod=%s" sym mod)
		    ;;(message "2 get-s-d.dependon=%s" dependon)
		    )))
	      (when (listp sym)
		;;(when (listp (cdr sym)) ;; must use cdr to distinguish cons/list
		;;(message "ord=%s" ord)
		(unless (and has-args (= ord 3))
		  ;;(error ";; FIX-ME: take care of the return value!!!:")
		  (mapc (lambda (sym) 
			  (add-to-list 'dependon sym))
			(elisp-dep-get-sexp-depends sym new-let-state)))
		;; Only the first list after let is declares -> reset counter
		(when (equal new-let-state 1) (setq new-let-state nil)))) ;;)
	    (setq sexp (when (consp sexp) (cdr sexp))))))
      dependon)))


  
(defun elisp-dep-get-buffer-depends()
  ;;(message "get-buffer-depends=%s" (buffer-string))
  (let ((depon ()))
    (goto-char (point-min))
    (while (forward-comment 1))
    (while (not (eobp))
      ;;(message "substring=%s" (buffer-substring (point) (min (point-max) (+ (point) 50))))
      (let* ((sexp (read (current-buffer)))
	     (sexpdep (elisp-dep-get-sexp-depends sexp))
	     )
	(mapc (lambda(sym)
		(add-to-list 'depon sym))
	      sexpdep))
      (while (forward-comment 1))
      ;;(message "(%s)" (buffer-substring (point) (point-max)))
      )
    ;;(message "depon=%s" depon)
    depon))



(defun elisp-dep-get-file-depends(file)
  (message "**** get-file-depends: %s" file)
  ;;(sleep-for 0 500)
  ;;(with-temp-buffer
  ;;(emacs-lisp-mode)
  ;;(insert-file-contents file)
  (find-file file)
  (let* ((module (file-name-nondirectory (file-name-sans-extension file)))
	 (elisp-dep-processing-module module)
	 (depends)
	 )
    (if  (member module elisp-dep-dontcompile)
	(elisp-dep-warning (format "skipping %s - member of DONTCOMPILE" module))
      (setq depends (elisp-dep-get-buffer-depends))
      ;;(message "%s=%s" module depends)
      (puthash module depends elisp-dep-files-hash)))
  (kill-buffer (current-buffer)))



(defun elisp-dep-search-dir(dir)
  ;;(message "search-dir: %s" dir)
  (let ((files (directory-files dir)))
    ;;(message "%s" files)
    (mapc (lambda (file)
	    (let ((full (expand-file-name file dir)))
	      ;;(message "full=%s isdir=%s" full (file-directory-p full))
	      (cond
	       ((equal "." file))
	       ((equal ".." file))
	       ((file-directory-p full) (elisp-dep-search-dir full))
	       ((equal (substring full -3) ".el")
		(when (file-regular-p full)
		  (elisp-dep-get-file-depends full)
		  ))
	       )))
	  files)))



(defconst elisp-dep-dep-lst-file (expand-file-name "../lisp/dep.lst" exec-directory))



(defun elisp-dep-write-dep-lst()
  (save-excursion
    (find-file elisp-dep-dep-lst-file)
    (maphash (lambda (key value)
	       (princ (format "(%s %s)\n" key value) (current-buffer)))
	     elisp-dep-files-hash)
    (sort-lines nil (point-min) (point-max))
    (save-buffer)
    (kill-buffer (current-buffer))
    (message "New dependency file written: %s" elisp-dep-dep-lst-file)))



(defun elisp-dep-mk-new-dep-lst()
  (message "Will create new lisp depencency file")
  (sleep-for 2)
  (elisp-dep-get-autoloads)
  (elisp-dep-search-dir (expand-file-name "../lisp" exec-directory))
  (elisp-dep-add-extra-dep)
  (elisp-dep-write-dep-lst)
  )



(defun elisp-dep-update-dep-lst()
  (message "Found old lisp depencency file, will update this")
  (sleep-for 2)
  (error "NIY")
  )



(defun elisp-dep-save-warnings()
  (let ((warn-file (concat (file-name-sans-extension elisp-dep-dep-lst-file) ".warnings")))
    (find-file warn-file)
    (erase-buffer)
    (mapc (lambda (warning) (princ (format "%S\n" warning) (current-buffer)))
	  elisp-dep-warnings)
    (sort-lines nil (point-min) (point-max))
    (save-buffer)
    (kill-buffer (current-buffer))
    ))



(defun elisp-dep-mk-dep-lst()
  (save-excursion
    (set-buffer "*Messages*")
    (erase-buffer))
  (message "Started at %s" (current-time-string))
  (setq elisp-dep-warnings ())
  (setq elisp-dep-files-hash     (make-hash-table :test 'equal :size 1500))
  (elisp-dep-get-dontcompile)
  ;; Must set message-log-max here, otherwise messages disappear when it is set back
  (setq message-log-max t)
  (let ((start-time (current-time))
	(stop-time)
	(used-time)
	(max-lisp-eval-depth 500)
	(enable-local-eval nil))
    (if (file-readable-p elisp-dep-dep-lst-file)
	(elisp-dep-update-dep-lst)
      (elisp-dep-mk-new-dep-lst))
    (setq stop-time (current-time))
    (setq used-time (- (+ (* (nth 0 stop-time)  (expt 2 16)) (nth 1 stop-time))
		       (+ (* (nth 0 start-time) (expt 2 16)) (nth 1 start-time))))
    (message "Ready at %s" (current-time-string))
    (message "Used time: %s" used-time)
    (message "hash count=%s" (hash-table-count elisp-dep-files-hash))
    (message "%s" elisp-dep-files-hash)
    (elisp-dep-save-warnings)
    ))



(elisp-dep-mk-dep-lst)

(provide 'elisp-dep)

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: Current CVS doesn't bootstrap

[Luc Teirlinck]

Eli Zaretskii wrote:
   
   (Btw, I still didn't see any explanation why a "cvs up" that updates
   easymenu.el causes a failure to load cl-macs, which was the specific
   problem I reported.  Without seeing such an explanation, this entire
   argument sounds a bit academic to me.)

Do you have any evidence that it are the changes to easymenu.el that
caused your problem?  Changes in byte compilation often require
recompilation.  There was a non-trivial change to bytecomp.el five
days ago.  Some of the changes apparently affected cl.  I did not
study the changes in detail and hence I do not really know, but they
could be an explanation for your problem.

Sincerely,

Luc.

Re: Current CVS doesn't bootstrap

[Luc Teirlinck]

>From my previous message:

   There was a non-trivial change to bytecomp.el five days ago.

Seems harmless at second view.

   Some of the changes apparently affected cl.

Apparently just an optical illusion.

But Kim's change solved the problem anyway.

Sincerely,

Luc.

Re: Current CVS doesn't bootstrap

[Eli Zaretskii]

> Date: Sun, 7 Nov 2004 20:03:00 -0600 (CST)
> From: Luc Teirlinck <teirllm@dms.auburn.edu>
> CC: emacs-devel@gnu.org
> 
> Do you have any evidence that it are the changes to easymenu.el that
> caused your problem?

Yes: easymenu.el was one of a very few files updated by "cvs up" that was
immediately followed by

   ./configure
   make bootstrap

The bootstrap attempt before "cvs up" failed while compiling
printing.el, so the new one should have simply continued from that
spot.

The files updated by "cvs up" were: align.el, tempo.el, outline.el,
macros.el, and easymenu.el.  None of the changes to those files seem
to be candidates for causing the trouble I reported, but perhaps I
miss something.

Note that the command that failed after "cvs up" was this:

    ../src/bootstrap-emacs -batch --no-site-file --multibyte -l autoload --eval '(setq generated-autoload-file "/home/e/eliz/emacs.cvs/emacs/lisp/loaddefs.el")' -f batch-update-autoloads $wins

i.e., it failed while generating loaddefs.el.

> Changes in byte compilation often require
> recompilation.  There was a non-trivial change to bytecomp.el five
> days ago.  Some of the changes apparently affected cl.

I'm looking for some change that would explain the effect on cl.
Until now, I didn't find it.

Also note that similar failure was reported in this thread:

  http://lists.gnu.org/archive/html/emacs-devel/2004-09/msg00317.html

RE: Current CVS doesn't bootstrap

[Drew Adams]

  -----Original Message-----From: Eli Zaretskii
  ...make a list of dependencies between Lisp files.
  Perhaps we could have a Lisp function that would glean
  such a list and put them into lisp/Makefile.

Good idea. Useful in general by users for distributing their libraries.

Re: Current CVS doesn't bootstrap

[Stefan Monnier]

>   Perhaps we could have a Lisp function that would glean
>   such a list and put them into lisp/Makefile.

The byte-compiler should probably do that.
It just needs to keep track of the macros it called and maybe the packages
it required.  But this won't help us get to a new release.


        Stefan

Re: Current CVS doesn't bootstrap

[Luc Teirlinck]

Eli Zaretskii wrote:

   >    The last portion of the Make output is attached below.  I worked
   >    around the problem by doing a "make maintainer-clean" and then
   >    "make bootstrap".
   > 
   >    Any ideas, anyone?
   > 
   > I believe we discussed this before.

   Can you point me to these discussions?  The only one I found by
   searching the archives was inconclusive (because the problem
   disappeared and the OP couldn't reproduce it anymore).

To avoid confusion:  I was talking about removing or not removing
.elc files at the beginnning of bootstrapping.

We discussed this in the thread that started talking about this at:

http://lists.gnu.org/archive/html/emacs-devel/2004-09/msg00151.html

That one concrete message in question is characteristic of the fact
that many (I believe most) people mistakenly seem to believe that
`make bootstrap" and "certainly" `make distclean' do remove all .elc
files.

Sincerely,

Luc.

Re: Current CVS doesn't bootstrap

[Luc Teirlinck]

Eli Zaretskii wrote:

   > I believe we discussed this before.

   Can you point me to these discussions?

I believe the discussion on the thread I sent before continued on
another one.  I believe that the message below was the final one at
that time and represented a consensus of people involved in the
discussion at the time.  It never got implemented.

http://lists.gnu.org/archive/html/emacs-devel/2004-09/msg00720.html

Sincerely,

Luc.

Current CVS doesn't bootstrap

[Andreas Schwab]

I'm getting this error during bootstrapping:

Loading emacs-lisp/byte-run (source)...
Loading emacs-lisp/backquote (source)...
Loading subr (source)...
Loading version.el (source)...
Loading widget (source)...
Loading custom (source)...
Loading emacs-lisp/map-ynp (source)...
Loading env (source)...
Loading cus-start (source)...
Symbol's value as variable is void: dos-unsupported-char-glyph

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Current CVS doesn't bootstrap

[Lute Kamstra]

Andreas Schwab <schwab@suse.de> writes:

> I'm getting this error during bootstrapping:
>
> Loading emacs-lisp/byte-run (source)...
> Loading emacs-lisp/backquote (source)...
> Loading subr (source)...
> Loading version.el (source)...
> Loading widget (source)...
> Loading custom (source)...
> Loading emacs-lisp/map-ynp (source)...
> Loading env (source)...
> Loading cus-start (source)...
> Symbol's value as variable is void: dos-unsupported-char-glyph

I just committed a fix.

Lute.

Re: Current CVS doesn't bootstrap

[Reiner Steib]

On Mon, Feb 14 2005, Andreas Schwab wrote:

> I'm getting this error during bootstrapping:
[...]
> Loading cus-start (source)...
> Symbol's value as variable is void: dos-unsupported-char-glyph

It seems to be related to Richards recent changes to cus-start.el.  A
possible workaround (I don't know how to fix it properly) is to revert
lisp/cus-start.el to revision 1.64 (or 1.65?).

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: Current CVS doesn't bootstrap

[Richard Stallman]

    It seems to be related to Richards recent changes to cus-start.el.  A
    possible workaround (I don't know how to fix it properly) is to revert
    lisp/cus-start.el to revision 1.64 (or 1.65?).

When there is a bug, please do not be so quick to suggest "let's
revert the previous change" as a "solution".  That might in some cases
be necessary, but unless the previous change served no purpose, just
reverting it cannot be the real solution.

The useful thing to do is to debug the problem.  Would you like to help
do that?

Re: Current CVS doesn't bootstrap

[Reiner Steib]

On Tue, Feb 15 2005, Richard Stallman wrote:

>     It seems to be related to Richards recent changes to cus-start.el.  A
>     possible workaround (I don't know how to fix it properly) is to revert
>     lisp/cus-start.el to revision 1.64 (or 1.65?).
>
> When there is a bug, please do not be so quick to suggest "let's
> revert the previous change" as a "solution".

I wrote that reverting is "workaround", not a "solution".  I didn't
mean to suggest to revert your change(s) in CVS.

> The useful thing to do is to debug the problem.  Would you like to help
> do that?

Lute Kamstra already did debug it, I think.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: Current CVS doesn't bootstrap

[Richard Stallman]

    I wrote that reverting is "workaround", not a "solution".  I didn't
    mean to suggest to revert your change(s) in CVS.

Whatever you call it, it is the wrong way to be helpful.
We need to ask people to debug the problem, not to avoid it.

Re: Current CVS doesn't bootstrap

[Richard Stallman]

Shouldn't it use default-boundp?

*** cus-start.el	15 Feb 2005 01:22:16 -0500	1.67
--- cus-start.el	15 Feb 2005 02:36:28 -0500	
***************
*** 312,318 ****
  	  ;; use the current value as the standard value.
  	  standard (if (nthcdr 4 this)
  		       (nth 4 this)
! 		     (when (boundp symbol)
  		       (funcall quoter (default-value symbol))))
  	  ;; Don't complain about missing variables which are
  	  ;; irrelevant to this platform.
--- 312,318 ----
  	  ;; use the current value as the standard value.
  	  standard (if (nthcdr 4 this)
  		       (nth 4 this)
! 		     (when (default-boundp symbol)
  		       (funcall quoter (default-value symbol))))
  	  ;; Don't complain about missing variables which are
  	  ;; irrelevant to this platform.

Re: Current CVS doesn't bootstrap

[Lute Kamstra]

Richard Stallman <rms@gnu.org> writes:

> Shouldn't it use default-boundp?
>
> *** cus-start.el	15 Feb 2005 01:22:16 -0500	1.67
> --- cus-start.el	15 Feb 2005 02:36:28 -0500	
> ***************
> *** 312,318 ****
>   	  ;; use the current value as the standard value.
>   	  standard (if (nthcdr 4 this)
>   		       (nth 4 this)
> ! 		     (when (boundp symbol)
>   		       (funcall quoter (default-value symbol))))
>   	  ;; Don't complain about missing variables which are
>   	  ;; irrelevant to this platform.
> --- 312,318 ----
>   	  ;; use the current value as the standard value.
>   	  standard (if (nthcdr 4 this)
>   		       (nth 4 this)
> ! 		     (when (default-boundp symbol)
>   		       (funcall quoter (default-value symbol))))
>   	  ;; Don't complain about missing variables which are
>   	  ;; irrelevant to this platform.

Before your change, the `(funcall quoter (default-value symbol))' was
within the `(if (boundp symbol) ...)' test below.  You took it outside
this test, which caused the problem of accessing an MS-DOS-specific
built-in variable on GNU/Linux.  So I figured I should test for
`(boundp symbol)' again.  `(default-boundp symbol)' does seem the
right test though.  Apparently, the built-in variable in `symbol' is
never local-only.

Lute.