bug#821: 23.0.60; bootstrap fails due to define-minor-mode in textmode/paragraphs.el

bug#821: 23.0.60; bootstrap fails due to define-minor-mode in textmode/paragraphs.el

[Teodor Zlatanov]

cvs up -A
make maintainer-clean
configure
make bootstrap (or just make)

... bootstrap creation messages ...

Loading textmodes/paragraphs.el (source)...
Attempt to autoload define-minor-mode while preparing to dump
make[1]: *** [emacs] Error 255
make[1]: Leaving directory `/home/tzz/source/emacs/src'
make: *** [src] Error 2

editing loadup.el to remove that file and textmodes/text-mode.el made
the bootstrap work, but then `use-hard-newlines' (provided by
textmodes/paragraphs.el) was required by add-log.el and the
compilation failed.

This was introduced between 2008-08-28 and 2008-08-29 10:00 CST (I
checked with cvs up -D 'yesterday' but didn't have the time to narrow
it further).

Ted

Re: bug#821: 23.0.60; bootstrap fails due to define-minor-mode in textmode/paragraphs.el

[Ted Zlatanov]

On Fri, 29 Aug 2008 10:07:30 -0500 Teodor Zlatanov <tzz@lifelogs.com> wrote: 

TZ> cvs up -A
TZ> make maintainer-clean
TZ> configure
TZ> make bootstrap (or just make)

TZ> ... bootstrap creation messages ...

TZ> Loading textmodes/paragraphs.el (source)...
TZ> Attempt to autoload define-minor-mode while preparing to dump
TZ> make[1]: *** [emacs] Error 255
TZ> make[1]: Leaving directory `/home/tzz/source/emacs/src'
TZ> make: *** [src] Error 2

TZ> editing loadup.el to remove that file and textmodes/text-mode.el made
TZ> the bootstrap work, but then `use-hard-newlines' (provided by
TZ> textmodes/paragraphs.el) was required by add-log.el and the
TZ> compilation failed.

TZ> This was introduced between 2008-08-28 and 2008-08-29 10:00 CST (I
TZ> checked with cvs up -D 'yesterday' but didn't have the time to narrow
TZ> it further).

Correction: this has been broken for at least 10 days, I didn't look
further back.  Perhaps it's a problem with my setup.  I just upgraded to
Ubuntu 8.04.

Ted

bug#821: 23.0.60; bootstrap fails due to define-minor-mode in textmode/paragraphs.el

[Glenn Morris]

tags 821 unreproducible
stop

Ted Zlatanov wrote:

> TZ> Loading textmodes/paragraphs.el (source)...
> TZ> Attempt to autoload define-minor-mode while preparing to dump
[...]
> Correction: this has been broken for at least 10 days, I didn't look
> further back.  Perhaps it's a problem with my setup.  I just upgraded to
> Ubuntu 8.04.

Sorry, but you get the dread `unreproducible' tag again. :(

Does this at least mean that bug #327 (the "no rule to make target"
thing) has gone away?

I don't know why you have these problems...

Processed: Re: bug#821: 23.0.60; bootstrap fails due to define-minor-mode in textmode/paragraphs.el

[Emacs bug Tracking System]

Processing commands for control@emacsbugs.donarmstrong.com:

> tags 821 unreproducible
bug#821: 23.0.60; bootstrap fails due to define-minor-mode in textmode/paragraphs.el
There were no tags set.
Tags added: unreproducible

> stop
Stopping processing here.

Please contact me if you need assistance.

Don Armstrong
(administrator, Emacs bugs database)

Re: bug#821: 23.0.60; bootstrap fails due to define-minor-mode in textmode/paragraphs.el

[Ted Zlatanov]

On Fri, 29 Aug 2008 19:08:05 -0400 Glenn Morris <rgm@gnu.org> wrote: 

GM> tags 821 unreproducible
GM> stop

GM> Ted Zlatanov wrote:

TZ> Loading textmodes/paragraphs.el (source)...
TZ> Attempt to autoload define-minor-mode while preparing to dump
GM> [...]
>> Correction: this has been broken for at least 10 days, I didn't look
>> further back.  Perhaps it's a problem with my setup.  I just upgraded to
>> Ubuntu 8.04.

GM> Sorry, but you get the dread `unreproducible' tag again. :(
GM> I don't know why you have these problems...

How frustrating.  I have not done anything unusual on my system, yet
keep having these compilation problems.  It may be one of the packages
I've installed interfering with the build process.  I'll think about a
full reinstall, but how bizarre that it should come to that.

GM> Does this at least mean that bug #327 (the "no rule to make target"
GM> thing) has gone away?

For a while (until this bug) I was editing ELCFILES and removing the
contents; the .el files would get compiled with a warning that they were
not in ELCFILES but otherwise everything worked.  The latest bug is not
something I can work around, unfortunately, so I'll have to dig deeper
into the system breakage that's causing the problem.

Thanks for your attention...
Ted

bug#821: 23.0.60; bootstrap fails due to define-minor-mode in textmode/paragraphs.el

[Glenn Morris]

Ted Zlatanov wrote:

> TZ> Loading textmodes/paragraphs.el (source)...
> TZ> Attempt to autoload define-minor-mode while preparing to dump

Your specific problem is that textmodes/paragraphs.el has not been
byte-compiled as it should have been. Can you look into why?

> How frustrating.  I have not done anything unusual on my system, yet
> keep having these compilation problems.  It may be one of the packages
> I've installed interfering with the build process.  I'll think about a
> full reinstall, but how bizarre that it should come to that.

Have you tried a fresh cvs checkout? That should not be necessary
either of course, but is less drastic. :(

> GM> Does this at least mean that bug #327 (the "no rule to make target"
> GM> thing) has gone away?
>
> For a while (until this bug) I was editing ELCFILES and removing the
> contents; the .el files would get compiled with a warning that they were
> not in ELCFILES but otherwise everything worked.  The latest bug is not
> something I can work around, unfortunately, so I'll have to dig deeper
> into the system breakage that's causing the problem.

Re: bug#821: 23.0.60; bootstrap fails due to define-minor-mode in textmode/paragraphs.el

[Ted Zlatanov]

On Wed, 03 Sep 2008 14:04:00 -0400 Glenn Morris <rgm@gnu.org> wrote: 

GM> Ted Zlatanov wrote:
TZ> Loading textmodes/paragraphs.el (source)...
TZ> Attempt to autoload define-minor-mode while preparing to dump

GM> Your specific problem is that textmodes/paragraphs.el has not been
GM> byte-compiled as it should have been. Can you look into why?

I set up a new user with a clean environment, using bash:

SHELL=/bin/bash
TERM=xterm
USER=lank
MAIL=/var/mail/lank
PATH=/usr/local/bin:/usr/bin:/bin:/usr/games
PWD=/home/lank/emacs/lisp
LANG=en_US.UTF-8
HISTCONTROL=ignoreboth
SHLVL=1
HOME=/home/lank
LOGNAME=lank
LESSOPEN=| /usr/bin/lesspipe %s
LESSCLOSE=/usr/bin/lesspipe %s %s
OLDPWD=/home/lank/emacs
_=/usr/bin/env

Uninstalled and reinstalled many Debian packages:

Removed: apel bbdb bzr-builddeb caspar cvs-autoreleasedeb cvs-buildpackage debhelper devscripts dpkg-dev eclipse-cdt elib elscreen erc git-buildpackage linda lintian maemo-sdk mailcrypt make oneliner-el oo-browser rtai rtai-source
Removed: g++ gcc gcc-multilib ghc6 hmake
Removed: cedet-common cedet-contrib cogre crypt++el css-mode ecb ede eieio emacs emacs-goodies-el emacs-snapshot-bin-common emacs-snapshot-common emacs-snapshot-el emacs21 emacs21-bin-common emacs21-common emacs22 emacs22-bin-common emacs22-common emacs22-el emacs22-gtk emacsen-common erlang-mode gnuserv haskell-mode html-helper-mode lua-mode muse-el nethack-el nxml-mode planner-el remember-el ruby-elisp ruby1.8-elisp semantic slime speedbar tramp w3m-el x-face-el xemacs21 xemacs21-bin xemacs21-mule xemacs21-support xemacs21-supportel
Removed: autoconf automake automake1.9 autoproject

Installed: make 3.81-3build1
Installed: g++/gcc 4:4.2.3-1ubuntu6
Installed: autoconf 2.61-4, automake 1:1.10.1-2

Now, I did a CVS checkout of Emacs to an empty directory.

`./configure' works.
`make':

...
Dumping under the name emacs
73478 pure bytes used
mv -f emacs bootstrap-emacs
cd ../lisp; make -w compile-first EMACS=../src/bootstrap-emacs
make[2]: Entering directory `/home/lank/emacs/lisp'
make[2]: *** No rule to make target `/home/lank/emacs/lisp/emacs-lisp/bytecomp.elc', needed by `compile-first'.  Stop.
make[2]: Leaving directory `/home/lank/emacs/lisp'
make[1]: *** [bootstrap-emacs] Error 2
make[1]: Leaving directory `/home/lank/emacs/src'
make: *** [src] Error 2

If I do another `make' at this point it goes on to the next problem
(textmode autoloads).  I don't think the bytecomp breakage was happening
before.

I wanted to investigate this one as it seemed related to the byte-compile
issues.  This file (bytecomp.el) is mentioned in COMPILE_FIRST and in
ELCFILES; this may bring us back to why ELCFILES is not picked up
correctly.  So:

I ran this manually under lisp/ (discovering which files were needed by
`make -w compile-first'):

../src/bootstrap-emacs --batch --eval '(byte-compile-file "emacs-lisp/bytecomp.el")'
../src/bootstrap-emacs --batch --eval '(byte-compile-file "emacs-lisp/byte-opt.el")'
../src/bootstrap-emacs --batch --eval '(byte-compile-file "emacs-lisp/autoload.el")'

Then `make -w compile-first' did not complain, and ran cleanly.  Now I
went back to the base emacs directory and did another `make'.

The bootstrap completed:

...
Dumping under the name emacs
1246848 pure bytes used
Adding name emacs-23.0.60.1
ln -f emacs bootstrap-emacs
...

But then again ELCFILES broke things:

...
Compiling /home/lank/emacs/lisp/bs.el
Wrote /home/lank/emacs/lisp/bs.elc
make[1]: *** No rule to make target `/home/lank/emacs/lisp/calc/calc-aent.elc', needed by `compile-main'.  Stop.
make[1]: Leaving directory `/home/lank/emacs/lisp'
make: *** [lisp] Error 2

This is fixed by clearing ELCFILES manually.  I get many warnings like
this:

Maintainer warning: $(lisp)/epa-dired.el missing from $ELCFILES?
Compiling /home/lank/emacs/lisp/epa-dired.el

for every ex-ELCFILES file.

I saw one more failure:

...
make[1]: *** No rule to make target `/home/tzz/source/emacs/leim/ja-dic/ja-dic.elc', needed by `all'.  Stop.
make[1]: Leaving directory `/home/tzz/source/emacs/leim'
make: *** [leim] Error 2

but running `make' again gets it done and the build can proceed to a
semi-glorious resolution.

The root of all these problems seems to be Make itself.  Unfortunately,
my questions to the Make mailing lists have not been answered by any of
the developers or users.

I may still reinstall the system for unrelated reasons, but I'd like to
find out what's really going on.  It's unpleasant to fight against the
build process so hard.

Ted

bug#821: 23.0.60; bootstrap fails due to define-minor-mode in textmode/paragraphs.el

[Glenn Morris]

Ted Zlatanov wrote:

> Installed: make 3.81-3build1
[...]
> cd ../lisp; make -w compile-first EMACS=../src/bootstrap-emacs
> make[2]: Entering directory `/home/lank/emacs/lisp'
> make[2]: *** No rule to make target `/home/lank/emacs/lisp/emacs-lisp/bytecomp.elc', needed by `compile-first'.  Stop.

This is bug #327 again, in that make for some reason does not
understand the implicit rule that generates .elc from .el. This time
it is failing straight away, rather than partway through the .elc
files.

> I wanted to investigate this one as it seemed related to the byte-compile
> issues.  This file (bytecomp.el) is mentioned in COMPILE_FIRST and in
> ELCFILES; this may bring us back to why ELCFILES is not picked up
> correctly.

Good point. One thing to try is deleting bytecomp, byte-opt, and
autoload from ELCFILES. I don't hold out much hope though...

> Compiling /home/lank/emacs/lisp/bs.el
> Wrote /home/lank/emacs/lisp/bs.elc
> make[1]: *** No rule to make target `/home/lank/emacs/lisp/calc/calc-aent.elc', needed by `compile-main'.  Stop.
> make[1]: Leaving directory `/home/lank/emacs/lisp'
> make: *** [lisp] Error 2

Again, a failure of make to understand an implicit rule.

> The root of all these problems seems to be Make itself.

Yes. Perhaps try compiling your own GNU make from source?

Re: bug#821: 23.0.60; bootstrap fails due to define-minor-mode in textmode/paragraphs.el

[Ted Zlatanov]

On Wed, 03 Sep 2008 22:45:02 -0400 Glenn Morris <rgm@gnu.org> wrote: 

GM> Ted Zlatanov wrote:
>> Installed: make 3.81-3build1
GM> [...]
>> cd ../lisp; make -w compile-first EMACS=../src/bootstrap-emacs
>> make[2]: Entering directory `/home/lank/emacs/lisp'
>> make[2]: *** No rule to make target `/home/lank/emacs/lisp/emacs-lisp/bytecomp.elc', needed by `compile-first'.  Stop.

GM> This is bug #327 again, in that make for some reason does not
GM> understand the implicit rule that generates .elc from .el. This time
GM> it is failing straight away, rather than partway through the .elc
GM> files.

>> I wanted to investigate this one as it seemed related to the byte-compile
>> issues.  This file (bytecomp.el) is mentioned in COMPILE_FIRST and in
>> ELCFILES; this may bring us back to why ELCFILES is not picked up
>> correctly.

GM> Good point. One thing to try is deleting bytecomp, byte-opt, and
GM> autoload from ELCFILES. I don't hold out much hope though...

>> Compiling /home/lank/emacs/lisp/bs.el
>> Wrote /home/lank/emacs/lisp/bs.elc
>> make[1]: *** No rule to make target `/home/lank/emacs/lisp/calc/calc-aent.elc', needed by `compile-main'.  Stop.
>> make[1]: Leaving directory `/home/lank/emacs/lisp'
>> make: *** [lisp] Error 2

GM> Again, a failure of make to understand an implicit rule.

>> The root of all these problems seems to be Make itself.

GM> Yes. Perhaps try compiling your own GNU make from source?

Amazing, that worked.  I uninstalled the Ubuntu make (3.81-3build1) and
installed make from source (3.81 as well).  The build process worked
perfectly (`make maintainer-clean; ./configure; make; make install').

It's unfortunate that make is the problem here, but if no one else has
experienced the bug it's really hard to track the problem, especially
since it's somewhere in a binary package.  I'll mention it on the make
dev list but I don't hold out too much home.

Anyhow, you can close bugs 921 and 327.  Thank you for your patience.

Ted

bug#821: 23.0.60; bootstrap fails due to define-minor-mode in textmode/paragraphs.el

[Glenn Morris]

Ted Zlatanov wrote:

> Amazing, that worked.  I uninstalled the Ubuntu make (3.81-3build1) and
> installed make from source (3.81 as well).  The build process worked
> perfectly (`make maintainer-clean; ./configure; make; make install').

Yay!

I wonder if anyone else has this problem on Ubuntu 8.04 with make
3.81-3build1? You might try filing an Ubuntu bug, if you're not
thoroughly sick of all this.

I know little about make, but it looks as if we might be overflowing
some internal make limit due to the size of ELCFILES.