Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Thorbjørn Ravn Andersen]

Tried to play with the latest Emacs for OS X as the one supplied from 
Apple has short comings.

I could not build it, and apparently the stock emacs 21.1.1 in OS X does 
not know some of the new things needed to build.

The problem is that emacs depends on {etc}DOC (or so), which in turn 
depends on the compiled lisp files, which in turn depends on emacs to 
compile them.

Is there a quick work around to get emacs up and running?

-- 
  Thorbjørn Ravn Andersen      Scandiatransplant
                               Skejby Sygehus, indgang 3
  +45 89 49 53 01              DK-8200 Århus N
  http://biobase.dk/~tra         

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Thorbjørn Ravn Andersen]

Thorbjørn Ravn Andersen wrote:

> Is there a quick work around to get emacs up and running?

My apologies.  Apparently I did not read the Install document well 
enough - it appears that I should have run "make bootstrap" instead of 
"make".  Of course this popped up before me about 15 seconds after 
sending the mail.

-- 
  Thorbjørn Ravn Andersen      Scandiatransplant
                               Skejby Sygehus, indgang 3
  +45 89 49 53 01              DK-8200 Århus N
  http://biobase.dk/~tra         

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Kim F. Storm]

Thorbjørn Ravn Andersen <tra@biobase.dk> writes:

> Thorbjørn Ravn Andersen wrote:
> 
> > Is there a quick work around to get emacs up and running?
> 
> My apologies.  Apparently I did not read the Install document well
> enough - it appears that I should have run "make bootstrap" instead of
> "make".  Of course this popped up before me about 15 seconds after
> sending the mail.

This question pops up time and again!

Maybe we could add a test to the makefile, so that doing 'make' if
e.g. lisp/files.elc does not exist, it does 'cat INSTALL_CVS'
and exits?

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Thorbjørn Ravn Andersen]

Kim F. Storm wrote:

>This question pops up time and again!
>
>Maybe we could add a test to the makefile, so that doing 'make' if
>e.g. lisp/files.elc does not exist, it does 'cat INSTALL_CVS'
>and exits?
>
I didn't find this document.  I got around by touching "etc/DOC" and 
then building, which gave me a slightly crippled Emacs.  

Why does the main "make" not compile the lisp files?  Can't this be done 
with temacs?

-- 
  Thorbjørn Ravn Andersen      Scandiatransplant
                               Skejby Sygehus, indgang 3
  +45 89 49 53 01              DK-8200 Århus N
  http://biobase.dk/~tra         

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Eli Zaretskii]

On Thu, 31 Oct 2002, =?ISO-8859-1?Q?Thorbj=F8rn_Ravn_Andersen?= wrote:

> Kim F. Storm wrote:
> 
> >This question pops up time and again!
> >
> >Maybe we could add a test to the makefile, so that doing 'make' if
> >e.g. lisp/files.elc does not exist, it does 'cat INSTALL_CVS'
> >and exits?
> >
> I didn't find this document.

The correct file name is INSTALL-CVS.  Do you see that file?  If not, 
please fetch it from the CVS.

> Why does the main "make" not compile the lisp files?

It does, if you say "make bootstrap".  Plain "make" assumes the *.elc 
files are already there.

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Thorbjørn Ravn Andersen]

[-- Attachment #1: Type: text/plain, Size: 934 bytes --]

Eli Zaretskii wrote:

>>I didn't find this document
>>
>
>The correct file name is INSTALL-CVS.  Do you see that file?  If not, 
>please fetch it from the CVS.
>  
>
I scanned the INSTALL document, and the README and INSTALL files in mac/

The INSTALL-CVS file was not mentioned in either of these.

>>Why does the main "make" not compile the lisp files?
>>    
>>
>
>It does, if you say "make bootstrap".  Plain "make" assumes the *.elc 
>files are already there.
>  
>
I understood that.  I still do not understand why the normal "make" 
command in the "./compile; make; make install" cycle does not do this?

Well, so much for being a neophyte in the intricates of emacs 
development.  I would, however, like to second Kim's suggestion.


-- 
  Thorbjørn Ravn Andersen      Scandiatransplant
                               Skejby Sygehus, indgang 3
  +45 89 49 53 01              DK-8200 Århus N
  http://biobase.dk/~tra         


[-- Attachment #2: Type: text/html, Size: 1656 bytes --]

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Andreas Schwab]

Thorbjørn Ravn Andersen <tra@biobase.dk> writes:

|> I understood that.  I still do not understand why the normal "make"
|> command in the "./compile; make; make install" cycle does not do this?

Because compiling from CVS is different than compiling from a distribution
tar file.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux AG, Deutschherrnstr. 15-19, D-90429 Nürnberg
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Thorbjørn Ravn Andersen]

Andreas Schwab wrote:

>Thorbjørn Ravn Andersen <tra@biobase.dk> writes:
>
>|> I understood that.  I still do not understand why the normal "make"
>|> command in the "./compile; make; make install" cycle does not do this?
>
>Because compiling from CVS is different than compiling from a distribution
>tar file.
>  
>
In what way?

Most, if not all, of those projects I have retrieved from CVS I have 
been able to use the "./configure; make; make install" incantation, so - 
recognizing that Emacs is a large and complex program with different 
needs to most other programs - I would still like to know what the 
reason for this is.

Is this question so ignorant that I should go hiding under a rock after 
reading some INSTALL-CVS-FAQ-READ-ME-FIRST file, I have missed (if I 
missed one file, I can miss even more), I will be happy to do so, but I 
would appreciate a gentle nudge mentioning the name of this file.  The 
INSTALL-CVS file did not indicate the existance of such a file.

Thanks in advance.

-- 
  Thorbjørn Ravn Andersen      Scandia transplant
                               Skejby Sygehus, indgang 3
  +45 89 49 53 01              DK-8200 Århus N
  http://biobase.dk/~tra         

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Stefan Monnier]

> Most, if not all, of those projects I have retrieved from CVS I have
> been able to use the "./configure; make; make install" incantation, so -
> recognizing that Emacs is a large and complex program with different
> needs to most other programs - I would still like to know what the
> reason for this is.

It can be done, but Emacs' Makefile has not been worked on sufficiently
to allow it to both work and be fast: checking whether bootstrap
is needed or not can be costly if you want to do it right (it's OK,
tho, since we can just be conservative as we do now) but more importantly
the best way to bootstrap is not always the same and since bootstrapping
can take a while...
The procedure has already been streamlined a little, but it still needs
more work.  The reason why it hasn't been done earlier is that .elc
files used to be stored in CVS (and in RCS before that) making bootstrapping
mostly unnecessary.  And also because once you've bootstrapped you almost
don't need to bootstrap ever again: I've had to bootstrap maybe 3 times
last year: not much incentive to improve it.

What I'm wondering is: when you tried to build Emacs, it should
have told you "use make bootstrap" or something like that.
Did it ?  Or did you simply not pay attention ?


	Stefan

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Steven Tamm]

For the make-package script in the Mac OS X, it will run make bootstrap 
if lisp/abbrev.elc is missing.
What would be the argument(s) against adding to the root Makefile a 
dependency on a specific .elc file that would force "make bootstrap" if 
starting from scratch?  It is not "doing it right", but it seems to 
work.

-Steven

On Thursday, October 31, 2002, at 09:01  AM, Stefan Monnier wrote:

>> Most, if not all, of those projects I have retrieved from CVS I have
>> been able to use the "./configure; make; make install" incantation, 
>> so -
>> recognizing that Emacs is a large and complex program with different
>> needs to most other programs - I would still like to know what the
>> reason for this is.
>
> It can be done, but Emacs' Makefile has not been worked on sufficiently
> to allow it to both work and be fast: checking whether bootstrap
> is needed or not can be costly if you want to do it right (it's OK,
> tho, since we can just be conservative as we do now) but more 
> importantly
> the best way to bootstrap is not always the same and since 
> bootstrapping
> can take a while...

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Stefan Monnier]

> For the make-package script in the Mac OS X, it will run make bootstrap 
> if lisp/abbrev.elc is missing.
> What would be the argument(s) against adding to the root Makefile a 
> dependency on a specific .elc file that would force "make bootstrap" if 
> starting from scratch?  It is not "doing it right", but it seems to 
> work.

The argument against it (for me anyway) is that the bootstrap
target in the Makefile does a pretty thorough `clean' before doing
its job: I don't want to have to rebuild each and every elc file just
because I happen to have deleted abbrev.elc.

But I guess this begs the question: why does the bootstrap target
do such a thorough `clean' at all ?


	Stefan

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Andreas Schwab]

"Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> writes:

|> But I guess this begs the question: why does the bootstrap target
|> do such a thorough `clean' at all ?

You need to make sure there are no old elc containing out-of-date macro
implementations around.  Otherwise you cannot guarantee consistency.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux AG, Deutschherrnstr. 15-19, D-90429 Nürnberg
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Stefan Monnier]

> |> But I guess this begs the question: why does the bootstrap target
> |> do such a thorough `clean' at all ?
> 
> You need to make sure there are no old elc containing out-of-date macro
> implementations around.  Otherwise you cannot guarantee consistency.

Does anybody care ?
As I said, I almost never bootstrap and I'm pretty sure I still have .elc
files compiled with very old versions of define-derived-mode or
define-minor-mode (the two macros that I know have changed), but it doesn't
create any problem.  I understand that you want such consistency when
building the tarball for distribution or when doing a daily "bootstrap
regression test", but in those cases you can always do
`make bootstrap-clean bootstrap'.


	Stefan

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Juanma Barranquero]

On Sat, 02 Nov 2002 21:24:47 -0500
"Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> wrote:

> Does anybody care ?
> As I said, I almost never bootstrap

Well, I bootstrap daily. Some days more than once.

Mostly as a test, to check that it still bootstraps. It´s not unheard of
that a change in the sources breaks bootstrapping *precisely* because
bootstrapping, being almost unnecesary, is often not checked after a
change.

-- 
Juanma Barranquero <lektu@terra.es>

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Kai Großjohann]

Juanma Barranquero <lektu@terra.es> writes:

> Well, I bootstrap daily. Some days more than once.

FWIW, I also like the clean feeling that bootstrapping gives me.
Then I know that all the object code is fresh :-)

kai
-- 
~/.signature is: umop ap!sdn    (Frank Nobis)

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Stefan Monnier]

> 
> On Sat, 02 Nov 2002 21:24:47 -0500
> "Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> wrote:
> 
> > Does anybody care ?
> > As I said, I almost never bootstrap
> 
> Well, I bootstrap daily. Some days more than once.
> 
> Mostly as a test, to check that it still bootstraps. It´s not unheard of
> that a change in the sources breaks bootstrapping *precisely* because
> bootstrapping, being almost unnecesary, is often not checked after a
> change.

The question was: do you care whether you have to say

	make bootstrap
or
	make clean bootstrap

when you want to do your consistency-check bootstrap ?


	Stefan

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Andreas Schwab]

"Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> writes:

|> The question was: do you care whether you have to say
|> 
|> 	make bootstrap
|> or
|> 	make clean bootstrap
|> 
|> when you want to do your consistency-check bootstrap ?

make clean does not rm *.elc, only bootstrap-clean does.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux AG, Deutschherrnstr. 15-19, D-90429 Nürnberg
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Stefan Monnier]

> "Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> writes:
> 
> |> The question was: do you care whether you have to say
> |> 
> |> 	make bootstrap
> |> or
> |> 	make clean bootstrap
> |> 
> |> when you want to do your consistency-check bootstrap ?
> 
> make clean does not rm *.elc, only bootstrap-clean does.

I wouldn't know, I use cvsclean instead.


	Stefan


#!/usr/bin/perl

sub cvsclean {
    my($path) = @_;
    my(%files) = ();
    my(@subdirs);

    print STDOUT "Cleaning $path\n";
    opendir (DIR, "$path/") || die "No directory $path";
    open (ENTRIES, "$path/CVS/Entries") || die "No $path/CVS/Entries file";
    while (<ENTRIES>) {
        if (m[^D/([^/]+)]) {
            push (@subdirs, "$path/$1");
        } elsif (m[^/([^/]+)/[^/-]]) {
            $files{$1} = "managed";
        }
    }

    foreach $entry (readdir(DIR)) {
        if (!exists ($files{$entry})) {
            $entry = "$path/$entry";
            if (-f $entry) {
                print STDOUT "unlink $entry\n";
                unlink $entry;
            }
        }
    }
    
    foreach $subdir (@subdirs) {
        &cvsclean ($subdir);
    }
}

&cvsclean (".");

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Juanma Barranquero]

On Mon, 04 Nov 2002 09:53:51 -0500, "Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> wrote:

> The question was: do you care whether you have to say
> 
> 	make bootstrap
> or
> 	make clean bootstrap
> 
> when you want to do your consistency-check bootstrap ?

No, of course I don't care as long as it's clearly documented somewhere
:)



                                                           /L/e/k/t/u

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Stefan Monnier]

> > The question was: do you care whether you have to say
> > 
> > 	make bootstrap
> > or
> > 	make clean bootstrap
> > 
> > when you want to do your consistency-check bootstrap ?
> 
> No, of course I don't care as long as it's clearly documented somewhere

It should probably be documented at the same place as the fact
that after modifying lisp.h `make' will not rebuild all the .o files
that depend on lisp.h and that you might thus need a `make clean' as
well.


	Stefan

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Juanma Barranquero]

On Mon, 04 Nov 2002 10:43:04 -0500, "Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> wrote:

> It should probably be documented at the same place as the fact
> that after modifying lisp.h `make' will not rebuild all the .o files
> that depend on lisp.h and that you might thus need a `make clean' as
> well.

That'd be INSTALL and/or INSTALL-CVS, for starters.

                                                           /L/e/k/t/u

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Richard Stallman]

    For the make-package script in the Mac OS X, it will run make bootstrap 
    if lisp/abbrev.elc is missing.

This sounds like what emacs/Makefile.in used to do.
We have changed it since.  Could you update your script?

    What would be the argument(s) against adding to the root Makefile a 
    dependency on a specific .elc file that would force "make bootstrap" if 
    starting from scratch?

It is a bad idea to do a boostrap if the user did not explicitly ask
for one.

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Thorbjørn Ravn Andersen]

Stefan Monnier wrote:

>What I'm wondering is: when you tried to build Emacs, it should
>have told you "use make bootstrap" or something like that.
>Did it ?  Or did you simply not pay attention ?
>  
>
Thank you for your explanation.

The mac/INSTALL file says

> You should be able to build Emacs on Mac OS X by typing the following
> commands at the top-level directory after the source distribution is
> un-tarred.
>
>   ./configure
>   make
>   make install
>
> The last step must be performed as root.


The "make bootstrap" is mentioned in passing a bit further down.

I probably should not have thought that Emacs was like everything else, 
so I should have read all the various documents more carefully before 
plunging in the mindless "configure; make" cycle.  Credits for the GNU 
organization for making it so - in the "good" old days, you had to read 
everything and adapt Makefiles carefullly before even thinking of typing 
make.

I'll get back to testing the Aqua Emacs and keeping an eye on this list.

-- 
  Thorbjørn Ravn Andersen      Scandiatransplant
                               Skejby Sygehus, indgang 3
  +45 89 49 53 01              DK-8200 Århus N
  http://biobase.dk/~tra         

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Stefan Monnier]

> > You should be able to build Emacs on Mac OS X by typing the following
> > commands at the top-level directory after the source distribution is
> > un-tarred.
> >
> >   ./configure
> >   make
> >   make install
> >
> > The last step must be performed as root.

No, I wasn't talking about any INSTALL file:
When you ran `make', you should have gotten a message saying
"use make bootstrap".  Did you see that ?


	Stefan

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Thorbjørn Ravn Andersen]

Stefan Monnier wrote:

>No, I wasn't talking about any INSTALL file:
>When you ran `make', you should have gotten a message saying
>"use make bootstrap".  Did you see that ?
>  
>
No.

I checked the "maybe_bootstrap" target, and the code is:

maybe_bootstrap:
        @bar=`echo ./lisp/*.elc`; \
        if [ "$bar" = './lisp/*.elc' ]; then \
          echo "Your tree does not include the compiled Lisp files."; \
          echo "You need to do \`make bootstrap' to build Emacs."; \
          echo "Emacs now requires Texinfo version 4.2."; \
          exit 1;\
        fi

I am not _that_ familiar with shell code, but does "@bar" and "$bar" 
refer to the same variable?

-- 
  Thorbjørn Ravn Andersen      Scandiatransplant
                               Skejby Sygehus, indgang 3
  +45 89 49 53 01              DK-8200 Århus N
  http://biobase.dk/~tra         

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Alfred M. Szmidt]

   I am not _that_ familiar with shell code, but does "@bar" and "$bar" 
   refer to the same variable?

The @ supresses output in make.  So yes.

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Thorbjørn Ravn Andersen]

Alfred M. Szmidt wrote:

>   I am not _that_ familiar with shell code, but does "@bar" and "$bar" 
>   refer to the same variable?
>
>The @ supresses output in make.  So yes.
>
Ok.

This code works under OS X:

maybe_bootstrap:
        @bar="`echo ./lisp/*.elc`"; \
        if [ \( "$(bar)" = './lisp/*.elc' \) -o \( "$(bar)" = '' \) ]; 
then \
          echo "Your tree does not include the compiled Lisp files."; \
          echo "You need to do \`make bootstrap' to build Emacs."; \
          echo "Emacs now requires Texinfo version 4.2."; \
          exit 1;\
        fi

There was also a TAB missing.  I do not know if these still are mandatory.

-- 
  Thorbjørn Ravn Andersen      Scandiatransplant
                               Skejby Sygehus, indgang 3
  +45 89 49 53 01              DK-8200 Århus N
  http://biobase.dk/~tra         

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Richard Stallman]

    maybe_bootstrap:
	    @bar="`echo ./lisp/*.elc`"; \
	    if [ \( "$(bar)" = './lisp/*.elc' \) -o \( "$(bar)" = '' \) ]; then \

It looks like that should work in GNU and Unix too, so we may as well
make that change.

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Andreas Schwab]

Thorbjørn Ravn Andersen <tra@biobase.dk> writes:

|> Most, if not all, of those projects I have retrieved from CVS I have been
|> able to use the "./configure; make; make install" incantation

Many projects require you to run something like autogen.sh before you can
build from CVS.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux AG, Deutschherrnstr. 15-19, D-90429 Nürnberg
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Richard Stallman]

    Many projects require you to run something like autogen.sh before you can
    build from CVS.

Is that a semi-standard convention?  If so, maybe Emacs should follow
the convention, maybe we should add that script.

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Karl Fogel]

Richard Stallman <rms@gnu.org> writes:
>     Many projects require you to run something like autogen.sh before you can
>     build from CVS.
> 
> Is that a semi-standard convention?  If so, maybe Emacs should follow
> the convention, maybe we should add that script.

It seems to be at least semi-standard now, yes.

-Karl

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Richard Stallman]

    >     Many projects require you to run something like autogen.sh before you can
    >     build from CVS.
    > 
    > Is that a semi-standard convention?  If so, maybe Emacs should follow
    > the convention, maybe we should add that script.

    It seems to be at least semi-standard now, yes.

Can you write such a script for Emacs?

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Karl Fogel]

Richard Stallman <rms@gnu.org> writes:
>     > Is that a semi-standard convention?  If so, maybe Emacs should follow
>     > the convention, maybe we should add that script.
> 
>     It seems to be at least semi-standard now, yes.
> 
> Can you write such a script for Emacs?

Okay, I'll look into doing so.  I'm not familiar with the exact
mechanism of 'make bootstrap', so will need about a week.

-K

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Richard Stallman]

    Okay, I'll look into doing so.  I'm not familiar with the exact
    mechanism of 'make bootstrap', so will need about a week.

Thanks.

I don't think you need to change the mechanism of `make bootstrap' much.
What the script has to do is cause `make' to run `make bootstrap'
unconditionally, the next time only.

Right now there is a target `maybe_bootstrap' that checks whether
a bootstrap is needed; but it doesn't actually do a bootstrap,
rather it exits with instructions to do `make bootstrap'.

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Karl Fogel]

Richard Stallman <rms@gnu.org> writes:
>     >     Many projects require you to run something like autogen.sh
>     >     before you can build from CVS.
>     > 
>     > Is that a semi-standard convention?  If so, maybe Emacs should follow
>     > the convention, maybe we should add that script.
> 
>     It seems to be at least semi-standard now, yes.
> 
> Can you write such a script for Emacs?
>
>     Okay, I'll look into doing so.  I'm not familiar with the exact
>    mechanism of 'make bootstrap', so will need about a week.

Okay, the patch is at the end of this mail.  I've also read over this
whole thread, and realized that people are probably talking from
different assumptions.  Please bear with me while I try to sort some
things out:

It is normal these days for a CVS working copy of a project to require
one extra step in its build procedure:

   $ ./autogen.sh        <=== this is the extra step
   $ ./configure
   $ ./make
   $ ./make install

In other words, the purpose of autogen.sh is to get your working copy
to the "same" state as a distribution tarball, at least as far as the
build procedure is concerned.

In most projects, the autogen.sh script exists because the project
does not keep generated files (such as configure, Makefile, etc) in
the repository.  Instead, the repository holds only `configure.in',
`Makefile.in', and so on.  When you check out the project, you do not
get the derived files.  That's why you have to run autogen.sh -- to
get `./configure', so you can run that to get `Makefile', so you can
run 'make'.  Thus, at its most basic, autogen.sh looks like this:

   #!/bin/sh
   autoconf

Of course, in real life, most autogen.sh scripts are considerably more
complicated.  (For example, they often check the version of autoconf,
and exit telling you to upgrade if the version is too low.)
   
Anyway, the Emacs repository does not follow the convention of not
storing generated files.  It stores `configure', and what's more, the
bootstrap process is 'make bootstrap', which depends on `Makefile'
having been generated.  (And for a final twist, 'make bootstrap'
actually builds the whole system, obviating the need to run 'make'
afterwards.)

Because of all this, the traditional autogen.sh strategy doesn't
really work for Emacs, at least not without a larger change to the way
Emacs is versioned.  So what I have done is implemented autogen.sh in
roughly the way that RMS suggested:

> I don't think you need to change the mechanism of `make bootstrap' much.
> What the script has to do is cause `make' to run `make bootstrap'
> unconditionally, the next time only.
> 
> Right now there is a target `maybe_bootstrap' that checks whether
> a bootstrap is needed; but it doesn't actually do a bootstrap,
> rather it exits with instructions to do `make bootstrap'.

However, the result is a very unusual autogen.sh.  It will probably
confuse people who thought (based on other projects) that they
understood what autogen.sh scripts were all about.

For a long term goal, I think it would be best if we separated out two
distinct sets of requirements:

   1. The ways in which bootstrapping Emacs from a CVS working copy is
      the same as bootstrapping most other projects (e.g., the need
      to get a `configure' from `configure.in')

   2. The ways in which bootstrapping Emacs from a CVS working copy is
      special (e.g., the need to compile .elc files, and to reload and
      redump Emacs after a new build)

Everything in category (1) should be handled by autogen.sh, and we
would stop versioning the `configure' file.  Category (2) would be
handled by a separate script, or by good advice printed by autogen.sh
at the end of its run.

I'm not volunteering for this; I just want to see whether people think
it's a good direction to go.

As Eli Zaretskii said recently:

   > Please remember that anon CVS access to Emacs development sources
   > is relatively new.

Mmm, yes :-).

Here's the patch, not applying until get some feedback:

2002-11-19  Karl Fogel  <kfogel@red-bean.com>

	* autogen.sh: New file.

	* INSTALL-CVS: Adjust for presence of autogen.sh.

Index: autogen.sh
===================================================================
RCS file: autogen.sh
diff -N autogen.sh
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ autogen.sh	20 Nov 2002 05:41:37 -0000
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+if make maybe_bootstrap > /dev/null 2>&1; then
+  ./configure
+  echo ""
+  echo "Your working copy is now ready for you to run 'make'."
+  echo "If you want to be extra thorough, then run:"
+  echo ""
+  echo "  make"
+  echo "  cd lisp"
+  echo "  make recompile EMACS=../src/emacs"
+  echo "  cd .."
+  echo "  make"
+  echo "  make install     ### optional ###"
+  echo ""
+else
+  echo "$0: Bootstrapping..."
+  ./configure
+  make bootstrap
+  echo "$0: Done bootstrapping."
+fi
Index: INSTALL-CVS
===================================================================
RCS file: /cvsroot/emacs/emacs/INSTALL-CVS,v
retrieving revision 1.1
diff -u -r1.1 INSTALL-CVS
--- INSTALL-CVS	17 Jun 2002 06:07:24 -0000	1.1
+++ INSTALL-CVS	20 Nov 2002 05:41:37 -0000
@@ -2,42 +2,31 @@
 
 Some of the files that are included in the Emacs tarball, such as
 byte-compiled Lisp files, are not stored in the CVS repository.
-Therefore, to build from CVS you must run "make bootstrap"
-instead of just "make":
+Therefore, to build from a CVS working copy, you should first run
 
-  $ ./configure
-  $ make bootstrap
+   ./autogen.sh
 
-The bootstrap process makes sure all necessary files are rebuilt
-before it builds the final Emacs binary.
+then run the appropriate 'make' commands afterwards, if necessary (the
+output from autogen.sh will give you some guidance).
 
-Normally, it is not necessary to use "make bootstrap" after every CVS
-update.  Unless there are problems, we suggest the following
-procedure:
-
-  $ ./configure
-  $ make
-  $ cd lisp
-  $ make recompile EMACS=../src/emacs
-  $ cd ..
-  $ make
-
-(If you want to install the Emacs binary, type "make install" instead
-of "make" in the last command.)
-
-If the above procedure fails, try "make bootstrap".
-
-Users of non-Posix systems (MS-Windows etc.) should run the
-platform-specific configuration scripts (nt/configure.bat, config.bat,
-etc.) before "make bootstrap" or "make"; the rest of the procedure is
+Users of non-Posix systems (MS-Windows etc.) should follow the
+procedure in autogen.sh, but run the appropriate platform-specific
+configuration scripts (nt/configure.bat, config.bat, etc.) before the
+"make bootstrap" or "make" step; the rest of the procedure is
 applicable to those systems as well.
 
-Note that "make bootstrap" overwrites some files that are under CVS
-control, such as lisp/loaddefs.el.  This could produce CVS conflicts
-next time that you resync with the CVS.  If you see such conflicts,
-overwrite your local copy of the file with the clean version from the
-CVS repository.  For example:
+What to do if you get unexpected CVS conflicts later:
+=====================================================
+
+Bootstrapping overwrites some files that are under CVS control, such
+as lisp/loaddefs.el.  This could produce CVS conflicts next time that
+you do a 'cvs update'.  If you see such conflicts, overwrite your
+local copy of the file with the clean version from the CVS repository.
+For example:
 
     cvs update -C lisp/loaddefs.el
+
+Bugs:
+=====
 
 Please report any bugs in the CVS versions to emacs-pretest-bug@gnu.org.

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Kim F. Storm]

Karl Fogel <kfogel@floss.red-bean.com> writes:

> In other words, the purpose of autogen.sh is to get your working copy
> to the "same" state as a distribution tarball, at least as far as the
> build procedure is concerned.

This is obviously not the case for the CVS emacs autogen.sh, so 
couldn't ./autogen.sh simply print a message telling the user to
follow the steps described in the (unmodified) INSTALL-CVS file?

> +Therefore, to build from a CVS working copy, you should first run
> +   ./autogen.sh

With the proposed change to use autogen.sh, the text in INSTALL-CVS
becomes more cryptic and less useful, e.g.

> -Normally, it is not necessary to use "make bootstrap" after every CVS
> -update.  Unless there are problems, we suggest the following
> -procedure:

Here you entirely remove the "recompile" instructions -- assuming that
the user will run autogen.sh _again_ to get those instructions.  Not
very likely IMO.

Also for non-POSIX systems has become more cryptic ... instead of
simply stating that the user should run e.g. nt/configure.bat instead
of configure, it instructs the user to study the autogen.sh script
to determine what to do.

>  
> +Users of non-Posix systems (MS-Windows etc.) should follow the
> +procedure in autogen.sh, but run the appropriate platform-specific
> +configuration scripts (nt/configure.bat, config.bat, etc.) before the
> +"make bootstrap" or "make" step; the rest of the procedure is


All in all, I think the proposed changes is a change to the worse,
making it harder rather than easier to build and maintain a CVS emacs
checkout.

But I like the header lines you added in the INSTALL-CVS file :-)

As I suggested, the following autogen.sh would suffice:

        #!/bin/sh
        echo "Please read INSTALL-CVS for instructions how to build emacs."

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Karl Fogel]

storm@cua.dk (Kim F. Storm) writes:
> As I suggested, the following autogen.sh would suffice:
> 
>         #!/bin/sh
>         echo "Please read INSTALL-CVS for instructions how to build emacs."

I think that would be fine.

I'll just commit the above, if no one objects.

-K

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Richard Stallman]

    >         #!/bin/sh
    >         echo "Please read INSTALL-CVS for instructions how to build emacs."

    I think that would be fine.

    I'll just commit the above, if no one objects.

Please don't do that.  My first message describes
a much better method.

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Karl Fogel]

Richard Stallman <rms@gnu.org> writes:
> Please don't do that.  My first message describes
> a much better method.

Whups, already committed now, but even as it is it's an improvement
over the previous situation (an autogen.sh that tells the user to read
INSTALL-CVS is better than no autogen.sh at all).

The method you described wasn't completely fleshed out, and left a few
practical issues unaddressed.  Nevertheless, my original patch
implemented it, or something very close to it, and Kim objected, for
reasons that make a lot of sense to me.

Emacs' bootstrapping is both different from, and more complex than,
most other projects.  If we try to build too much intelligence into
autogen.sh, I think we'll just end up hampering developers.  IMHO it's
better for them to just read & understand INSTALL-CVS.

However, if we can come up with some precise description of how
autogen.sh should behave, I'd be happy to implement it.  Can you
describe exactly what you had in mind?

-Karl

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Richard Stallman]

    The method you described wasn't completely fleshed out, and left a few
    practical issues unaddressed.  Nevertheless, my original patch
    implemented it, or something very close to it,

It was not close at all.  Your first script actually ran the bootstrap
procedure.  That is very different from what I suggested.

    However, if we can come up with some precise description of how
    autogen.sh should behave, I'd be happy to implement it.

Here's the description I wrote before:

      I think it should create a file `must-bootstrap'.  Then
    the next `make all' run should notice the existence of must-bootstrap,
    and do a bootstrap automatically.

Could you be more specific about why you think that is hard?
It could be implemented inside the rule maybe-bootstrap.

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Richard Stallman]

I don't think that autogen.sh should do the configure and make
bootstrap.  I think it should create a file `must-bootstrap'.  Then
the next `make all' run should notice the existence of must-bootstrap,
and do a bootstrap automatically.  That way the usual sequence of
commands

       $ ./autogen.sh        <=== this is the extra step
       $ ./configure
       $ ./make
       $ ./make install

will work as expected.

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Richard Stallman]

The Emacs tar file is the official release of Emacs.  It includes the
.elc files, and ./configure; make works to build it.

It's not necessary for the CVS sources to support this.
It would be NICE if they did, all else being equal.
However, doing it in the obvious way (by bootstrapping)
would be a big pain in the neck for people who
build from CVS regularly.  It would be too painful.

Re: Emacs in CVS as today does not build - emacs depends on *.elc, which cannot be built without emacs

[Eli Zaretskii]

> Date: Thu, 31 Oct 2002 15:40:02 +0100
> From: =?ISO-8859-1?Q?Thorbj=F8rn_Ravn_Andersen?= <tra@biobase.dk>
> 
> >>Why does the main "make" not compile the lisp files?
> >
> >It does, if you say "make bootstrap".  Plain "make" assumes the *.elc 
> >files are already there.
> >
> I understood that.  I still do not understand why the normal "make" 
> command in the "./compile; make; make install" cycle does not do this?

Because compiling them takes a lot of time if we compile all of them.

It's possible to compile only those which aren't compiled, but setting
this up is not easy, as Stefan explained, and was not done yet.
(Please remember that anon CVS access to Emacs development sources is
relatively new.)