Bootstrapping Emacs-23.0.60-trunk still fails (GNU/Linux, Cygwin)

Bootstrapping Emacs-23.0.60-trunk still fails (GNU/Linux, Cygwin)

[Angelo Graziosi]

On GNU/Linux Kubuntu 7.10 and Cygwin, the bootstrap still fails:

[...]
Compiling /tmp.builds/emacs/lisp/./xt-mouse.el
Wrote /tmp.builds/emacs/lisp/xt-mouse.elc
Compiling /tmp.builds/emacs/lisp/./calc/calc-aent.el

In toplevel form:
../../emacs/lisp/calc/calc-aent.el:32:1:Error: Cannot open load file: 
calc-loaddefs.el
make[2]: *** [compile] Error 1
make[2]: Leaving directory `/tmp.builds/build/lisp'
make[1]: *** [bootstrap-build] Error 2
make[1]: Leaving directory `/tmp.builds/build'
make: *** [bootstrap] Error 2

Every time, I have deleted the old CVS then a fresh checkout:

cvs -z3 -d:pserver:anonymous@cvs.savannah.gnu.org:/sources/emacs co -P emacs

but always the same failure.

There are some posts ([1]) regarding this issue.

Cheers,
    Angelo.

---
[1]
http://lists.gnu.org/archive/html/emacs-devel/2008-04/msg00846.html
http://lists.gnu.org/archive/html/emacs-devel/2008-04/msg00845.html
http://lists.gnu.org/archive/html/emacs-devel/2008-04/msg00790.html
http://lists.gnu.org/archive/html/emacs-devel/2008-04/msg00760.html

Bootstrapping Emacs-23.0.60-trunk newly broken (GNU/Linux, Cygwin)

[Angelo Graziosi]

(GNU/Linux Kubuntu 7.10 and Cygwin)

The bootstrap is newly broken. About 15 hours ago was fixed with:

----------------------------------------------------------
2008-04-11  Stefan Monnier  <...>
[...]

         * dired.el (dired-read-dir-and-switches): Use 
read-directory-name even
         for non-dialogs.

         * Makefile.in (bootstrap-prepare): Don't copy ldefs-boot over 
loaddefs.

         * loadup.el: Load ldefs-boot.el if loaddefs.el doesn't exist.
----------------------------------------------------------------------

(It failed as described in [1])

Now it fails as described in [2]!!!

Just a curiosity: who submits patches to CVS, will test them with a 
clean bootstrap???

'Clean' does not mean 'make maintainer-clean && make bootstrap', but:

mkdir new_dir
cd new_dir
cvs -z3 -d:pserver:anonymous@cvs.savannah.gnu.org:/sources/emacs co -P emac

cd emacs
./configure
make bootstrap


Yes, it is heavy, but it is the only clean method that will guarantee 
the patches do no introduce regressions.


Cheers,
    Angelo.

---
[1] http://lists.gnu.org/archive/html/emacs-devel/2008-04/msg00865.html
[2] http://lists.gnu.org/archive/html/emacs-devel/2008-04/msg00921.html

"I have a dream..."

                       Martin Luther KING, Washington 28 August 1963

Re: Bootstrapping Emacs-23.0.60-trunk newly broken (GNU/Linux, Cygwin)

[Eli Zaretskii]

> Date: Sat, 12 Apr 2008 09:45:00 +0200
> From: Angelo Graziosi <angelo.graziosi@alice.it>
> 
> cd emacs
> ./configure
> make bootstrap
> 
> 
> Yes, it is heavy, but it is the only clean method that will guarantee 
> the patches do no introduce regressions.

Actually, there is no practical way to know for sure that changes
don't introduce regressions, in general.  While "make bootstrap" will
make sure a bootstrap works on the submitter's machine (and is
recommended to those that can afford it), it says nothing about other
ways of building Emacs, nor about other machines and OSes we support.
It also doesn't tell anything about run-time behavior, and as long as
we don't have a test suite, there will always be a significant doubt.
(Even when we do have a test suite, we cannot ask contributors to run
it on more than one system -- her own.)

IOW, I submit that asking people to be sure they don't cause
regressions in these circumstances is tantamount to asking them not to
commit changes at all.

So please calm down.  The development trunk will inevitably become
broken from time to time, no matter what.  If you must build the trunk
every day or even more frequently, and depend on the resulting binary
for your daily work, I suggest to make some arrangements to have the
previous working build available until you are sure the new one builds
and works successfully.

Re: Bootstrapping Emacs-23.0.60-trunk newly broken (GNU/Linux, Cygwin)

[Angelo Graziosi]

Eli Zaretskii ha scritto:
>> Date: Sat, 12 Apr 2008 09:45:00 +0200
>> From: Angelo Graziosi <angelo.graziosi@alice.it>
>>
>> cd emacs
>> ./configure
>> make bootstrap
>>
>>
>> Yes, it is heavy, but it is the only clean method that will guarantee 
>> the patches do no introduce regressions.
> 
> Actually, there is no practical way to know for sure that changes
> don't introduce regressions, in general.  While "make bootstrap" will
> make sure a bootstrap works on the submitter's machine (and is
> recommended to those that can afford it), it says nothing about other
> ways of building Emacs, nor about other machines and OSes we support.

Obviously.

But how could the trunk users to test a new patch if it breaks the 
bootstrap?

I thought that passing the bootstrap was the minimal request for 
submitting a patch and that this would be reached if all start with the 
same conditions in bootstrapping.

Sorry if I has been wrong.

> So please calm down.  The development trunk will inevitably become
> broken from time to time, no matter what.  If you must build the trunk
> every day or even more frequently, and depend on the resulting binary
> for your daily work, I suggest to make some arrangements to have the
> previous working build available until you are sure the new one builds
> and works successfully.

I think that on my backup DVDs I have working build until two months ago.

And if I build so often, it is to avoid the pile-up of bugs (when they 
are present) that would make hard to solve, mainly on Cygwin, on which 
no cygwin-guys take care of them.

Sorry newly for the noise,
   Angelo.

Re: Bootstrapping Emacs-23.0.60-trunk newly broken (GNU/Linux, Cygwin)

[Juanma Barranquero]

On Sat, Apr 12, 2008 at 11:18 AM, Angelo Graziosi
<angelo.graziosi@alice.it> wrote:

>  But how could the trunk users to test a new patch if it breaks the
> bootstrap?

There is no need to bootstrap after every cvs update...

>  I thought that passing the bootstrap was the minimal request for submitting
> a patch and that this would be reached if all start with the same conditions
> in bootstrapping.

In an ideal world, it'd be nice if each change were to be known not to
break bootstrapping. But the fact is, most patches do not break it
[1], and the ones that do are usually detected quite fast, so it is a
waste of resources to require everyone to bootstrap before each commit
[2].

[1] Some changes (configure, makefiles, autoloads, etc.) are more
likely than others; we're hit such a spot right now :(
[2] Bear in mind that tiny, apparently trivial changes can break it:
there's no complexity threshold below which a change is totally safe.

 Juanma

Re: Bootstrapping Emacs-23.0.60-trunk newly broken (GNU/Linux, Cygwin)

[Romain Francoise]

"Juanma Barranquero" <lekktu@gmail.com> writes:

> In an ideal world, it'd be nice if each change were to be known
> not to break bootstrapping. But the fact is, most patches do not
> break it [1], and the ones that do are usually detected quite
> fast, so it is a waste of resources to require everyone to
> bootstrap before each commit [2].

While that is certainly true, it's also a waste of time for our
users to have to report bugs that could be avoided, and a waste of
our own time because we have to handle the reports.

For the last few months I've been running a buildbot for the trunk
and the 22 branch (and before the merge, for the unicode branch), it
does a clean build every night (on GNU/Linux only) and usually
catches bootstrap failures.  See <URL:http://build.orebokech.com/>.

If there's interest we could add Cygwin and Windows build slaves
and possibly increase the build frequency (the usual way of using
buildbot is with a notification script in the CVS commit hooks, but
I don't think this is allowed on Savannah).

Re: Bootstrapping Emacs-23.0.60-trunk newly broken (GNU/Linux, Cygwin)

[Juanma Barranquero]

On Sat, Apr 12, 2008 at 1:46 PM, Romain Francoise <romain@orebokech.com> wrote:

>  While that is certainly true, it's also a waste of time for our
>  users to have to report bugs that could be avoided, and a waste of
>  our own time because we have to handle the reports.
>
>  For the last few months I've been running a buildbot for the trunk
>  and the 22 branch (and before the merge, for the unicode branch), it
>  does a clean build every night (on GNU/Linux only) and usually
>  catches bootstrap failures.  See <URL:http://build.orebokech.com/>.

Having an automatic method of detecting bootstrap failures (and
reporting to the committer(s)) would be good IMHO. Still, that does
not change the previous point: those building from the trunk should
know to expect trouble (and not just from building/bootstrapping)
every now and then.

 Juanma

Re: Bootstrapping Emacs-23.0.60-trunk newly broken (GNU/Linux, Cygwin)

[martin rudalics]

 > While that is certainly true, it's also a waste of time for our
 > users to have to report bugs that could be avoided, and a waste of
 > our own time because we have to handle the reports.

I violently agree.

 > For the last few months I've been running a buildbot for the trunk
 > and the 22 branch (and before the merge, for the unicode branch), it
 > does a clean build every night (on GNU/Linux only) and usually
 > catches bootstrap failures.  See <URL:http://build.orebokech.com/>.

Awesome.  In the future I shall _always_ consult your page before
bootstrapping.  Please keep on maintaining it.  Please add the URL and a
corresponding explanation to the INSTALL files.

 > If there's interest we could add Cygwin and Windows build slaves

I'd be very much interested.

 > and possibly increase the build frequency

The current frequency is OK with me.

 > (the usual way of using
 > buildbot is with a notification script in the CVS commit hooks, but
 > I don't think this is allowed on Savannah).

What would that do precisely?

Re: Bootstrapping Emacs-23.0.60-trunk newly broken (GNU/Linux, Cygwin)

[Romain Francoise]

martin rudalics <rudalics@gmx.at> writes:

>> (the usual way of using buildbot is with a notification script in
>> the CVS commit hooks, but I don't think this is allowed on
>> Savannah).

> What would that do precisely?

It would allow buildbot to trigger a rebuild automatically after a
commit (with configurable strategies to avoid rebuilding too often)
and notify the people who have committed changes (since the last
build) in case of failure.

Re: Bootstrapping Emacs-23.0.60-trunk newly broken (GNU/Linux, Cygwin)

[martin rudalics]

 > If there's interest we could add Cygwin and Windows build slaves
 > and possibly increase the build frequency (the usual way of using
 > buildbot is with a notification script in the CVS commit hooks, but
 > I don't think this is allowed on Savannah).

After glimpsing at the BuildBot manual: It might make sense to

(1) send a notification to emacs-devel as soon as a build of the trunk
failed and temporarily increase the build frequency,

(2) send another notification when the first build succeeds after that
and decrease the build frequency to the standard value (maybe after
another day).

In addition, it seems reasonable to temporarily increase the build
frequency after a major merge or other non-trivial changes.

Re: Bootstrapping Emacs-23.0.60-trunk newly broken (GNU/Linux, Cygwin)

[Romain Francoise]

martin rudalics <rudalics@gmx.at> writes:

> After glimpsing at the BuildBot manual: It might make sense to

> (1) send a notification to emacs-devel as soon as a build of the trunk
> failed and temporarily increase the build frequency,

> (2) send another notification when the first build succeeds after that
> and decrease the build frequency to the standard value (maybe after
> another day).

Okay so there are two aspects here: (i) increase the build frequency
in case of problems, and (ii) send notifications when the build
state changes (from passing to failing and back).

Implementing (i) requires some hacking because I don't think
buildbot has a built-in scheduler that changes behavior based on
build status, so to keep things simple I just increased the
frequency (to four builds per day).

There is no existing functionality for (ii) either, but it was
trivial to implement and seems to be working okay.

Stefan, Yidong, is it okay to send these notifications to
emacs-devel?

Re: Bootstrapping Emacs-23.0.60-trunk newly broken (GNU/Linux, Cygwin)

[Eli Zaretskii]

> From: Romain Francoise <romain@orebokech.com>
> Date: Sun, 13 Apr 2008 11:31:49 +0200
> Cc: Chong Yidong <cyd@stupidchicken.com>,
> 	Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel@gnu.org
> 
> Stefan, Yidong, is it okay to send these notifications to
> emacs-devel?

I'm neither Stefan nor Yidong, but IMHO emacs-devel is the _only_
place where these notifications should go.  Just be sure to subscribe
the buildbot's address to the list, because otherwise its
notifications will be held by mailman until I free them (since
emacs-devel is a moderated list), and you cannot rely on me doing that
more than once or twice a day.

Btw, aren't there facilities on Savannah (or maybe subversions) to run
scheduled builds like that?  If not, IMO they should add such a
feature, and if it already exists, we should use it in preference to
external tools.

Re: Bootstrapping Emacs-23.0.60-trunk newly broken (GNU/Linux, Cygwin)

[Richard Stallman]

I suggest appointing a few people as a committee to study this and
implement it, having discussions separately off this list, so as not
to take up the time and attention of everyone else.

Re: Bootstrapping Emacs-23.0.60-trunk newly broken (GNU/Linux, Cygwin)

[Stefan Monnier]

>> In an ideal world, it'd be nice if each change were to be known
>> not to break bootstrapping. But the fact is, most patches do not
>> break it [1], and the ones that do are usually detected quite
>> fast, so it is a waste of resources to require everyone to
>> bootstrap before each commit [2].

> While that is certainly true, it's also a waste of time for our
> users to have to report bugs that could be avoided, and a waste of
> our own time because we have to handle the reports.

The problem is that users are not supposed to just come and whine about
bootstrap failures.  They should at least do the work of figuring out
either a patch to fix it, or to find out the commit that broke it, ...
This is a development branch: expect it to be broken,


        Stefan


PS: The only problem that I would consider as really serious, even for
the development branch is a bug that risks losing data.

Re: Bootstrapping Emacs-23.0.60-trunk newly broken (GNU/Linux, Cygwin)

[Jason Rumney]

Romain Francoise wrote:
> (the usual way of using
> buildbot is with a notification script in the CVS commit hooks, but
> I don't think this is allowed on Savannah).
>   

You could subscribe your bot to emacs-commit@gnu.org

Re: Bootstrapping Emacs-23.0.60-trunk newly broken (GNU/Linux, Cygwin)

[Eric Hanchrow]

>>>>> "Romain" == Romain Francoise <romain@orebokech.com> writes:

    Romain> If there's interest we could add Cygwin and Windows build
    Romain> slaves 

If you do add a Windows build slave, would you be sure to announce it
here?  Not so much because I want to follow the Emacs build, but
because I'd like to learn how to use buildbot on Windows, and am
interested in talking to someone who's actually done it.

-- 
The old graybeards in the Smalltalk world may not seem
relevant, but if you ask them a question about ORM, they have been
thinking about it for 20 years.
        -- Avi Bryant

Re: Bootstrapping Emacs-23.0.60-trunk newly broken (GNU/Linux, Cygwin)

[Eli Zaretskii]

> Date: Sat, 12 Apr 2008 11:18:17 +0200
> From: Angelo Graziosi <angelo.graziosi@alice.it>
> CC: emacs-devel@gnu.org
> 
> But how could the trunk users to test a new patch if it breaks the 
> bootstrap?

If the changes also break a non-bootstrap build, then they can't, not
until the trunk is fixed to build again.  Or they can fix it
themselves, if they have time and knowhow.

> I thought that passing the bootstrap was the minimal request for 
> submitting a patch and that this would be reached if all start with the 
> same conditions in bootstrapping.

I don't know if this is the minimal requirement, and it's not my place
to introduce such requirements.  All I can say is that bootstrapping
takes a significant time, so requiring it from each submitter would be
tough on them.  FWIW, I bootstrap only very infrequently.

> Sorry newly for the noise,

No need to apologize.  Thanks for timely reports of the breakage.

Re: Bootstrapping Emacs-23.0.60-trunk newly broken (GNU/Linux, Cygwin)

[Eli Zaretskii]

> Date: Sat, 12 Apr 2008 09:45:00 +0200
> From: Angelo Graziosi <angelo.graziosi@alice.it>
> 
> (GNU/Linux Kubuntu 7.10 and Cygwin)
> 
> The bootstrap is newly broken.

I think I fixed it, please try again.