Build failure on Windows

Build failure on Windows

[Eric Hanchrow]

I did `cvs up' at about 2004-03-11T18:25:05Z, and then built like this:

    cd nt
    PATH=/d/MinGW/bin:$PATH

    /d/MinGW/bin/make

I got these link errors:

[Please ignore a syntax error on the next line - it is intentional]
Syntax error: Unterminated quoted string
[Please ignore a syntax error on the next line - it is intentional]
Syntax error: Unterminated quoted string
mkdir: cannot create directory `../src/oo-spd': File exists
make.exe[1]: [DOC] Error 1 (ignored)
mkdir: cannot create directory `../src/oo-spd/etc': File exists
make.exe[1]: [DOC] Error 1 (ignored)
[Please ignore a syntax error on the next line - it is intentional]
Syntax error: Unterminated quoted string
oo-spd/i386/temacs0.a(emacs.o.b)(.text+0x2095): In function `main':
d:/cygwin/usr/local/src/emacs-cvs/src/emacs.c:1514: undefined reference to `syms_of_image'
oo-spd/i386/temacs0.a(emacs.o.b)(.text+0x2101):d:/cygwin/usr/local/src/emacs-cvs/src/emacs.c:1597: undefined reference to `init_image'
oo-spd/i386/temacs1.a(alloc.o)(.text+0x35a5): In function `mark_image_cache':
d:/cygwin/usr/local/src/emacs-cvs/src/alloc.c:4677: undefined reference to `forall_images_in_image_cache'
oo-spd/i386/temacs1.a(xdisp.o)(.text+0x4288): In function `handle_single_display_prop':
d:/cygwin/usr/local/src/emacs-cvs/src/xdisp.c:3665: undefined reference to `lookup_image'
oo-spd/i386/temacs1.a(xdisp.o)(.text+0x43cf):d:/cygwin/usr/local/src/emacs-cvs/src/xdisp.c:3614: undefined reference to `valid_image_p'
oo-spd/i386/temacs1.a(xdisp.o)(.text+0xb7eb): In function `build_desired_tool_bar_string':
d:/cygwin/usr/local/src/emacs-cvs/src/xdisp.c:8447: undefined reference to `valid_image_p'
oo-spd/i386/temacs1.a(xdisp.o)(.text+0xb8a1):d:/cygwin/usr/local/src/emacs-cvs/src/xdisp.c:8506: undefined reference to `QCmargin'
oo-spd/i386/temacs1.a(xdisp.o)(.text+0xb961):d:/cygwin/usr/local/src/emacs-cvs/src/xdisp.c:8515: undefined reference to `QCconversion'
oo-spd/i386/temacs1.a(xdisp.o)(.text+0xb97a):d:/cygwin/usr/local/src/emacs-cvs/src/xdisp.c:8504: undefined reference to `QCmargin'
oo-spd/i386/temacs1.a(xdisp.o)(.text+0xb9a5):d:/cygwin/usr/local/src/emacs-cvs/src/xdisp.c:8482: undefined reference to `QCrelief'
oo-spd/i386/temacs1.a(xdisp.o)(.text+0xb9dc):d:/cygwin/usr/local/src/emacs-cvs/src/xdisp.c:8492: undefined reference to `QCrelief'
oo-spd/i386/temacs1.a(xdisp.o)(.text+0xdce0): In function `redisplay_internal':
d:/cygwin/usr/local/src/emacs-cvs/src/xdisp.c:10090: undefined reference to `clear_image_cache'
oo-spd/i386/temacs1.a(xdisp.o)(.text+0x1d25f): In function `produce_image_glyph':
d:/cygwin/usr/local/src/emacs-cvs/src/xdisp.c:17876: undefined reference to `prepare_image_for_display'
oo-spd/i386/temacs1.a(xdisp.o)(.text+0x1d26b):d:/cygwin/usr/local/src/emacs-cvs/src/xdisp.c:17878: undefined reference to `image_ascent'
oo-spd/i386/temacs1.a(xdisp.o)(.text+0x1dda3): In function `produce_stretch_glyph':
d:/cygwin/usr/local/src/emacs-cvs/src/xdisp.c:18298: undefined reference to `QCascent'
oo-spd/i386/temacs1.a(fringe.o)(.text+0x13c0): In function `Fdefine_fringe_bitmap':
d:/cygwin/usr/local/src/emacs-cvs/src/fringe.c:1162: undefined reference to `Qcenter'
oo-spd/i386/temacw32.a(w32fns.o)(.text+0xa831): In function `Fx_close_connection':
d:/cygwin/usr/local/src/emacs-cvs/src/w32fns.c:6678: undefined reference to `x_destroy_all_bitmaps'
oo-spd/i386/temacw32.a(w32term.o)(.text+0x2457): In function `x_setup_relief_colors':
d:/cygwin/usr/local/src/emacs-cvs/src/w32term.c:1778: undefined reference to `image_background'
oo-spd/i386/temacw32.a(w32term.o)(.text+0x2470):d:/cygwin/usr/local/src/emacs-cvs/src/w32term.c:1778: undefined reference to `image_background_transparent'
oo-spd/i386/temacw32.a(w32term.o)(.text+0x2b3a): In function `x_draw_image_foreground':
d:/cygwin/usr/local/src/emacs-cvs/src/w32term.c:1964: undefined reference to `image_ascent'
oo-spd/i386/temacw32.a(w32term.o)(.text+0x2f3a): In function `x_draw_image_relief':
d:/cygwin/usr/local/src/emacs-cvs/src/w32term.c:2056: undefined reference to `image_ascent'
oo-spd/i386/temacw32.a(w32term.o)(.text+0x30c7): In function `w32_draw_image_foreground_1':
d:/cygwin/usr/local/src/emacs-cvs/src/w32term.c:2104: undefined reference to `image_ascent'
oo-spd/i386/temacw32.a(w32term.o)(.text+0x941d): In function `w32_term_init':
d:/cygwin/usr/local/src/emacs-cvs/src/w32term.c:6188: undefined reference to `make_image_cache'
oo-spd/i386/temacw32.a(xfaces.o)(.text+0x162): In function `init_frame_faces':
d:/cygwin/usr/local/src/emacs-cvs/src/xfaces.c:889: undefined reference to `make_image_cache'
oo-spd/i386/temacw32.a(xfaces.o)(.text+0x2a5): In function `clear_face_cache':
d:/cygwin/usr/local/src/emacs-cvs/src/xfaces.c:1001: undefined reference to `clear_image_cache'
oo-spd/i386/temacw32.a(xfaces.o)(.text+0x671): In function `load_pixmap':
d:/cygwin/usr/local/src/emacs-cvs/src/xfaces.c:1180: undefined reference to `x_create_bitmap_from_file'
oo-spd/i386/temacw32.a(xfaces.o)(.text+0x6ba):d:/cygwin/usr/local/src/emacs-cvs/src/xfaces.c:1200: undefined reference to `x_bitmap_width'
oo-spd/i386/temacw32.a(xfaces.o)(.text+0x6d5):d:/cygwin/usr/local/src/emacs-cvs/src/xfaces.c:1203: undefined reference to `x_bitmap_height'
oo-spd/i386/temacw32.a(xfaces.o)(.text+0x7af):d:/cygwin/usr/local/src/emacs-cvs/src/xfaces.c:1174: undefined reference to `x_create_bitmap_from_data'
oo-spd/i386/temacw32.a(xfaces.o)(.text+0x1343): In function `load_face_colors':
d:/cygwin/usr/local/src/emacs-cvs/src/xfaces.c:1673: undefined reference to `x_destroy_bitmap'
oo-spd/i386/temacw32.a(xfaces.o)(.text+0x6841): In function `free_realized_face':
d:/cygwin/usr/local/src/emacs-cvs/src/xfaces.c:5114: undefined reference to `x_destroy_bitmap'
oo-spd/i386/temacw32.a(xfaces.o)(.text+0x1e1): In function `free_frame_faces':
d:/cygwin/usr/local/src/emacs-cvs/src/xfaces.c:932: undefined reference to `free_image_cache'
make.exe[1]: *** [oo-spd/i386/temacs.exe] Error 1
d:\MinGW\bin\make.exe: *** [all-other-dirs-gmake] Error 2

-- 
Always code as if the guy who ends up maintaining your code will
    be a violent psychopath who knows where you live.  John F. Woods

Build failure on Windows

[martin rudalics]

Current master fails to build on Windows XP as

floatfns.o: In function `ecount_leading_zeros':
c:\emacs-git\trunk\dbg\src/../../src/floatfns.c:298: undefined reference to `count_leading_zeros_ll'
collect2: ld returned 1 exit status
make[1]: *** [temacs.exe] Error 1
make[1]: Leaving directory `/c/emacs-git/trunk/dbg/src'
make: *** [src] Error 2

In the past weeks I spent many, many hours rebuilding and bootstrapping
master with all sorts of failures.  I'll stop pulling now until the
situation improves.

martin

Re: Build failure on Windows

[Andy Moreton]

On Sun 05 Mar 2017, martin rudalics wrote:

> Current master fails to build on Windows XP as
>
> floatfns.o: In function `ecount_leading_zeros':
> c:\emacs-git\trunk\dbg\src/../../src/floatfns.c:298: undefined reference to `count_leading_zeros_ll'
> collect2: ld returned 1 exit status
> make[1]: *** [temacs.exe] Error 1
> make[1]: Leaving directory `/c/emacs-git/trunk/dbg/src'
> make: *** [src] Error 2
>
> In the past weeks I spent many, many hours rebuilding and bootstrapping
> master with all sorts of failures.  I'll stop pulling now until the
> situation improves.

I have seen the same problem, which occurred after the most recent
import from gnulib. This is because nt/gnulib.mk needs to be regenerated
from lib/gnulib.mk (to use an extra gnulib module), but the build
machinery does not seem to handle that properly.

To fix the problem:
 - delete nt/gnulib.mk
 - run autogen.sh
 - build as normal

HTH,

    AndyM

Re: Build failure on Windows

[martin rudalics]

 > I have seen the same problem, which occurred after the most recent
 > import from gnulib. This is because nt/gnulib.mk needs to be regenerated
 > from lib/gnulib.mk (to use an extra gnulib module), but the build
 > machinery does not seem to handle that properly.
 >
 > To fix the problem:
 >   - delete nt/gnulib.mk
 >   - run autogen.sh
 >   - build as normal

Thanks for the hint.  But this gets me the usual

Loading abbrev.el (source)...
Eager macro-expansion failure: (error "(require cl-lib) while preparing to dump"
)
Eager macro-expansion failure: (error "(require cl-lib) while preparing to dump"
)
(require cl-lib) while preparing to dump
ln -f emacs.exe bootstrap-emacs.exe
ln: accessing `emacs.exe': No such file or directory
make[1]: *** [emacs.exe] Error 1
make[1]: Leaving directory `/c/emacs-git/trunk/dbg/src'
make: *** [src] Error 2

which means that with every pull I apparently have to bootstrap.  I
can't afford that.

martin

Re: Build failure on Windows

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Sun, 05 Mar 2017 11:03:06 +0000
> 
> I have seen the same problem, which occurred after the most recent
> import from gnulib. This is because nt/gnulib.mk needs to be regenerated
> from lib/gnulib.mk (to use an extra gnulib module), but the build
> machinery does not seem to handle that properly.

Actually, it does work for me, but I always build inside the source
tree.  So something is either missing or not quite right for
out-of-tree builds, ideas are welcome.  (If needed, I can explain how
this was supposed to work.)

Re: Build failure on Windows

[Eli Zaretskii]

> Date: Sun, 05 Mar 2017 14:27:58 +0100
> From: martin rudalics <rudalics@gmx.at>
> 
> Loading abbrev.el (source)...
> Eager macro-expansion failure: (error "(require cl-lib) while preparing to dump"
> )
> Eager macro-expansion failure: (error "(require cl-lib) while preparing to dump"
> )
> (require cl-lib) while preparing to dump
> ln -f emacs.exe bootstrap-emacs.exe
> ln: accessing `emacs.exe': No such file or directory
> make[1]: *** [emacs.exe] Error 1
> make[1]: Leaving directory `/c/emacs-git/trunk/dbg/src'
> make: *** [src] Error 2
> 
> which means that with every pull I apparently have to bootstrap.  I
> can't afford that.

No need to bootstrap (I never do, as it deletes the old binaries I
keep for testing and bisection).  What you need is manually
byte-compile a few of the Lisp files updated: cl-lib.el, pp.el,
re-builder.el, and abbrev.el.

Re: Build failure on Windows

[martin rudalics]

So with these messages ...

 >> Loading abbrev.el (source)...
 >> Eager macro-expansion failure: (error "(require cl-lib) while preparing to dump"
 >> )
 >> Eager macro-expansion failure: (error "(require cl-lib) while preparing to dump"
 >> )
 >> (require cl-lib) while preparing to dump
 >> ln -f emacs.exe bootstrap-emacs.exe
 >> ln: accessing `emacs.exe': No such file or directory
 >> make[1]: *** [emacs.exe] Error 1
 >> make[1]: Leaving directory `/c/emacs-git/trunk/dbg/src'
 >> make: *** [src] Error 2
 >>
 >> which means that with every pull I apparently have to bootstrap.  I
 >> can't afford that.
 >
 > No need to bootstrap (I never do, as it deletes the old binaries I
 > keep for testing and bisection).  What you need is manually
 > byte-compile a few of the Lisp files updated: cl-lib.el, pp.el,
 > re-builder.el, and abbrev.el.

... I am supposed to recompile abbrev.el and cl-lib.el and continue the
build.  Correct?  Then this should be added to the INSTALL instructions
either in the source directory or in the nt directory if this is Windows
specific.  Pretty please.

Thanks in advance, martin

Re: Build failure on Windows

[Andy Moreton]

On Sun 05 Mar 2017, Eli Zaretskii wrote:

>> From: Andy Moreton <andrewjmoreton@gmail.com>
>> Date: Sun, 05 Mar 2017 11:03:06 +0000
>> 
>> I have seen the same problem, which occurred after the most recent
>> import from gnulib. This is because nt/gnulib.mk needs to be regenerated
>> from lib/gnulib.mk (to use an extra gnulib module), but the build
>> machinery does not seem to handle that properly.
>
> Actually, it does work for me, but I always build inside the source
> tree.  So something is either missing or not quite right for
> out-of-tree builds, ideas are welcome.  (If needed, I can explain how
> this was supposed to work.)

I posted a partial fix in:
http://lists.gnu.org/archive/html/emacs-devel/2017-03/msg00082.html

Should autogen.sh handle updates to nt/gnulib.mk rather than just
initial creation ? If so, perhaps it should be using move-if-change when
generating this file.

    AndyM

Re: Build failure on Windows

[martin rudalics]

 >  > No need to bootstrap (I never do, as it deletes the old binaries I
 >  > keep for testing and bisection).  What you need is manually
 >  > byte-compile a few of the Lisp files updated: cl-lib.el, pp.el,
 >  > re-builder.el, and abbrev.el.

Currently I'm at:

Loading help...
Loading jka-cmpr-hook...
Growing hash table to: 120000
Attempt to autoload pcase while preparing to dump
ln -f emacs.exe bootstrap-emacs.exe
ln: accessing `emacs.exe': No such file or directory
make[1]: *** [emacs.exe] Error 1
make[1]: Leaving directory `/c/emacs-git/slow/dbg/src'
make: *** [src] Error 2

I recompiled both pcase.el and jka-cmpr-hook.el.  How to continue?

martin

Re: Build failure on Windows

[Eli Zaretskii]

> Date: Sun, 05 Mar 2017 18:59:08 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: andrewjmoreton@gmail.com, emacs-devel@gnu.org
> 
>  > No need to bootstrap (I never do, as it deletes the old binaries I
>  > keep for testing and bisection).  What you need is manually
>  > byte-compile a few of the Lisp files updated: cl-lib.el, pp.el,
>  > re-builder.el, and abbrev.el.
> 
> ... I am supposed to recompile abbrev.el and cl-lib.el and continue the
> build.  Correct?  Then this should be added to the INSTALL instructions
> either in the source directory or in the nt directory if this is Windows
> specific.  Pretty please.

The Lisp files that need to be recompiled by hand changes every time,
so it's hard to say something specific.  Besides, my impression is
that avoiding bootstrap is frowned upon in certain quarters...

Re: Build failure on Windows

[Eli Zaretskii]

> Date: Sun, 05 Mar 2017 19:32:15 +0100
> From: martin rudalics <rudalics@gmx.at>
> Cc: andrewjmoreton@gmail.com, emacs-devel@gnu.org
> 
> Loading help...
> Loading jka-cmpr-hook...
> Growing hash table to: 120000
> Attempt to autoload pcase while preparing to dump
> ln -f emacs.exe bootstrap-emacs.exe
> ln: accessing `emacs.exe': No such file or directory
> make[1]: *** [emacs.exe] Error 1
> make[1]: Leaving directory `/c/emacs-git/slow/dbg/src'
> make: *** [src] Error 2
> 
> I recompiled both pcase.el and jka-cmpr-hook.el.  How to continue?

Did you recompile pp.el?

Re: Build failure on Windows

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Sun, 05 Mar 2017 18:10:35 +0000
> 
> > Actually, it does work for me, but I always build inside the source
> > tree.  So something is either missing or not quite right for
> > out-of-tree builds, ideas are welcome.  (If needed, I can explain how
> > this was supposed to work.)
> 
> I posted a partial fix in:
> http://lists.gnu.org/archive/html/emacs-devel/2017-03/msg00082.html

But that's unrelated to nt/gnulib.mk, right?  It's a separate problem,
AFAIU.

> Should autogen.sh handle updates to nt/gnulib.mk rather than just
> initial creation ?

There was opposition to asking people to run autogen.sh.  The
preference is that just "make" does TRT.  So I think we should avoid
placing this burden on autogen.sh if possible.

Re: Build failure on Windows

[martin rudalics]

 >> Loading help...
 >> Loading jka-cmpr-hook...
 >> Growing hash table to: 120000
 >> Attempt to autoload pcase while preparing to dump
 >> ln -f emacs.exe bootstrap-emacs.exe
 >> ln: accessing `emacs.exe': No such file or directory
 >> make[1]: *** [emacs.exe] Error 1
 >> make[1]: Leaving directory `/c/emacs-git/slow/dbg/src'
 >> make: *** [src] Error 2
 >>
 >> I recompiled both pcase.el and jka-cmpr-hook.el.  How to continue?
 >
 > Did you recompile pp.el?

I did now.  Stuck as before.

martin

Re: Build failure on Windows

[martin rudalics]

 > The Lisp files that need to be recompiled by hand changes every time,
 > so it's hard to say something specific.  Besides, my impression is
 > that avoiding bootstrap is frowned upon in certain quarters...

Well, I don't know where to go from here.  In the past two weeks I
bootstrapped more often than in the past five years.  And if I'm not
mistaken others experienced the same problem.  I think I'll take a break
for a couple of weeks or months.  Maybe things will settle down.

martin

Re: Build failure on Windows

[Paul Eggert]

martin rudalics wrote:
> I don't know where to go from here.  In the past two weeks I
> bootstrapped more often than in the past five years.  And if I'm not
> mistaken others experienced the same problem.

One possible improvement in this area is to use GNU Make's Automake-like 
features instead of using Automake. This should simplify the gnulib.mk business 
and should cause breakages to be less likely when the build procedure changes. 
We already assume GNU Make, so this should not introduce build dependencies. I 
mentioned this in January when we had similar build problems with macOS, but 
never got around to doing it. I'll try to bump the priority of this. See:

http://lists.gnu.org/archive/html/emacs-devel/2017-01/msg00097.html

Re: Build failure on Windows

[Eli Zaretskii]

> Date: Sun, 05 Mar 2017 22:28:33 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: andrewjmoreton@gmail.com, emacs-devel@gnu.org
> 
>  >> Attempt to autoload pcase while preparing to dump
>  >> ln -f emacs.exe bootstrap-emacs.exe
>  >> ln: accessing `emacs.exe': No such file or directory
>  >> make[1]: *** [emacs.exe] Error 1
>  >> make[1]: Leaving directory `/c/emacs-git/slow/dbg/src'
>  >> make: *** [src] Error 2
>  >>
>  >> I recompiled both pcase.el and jka-cmpr-hook.el.  How to continue?
>  >
>  > Did you recompile pp.el?
> 
> I did now.  Stuck as before.

Which other Lisp files are newer than their *.elc counterparts?

Re: Build failure on Windows

[Eli Zaretskii]

> Date: Sun, 05 Mar 2017 22:28:51 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: andrewjmoreton@gmail.com, emacs-devel@gnu.org
> 
>  > The Lisp files that need to be recompiled by hand changes every time,
>  > so it's hard to say something specific.  Besides, my impression is
>  > that avoiding bootstrap is frowned upon in certain quarters...
> 
> Well, I don't know where to go from here.  In the past two weeks I
> bootstrapped more often than in the past five years.  And if I'm not
> mistaken others experienced the same problem.  I think I'll take a break
> for a couple of weeks or months.  Maybe things will settle down.

I don't think master is broken, or that these things will become
better in the future.  One recipe to avoid hitting these problems is
to resync with upstream more often, because that minimizes the number
of Lisp files that need to be recompiled, and lowers the probability
of hitting some circular dependency with macros or somesuch.

Re: Build failure on Windows

[Eli Zaretskii]

> Cc: andrewjmoreton@gmail.com, emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Sun, 5 Mar 2017 13:38:18 -0800
> 
> martin rudalics wrote:
> > I don't know where to go from here.  In the past two weeks I
> > bootstrapped more often than in the past five years.  And if I'm not
> > mistaken others experienced the same problem.
> 
> One possible improvement in this area is to use GNU Make's Automake-like 
> features instead of using Automake. This should simplify the gnulib.mk business 
> and should cause breakages to be less likely when the build procedure changes. 

The issue with gnulib.mk is no longer Martin's problem.

Re: Build failure on Windows

[martin rudalics]

 > Which other Lisp files are newer than their *.elc counterparts?

About 15 others I think.  I do ‘byte-recompile-directory’ now before
making.  Whatever it breaks is better than the preceding nightmare.

Thanks for your advices, martin

Re: Build failure on Windows

[martin rudalics]

 > I don't think master is broken, or that these things will become
 > better in the future.

"These things" did not happen in the past ten years.

 > One recipe to avoid hitting these problems is
 > to resync with upstream more often, because that minimizes the number
 > of Lisp files that need to be recompiled, and lowers the probability
 > of hitting some circular dependency with macros or somesuch.

I used to synch with master once a day on a regular basis.  I stopped
doing that after build failures started to pile up.

martin

Re: Build failure on Windows

[Eli Zaretskii]

> Date: Mon, 06 Mar 2017 09:11:20 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: andrewjmoreton@gmail.com, emacs-devel@gnu.org
> 
>  > I don't think master is broken, or that these things will become
>  > better in the future.
> 
> "These things" did not happen in the past ten years.
> 
>  > One recipe to avoid hitting these problems is
>  > to resync with upstream more often, because that minimizes the number
>  > of Lisp files that need to be recompiled, and lowers the probability
>  > of hitting some circular dependency with macros or somesuch.
> 
> I used to synch with master once a day on a regular basis.  I stopped
> doing that after build failures started to pile up.

Hmm... that's not my impression, I didn't see any significant changes
in build problems lately.  Maybe that's again because I build inside
the source tree.

Do you have any idea when things started deteriorating, per chance?

Re: Build failure on Windows

[martin rudalics]

 > Hmm... that's not my impression, I didn't see any significant changes
 > in build problems lately.  Maybe that's again because I build inside
 > the source tree.

I need an optimized build for coding and non-optimized one for
debugging.  How else could I reconcile these?

 > Do you have any idea when things started deteriorating, per chance?

I usually bootstrap once every year after the traditional copyright
changes.  Till then there were no build problems.  I believe that I've
seen first problems around the beginning of February but didn't pay much
attention initially.  Things got very annoying in the past seven or ten
days and particularly on Windows: After yesterday's continuous failures
on Windows, pulling an even larger changeset on GNU/Linux today had all
builds proceed without complications.

martin

Re: Build failure on Windows

[Eli Zaretskii]

> Date: Mon, 06 Mar 2017 18:45:56 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: andrewjmoreton@gmail.com, emacs-devel@gnu.org
> 
>  > Hmm... that's not my impression, I didn't see any significant changes
>  > in build problems lately.  Maybe that's again because I build inside
>  > the source tree.
> 
> I need an optimized build for coding and non-optimized one for
> debugging.  How else could I reconcile these?

I didn't mean to suggest you should build from inside the tree, just
to explain why I might miss problems that you are bumping into.

>  > Do you have any idea when things started deteriorating, per chance?
> 
> I usually bootstrap once every year after the traditional copyright
> changes.  Till then there were no build problems.  I believe that I've
> seen first problems around the beginning of February but didn't pay much
> attention initially.  Things got very annoying in the past seven or ten
> days and particularly on Windows: After yesterday's continuous failures
> on Windows, pulling an even larger changeset on GNU/Linux today had all
> builds proceed without complications.

Are the failures always with outdated Lisp files?  Or are they due to
other aspects of the build?

Re: Build failure on Windows

[martin rudalics]

 >> I need an optimized build for coding and non-optimized one for
 >> debugging.  How else could I reconcile these?
 >
 > I didn't mean to suggest you should build from inside the tree, just
 > to explain why I might miss problems that you are bumping into.

After all nt/INSTALL says

   1. If you want to build Emacs outside of the source tree
      (recommended), create the build directory and chdir there.

Maybe we should remove that recommendation?

 > Are the failures always with outdated Lisp files?  Or are they due to
 > other aspects of the build?

I had three typical groups of failures (some of them also on GNU/Linux
IIRC):

- The "require cl-lib" ones - by far the largest group and by far the
   greatest annoyance.  Do we really need cl-lib in preloaded Elisp?  OK,
   this ship sailed long ago ...

- More specific cl-generic.el related errors.  IIRC usually something
   about cl-defgeneric but I don't recall the precise text any more.
   probably a subgroup of the first one.

- Something I certainly never saw before about invalid bytecode.

I also had errors I couldn't attribute to any of these groups like the
"autoload pcase while preparing to dump" failure.  Anyway, these too
seem to indicate that the dumped emacs fails due to some missing
recompilation.  Hence "outdated Lisp files" probably constitute the
major source of my problems.

Also, failures occurred only when a configure step was performed before
(which is probably obvious).  Pure makes never failed.

martin

Re: Build failure on Windows

[Eli Zaretskii]

> Date: Tue, 07 Mar 2017 10:45:16 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: andrewjmoreton@gmail.com, emacs-devel@gnu.org
> 
>  >> I need an optimized build for coding and non-optimized one for
>  >> debugging.  How else could I reconcile these?
>  >
>  > I didn't mean to suggest you should build from inside the tree, just
>  > to explain why I might miss problems that you are bumping into.
> 
> After all nt/INSTALL says
> 
>    1. If you want to build Emacs outside of the source tree
>       (recommended), create the build directory and chdir there.
> 
> Maybe we should remove that recommendation?

I don't see why.  Your problems are all related to outdated Lisp
files, so out-of-tree builds are unlikely to be a factor.  (Which
means I again don't understand why you report a perceptible
degradation in build success rates, while I don't see anything close
to that.)

>  > Are the failures always with outdated Lisp files?  Or are they due to
>  > other aspects of the build?
> 
> I had three typical groups of failures (some of them also on GNU/Linux
> IIRC):
> 
> - The "require cl-lib" ones - by far the largest group and by far the
>    greatest annoyance.  Do we really need cl-lib in preloaded Elisp?  OK,
>    this ship sailed long ago ...
> 
> - More specific cl-generic.el related errors.  IIRC usually something
>    about cl-defgeneric but I don't recall the precise text any more.
>    probably a subgroup of the first one.
> 
> - Something I certainly never saw before about invalid bytecode.

These are all related to outdated *.elc files.

> Also, failures occurred only when a configure step was performed before
> (which is probably obvious).  Pure makes never failed.

You mean, when configure is run automatically by "make"?

Re: Build failure on Windows

[martin rudalics]

 >>     1. If you want to build Emacs outside of the source tree
 >>        (recommended), create the build directory and chdir there.
 >>
 >> Maybe we should remove that recommendation?
 >
 > I don't see why.  Your problems are all related to outdated Lisp
 > files, so out-of-tree builds are unlikely to be a factor.  (Which
 > means I again don't understand why you report a perceptible
 > degradation in build success rates, while I don't see anything close
 > to that.)

Was the gnulib.mk issue from my first message in this thread related to
out-of-tree building?

 >>   > Are the failures always with outdated Lisp files?  Or are they due to
 >>   > other aspects of the build?
 >>
 >> I had three typical groups of failures (some of them also on GNU/Linux
 >> IIRC):
 >>
 >> - The "require cl-lib" ones - by far the largest group and by far the
 >>     greatest annoyance.  Do we really need cl-lib in preloaded Elisp?  OK,
 >>     this ship sailed long ago ...
 >>
 >> - More specific cl-generic.el related errors.  IIRC usually something
 >>     about cl-defgeneric but I don't recall the precise text any more.
 >>     probably a subgroup of the first one.
 >>
 >> - Something I certainly never saw before about invalid bytecode.
 >
 > These are all related to outdated *.elc files.

OK.  I never saw these in the past years.  Maybe I was just lucky.

 >> Also, failures occurred only when a configure step was performed before
 >> (which is probably obvious).  Pure makes never failed.
 >
 > You mean, when configure is run automatically by "make"?

Yes.  BTW, all rebuilds in the past minutes had configure run
automatically and succeeded without problems.  So maybe the issue is
resolved.

martin

Re: Build failure on Windows

[Eli Zaretskii]

> Date: Tue, 07 Mar 2017 17:13:43 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: andrewjmoreton@gmail.com, emacs-devel@gnu.org
> 
>  >>     1. If you want to build Emacs outside of the source tree
>  >>        (recommended), create the build directory and chdir there.
>  >>
>  >> Maybe we should remove that recommendation?
>  >
>  > I don't see why.  Your problems are all related to outdated Lisp
>  > files, so out-of-tree builds are unlikely to be a factor.  (Which
>  > means I again don't understand why you report a perceptible
>  > degradation in build success rates, while I don't see anything close
>  > to that.)
> 
> Was the gnulib.mk issue from my first message in this thread related to
> out-of-tree building?

I think so.  But the problems you cite as being the reason for most
failed builds are unrelated to gnulib.mk (which only changed once or
twice recently).

> 
>  >> - The "require cl-lib" ones - by far the largest group and by far the
>  >>     greatest annoyance.  Do we really need cl-lib in preloaded Elisp?  OK,
>  >>     this ship sailed long ago ...
>  >>
>  >> - More specific cl-generic.el related errors.  IIRC usually something
>  >>     about cl-defgeneric but I don't recall the precise text any more.
>  >>     probably a subgroup of the first one.
>  >>
>  >> - Something I certainly never saw before about invalid bytecode.
>  >
>  > These are all related to outdated *.elc files.
> 
> OK.  I never saw these in the past years.  Maybe I was just lucky.

Or maybe we have more people now working on macros that could
potentially break a build.

> Yes.  BTW, all rebuilds in the past minutes had configure run
> automatically and succeeded without problems.  So maybe the issue is
> resolved.

It should be.