* Building cvs emacs on a dual-core machine
@ 2006-12-20 11:47 Eric Lilja
2006-12-20 14:40 ` Kim F. Storm
` (2 more replies)
0 siblings, 3 replies; 29+ messages in thread
From: Eric Lilja @ 2006-12-20 11:47 UTC (permalink / raw)
Hi!
I recently upgraded my computer with a dual-core CPU. I've been enjoying
nice decreases in compile time in my own projects by specifying --jobs=2
when invoking make.
However, when I tried that with emacs it started compiling, very fast
with both cores busy, but the build craps out (sorry, no log) after a
little while. Maybe because it tries to build something that depends on
something else that has not been quite finished yet because when
multiple jobs are specified things doesn't happen in serial anymore? I
don't know, but I know it works if I exclude --jobs=2. But then emacs is
built and the total cpu utilization is only 50 percent. :-( Is there a
way I can utilise both cores when building emacs? I bought the new
processor mostly to get improved compile time performace. :-)
- Eric
^ permalink raw reply [flat|nested] 29+ messages in thread* Re: Building cvs emacs on a dual-core machine 2006-12-20 11:47 Building cvs emacs on a dual-core machine Eric Lilja @ 2006-12-20 14:40 ` Kim F. Storm 2006-12-20 19:05 ` Eli Zaretskii 2006-12-20 18:54 ` Eli Zaretskii 2006-12-20 22:19 ` Richard Stallman 2 siblings, 1 reply; 29+ messages in thread From: Kim F. Storm @ 2006-12-20 14:40 UTC (permalink / raw) Cc: emacs-devel Eric Lilja <mindcooler@gmail.com> writes: > However, when I tried that with emacs it started compiling, very fast > with both cores busy, but the build craps out (sorry, no log) after a > little while. Only a fractional part of the build is about compiling C files. Most of the time is spent inside emacs byte-compiling Lisp files, and when that is done, it starts loading and undumping. So most of the time, you have just one process running. > Maybe because it tries to build something that depends > on something else that has not been quite finished yet because when > multiple jobs are specified things doesn't happen in serial anymore? I > don't know, but I know it works if I exclude --jobs=2. But then emacs > is built and the total cpu utilization is only 50 percent. :-( Is > there a way I can utilise both cores when building emacs? Maybe you can build a GTK and a non-GTK version at the same time :-) -- Kim F. Storm <storm@cua.dk> http://www.cua.dk ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2006-12-20 14:40 ` Kim F. Storm @ 2006-12-20 19:05 ` Eli Zaretskii 2006-12-21 11:04 ` Kim F. Storm 2006-12-22 14:39 ` Chris Moore 0 siblings, 2 replies; 29+ messages in thread From: Eli Zaretskii @ 2006-12-20 19:05 UTC (permalink / raw) Cc: mindcooler, emacs-devel > From: storm@cua.dk (Kim F. Storm) > Date: Wed, 20 Dec 2006 15:40:22 +0100 > Cc: emacs-devel@gnu.org > > Only a fractional part of the build is about compiling C files. Perhaps I'm missing something, but I don't see the relevance of this, even if it were true that C compilation is only a small fraction of the build. There's nothing to prevent a parallel build from compiling 2 Lisp files at the same time, is there? > Most of the time is spent inside emacs byte-compiling Lisp files, Btw, byte compilation is only relevant to bootstrapping. A normal build of an Emacs tarball spends mots of its time in C compilation, I think. Don't you agree? ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2006-12-20 19:05 ` Eli Zaretskii @ 2006-12-21 11:04 ` Kim F. Storm 2006-12-22 14:39 ` Chris Moore 1 sibling, 0 replies; 29+ messages in thread From: Kim F. Storm @ 2006-12-21 11:04 UTC (permalink / raw) Cc: mindcooler, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> From: storm@cua.dk (Kim F. Storm) >> Date: Wed, 20 Dec 2006 15:40:22 +0100 >> Cc: emacs-devel@gnu.org >> >> Only a fractional part of the build is about compiling C files. > > Perhaps I'm missing something, but I don't see the relevance of this, > even if it were true that C compilation is only a small fraction of > the build. There's nothing to prevent a parallel build from compiling > 2 Lisp files at the same time, is there? No. Forget what I said. > >> Most of the time is spent inside emacs byte-compiling Lisp files, > > Btw, byte compilation is only relevant to bootstrapping. A normal > build of an Emacs tarball spends mots of its time in C compilation, I > think. Don't you agree? That is true, yes. -- Kim F. Storm <storm@cua.dk> http://www.cua.dk ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2006-12-20 19:05 ` Eli Zaretskii 2006-12-21 11:04 ` Kim F. Storm @ 2006-12-22 14:39 ` Chris Moore 2006-12-22 21:37 ` Eli Zaretskii 1 sibling, 1 reply; 29+ messages in thread From: Chris Moore @ 2006-12-22 14:39 UTC (permalink / raw) Cc: mindcooler, emacs-devel, Kim F. Storm Eli Zaretskii <eliz@gnu.org> writes: > There's nothing to prevent a parallel build from compiling 2 Lisp > files at the same time, is there? Yes, there is, at least on GNU/Linux make isn't used to determine which Lisp files to compile. During bootstrapping, it's done using the shell: [...] for el in [...] $els; do echo Compiling $el; [...]; done and during 'make recompile', it's done using emacs itself: ../src/emacs -batch --no-site-file --multibyte --eval (batch-byte-recompile-directory 0) /home/chris/programs/emacs2/lisp Neither of these will honour make's --jobs=2 flag. In fact using --jobs=2 makes the build ever so slightly slower on this dual-core machine. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2006-12-22 14:39 ` Chris Moore @ 2006-12-22 21:37 ` Eli Zaretskii 2006-12-22 23:39 ` Chris Moore 0 siblings, 1 reply; 29+ messages in thread From: Eli Zaretskii @ 2006-12-22 21:37 UTC (permalink / raw) Cc: mindcooler, emacs-devel, storm > Cc: storm@cua.dk (Kim F. Storm), mindcooler@gmail.com, emacs-devel@gnu.org > From: Chris Moore <dooglus@gmail.com> > Date: Fri, 22 Dec 2006 15:39:40 +0100 > > Eli Zaretskii <eliz@gnu.org> writes: > > > There's nothing to prevent a parallel build from compiling 2 Lisp > > files at the same time, is there? > > Yes, there is, at least on GNU/Linux make isn't used to determine > which Lisp files to compile. > > During bootstrapping, it's done using the shell: > > [...] for el in [...] $els; do echo Compiling $el; [...]; done > > and during 'make recompile', it's done using emacs itself: > > ../src/emacs -batch --no-site-file --multibyte --eval > (batch-byte-recompile-directory 0) /home/chris/programs/emacs2/lisp > > Neither of these will honour make's --jobs=2 flag. It's true that lisp/Makefile.in doesn't give much chance to exploit parallelism, but it's not true that no parallelism is possible there. `compile', `recompile', and `cvs-update' have prerequisites that can be produced in parallel. leim/Makefile.in is better organized for parallel make, and it's not inconceivable to have Emacs compile in parallel in lisp/ and in leim/. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2006-12-22 21:37 ` Eli Zaretskii @ 2006-12-22 23:39 ` Chris Moore 0 siblings, 0 replies; 29+ messages in thread From: Chris Moore @ 2006-12-22 23:39 UTC (permalink / raw) Cc: mindcooler, emacs-devel, storm Eli Zaretskii <eliz@gnu.org> writes: >> From: Chris Moore <dooglus@gmail.com> >> Eli Zaretskii <eliz@gnu.org> writes: >> >> > There's nothing to prevent a parallel build from compiling 2 Lisp >> > files at the same time, is there? >> >> Yes, there is, at least on GNU/Linux make isn't used to determine >> which Lisp files to compile. >> > It's true that lisp/Makefile.in doesn't give much chance to exploit > parallelism, but it's not true that no parallelism is possible there. > `compile', `recompile', and `cvs-update' have prerequisites that can > be produced in parallel. OK maybe it's possible, but it doesn't seem to happen to any significant extent. While compiling C code, both cores are used, and while compiling Lisp code one of them is pretty much idle. That's what I see, anyway. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2006-12-20 11:47 Building cvs emacs on a dual-core machine Eric Lilja 2006-12-20 14:40 ` Kim F. Storm @ 2006-12-20 18:54 ` Eli Zaretskii 2006-12-20 19:48 ` Eric Lilja 2006-12-20 22:19 ` Richard Stallman 2 siblings, 1 reply; 29+ messages in thread From: Eli Zaretskii @ 2006-12-20 18:54 UTC (permalink / raw) Cc: emacs-devel > From: Eric Lilja <mindcooler@gmail.com> > Date: Wed, 20 Dec 2006 12:47:13 +0100 > > I recently upgraded my computer with a dual-core CPU. I've been enjoying > nice decreases in compile time in my own projects by specifying --jobs=2 > when invoking make. > > However, when I tried that with emacs it started compiling, very fast > with both cores busy, but the build craps out (sorry, no log) after a > little while. Any such failures are bugs in the Emacs Makefiles, so please be sure to report them with full details, including the build log. If you didn't try this with the CVS version or with the latest pretest, please do try one of these, because many problems present in Emacs 21.x are already resolved in the latest development code. Thanks in advance. > Maybe because it tries to build something that depends on > something else that has not been quite finished yet because when > multiple jobs are specified things doesn't happen in serial anymore? A bug-free Makefile should never cause any such trouble, since all the dependencies are supposed to be spelled out in it, and Make will never start building a file until all its prerequisites are built. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2006-12-20 18:54 ` Eli Zaretskii @ 2006-12-20 19:48 ` Eric Lilja 2006-12-20 21:03 ` Eric Lilja 0 siblings, 1 reply; 29+ messages in thread From: Eric Lilja @ 2006-12-20 19:48 UTC (permalink / raw) Eli Zaretskii skrev: >> From: Eric Lilja <mindcooler@gmail.com> >> Date: Wed, 20 Dec 2006 12:47:13 +0100 >> >> I recently upgraded my computer with a dual-core CPU. I've been enjoying >> nice decreases in compile time in my own projects by specifying --jobs=2 >> when invoking make. >> >> However, when I tried that with emacs it started compiling, very fast >> with both cores busy, but the build craps out (sorry, no log) after a >> little while. > > Any such failures are bugs in the Emacs Makefiles, so please be sure > to report them with full details, including the build log. I will. This was with the cvs sources I downloaded yesterday (december 19th). I have been trying a few times since the beginning of november (that's when I got the dual core cpu), neither have been able to build when specifying --jobs=2. I will do a fresh checkout later today and hopefully provided you with the details you need. > > If you didn't try this with the CVS version or with the latest > pretest, please do try one of these, because many problems present in > Emacs 21.x are already resolved in the latest development code. > > Thanks in advance. > >> Maybe because it tries to build something that depends on >> something else that has not been quite finished yet because when >> multiple jobs are specified things doesn't happen in serial anymore? > > A bug-free Makefile should never cause any such trouble, since all the > dependencies are supposed to be spelled out in it, and Make will never > start building a file until all its prerequisites are built. - Eric ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2006-12-20 19:48 ` Eric Lilja @ 2006-12-20 21:03 ` Eric Lilja 2006-12-20 23:27 ` Eli Zaretskii 0 siblings, 1 reply; 29+ messages in thread From: Eric Lilja @ 2006-12-20 21:03 UTC (permalink / raw) Eric Lilja skrev: > Eli Zaretskii skrev: >>> From: Eric Lilja <mindcooler@gmail.com> >>> Date: Wed, 20 Dec 2006 12:47:13 +0100 >>> >>> I recently upgraded my computer with a dual-core CPU. I've been >>> enjoying nice decreases in compile time in my own projects by >>> specifying --jobs=2 when invoking make. >>> >>> However, when I tried that with emacs it started compiling, very fast >>> with both cores busy, but the build craps out (sorry, no log) after a >>> little while. >> >> Any such failures are bugs in the Emacs Makefiles, so please be sure >> to report them with full details, including the build log. > > I will. This was with the cvs sources I downloaded yesterday (december > 19th). I have been trying a few times since the beginning of november > (that's when I got the dual core cpu), neither have been able to build > when specifying --jobs=2. I will do a fresh checkout later today and > hopefully provided you with the details you need. Ok, here's where it errors out (same with --jobs=2 and --jobs=4): gcc -I. -DWIN32_LEAN_AND_MEAN -D_WIN32_WINNT=0x0400 -D_X86_=1 -c -mno-cygwin -mtune=pentium4 -O2 -Di386 -D_CRTAPI1 =_cdecl -o oo-spd/i386/preprep.o preprep.c gcc -o oo-spd/i386/preprep.exe \ -mno-cygwin oo-spd/i386/preprep.o Essential Lisp files seem to be missing. You should either do `make bootstrap' or create `lisp/abbrev.elc' somehow. mingw32-make: *** [maybe-bootstrap-SH] Error 255 Yes, sorry, I'm on Windows. This is from a fresh cvs checkout performed just now. My previous steps were: ./configure.bat --with-gcc --no-cygwin --no-debug mingw32-make --jobs=4 bootstrap You mentioned build log, what else information can I provide to help track this problem down? > >> >> If you didn't try this with the CVS version or with the latest >> pretest, please do try one of these, because many problems present in >> Emacs 21.x are already resolved in the latest development code. >> >> Thanks in advance. >> >>> Maybe because it tries to build something that depends on something >>> else that has not been quite finished yet because when multiple jobs >>> are specified things doesn't happen in serial anymore? >> >> A bug-free Makefile should never cause any such trouble, since all the >> dependencies are supposed to be spelled out in it, and Make will never >> start building a file until all its prerequisites are built. > > - Eric - Eric ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2006-12-20 21:03 ` Eric Lilja @ 2006-12-20 23:27 ` Eli Zaretskii 2006-12-21 0:14 ` Eric Lilja 0 siblings, 1 reply; 29+ messages in thread From: Eli Zaretskii @ 2006-12-20 23:27 UTC (permalink / raw) Cc: emacs-devel > From: Eric Lilja <mindcooler@gmail.com> > Date: Wed, 20 Dec 2006 22:03:56 +0100 > > Ok, here's where it errors out (same with --jobs=2 and --jobs=4): > gcc -I. -DWIN32_LEAN_AND_MEAN -D_WIN32_WINNT=0x0400 -D_X86_=1 -c > -mno-cygwin -mtune=pentium4 -O2 -Di386 -D_CRTAPI1 > =_cdecl -o oo-spd/i386/preprep.o preprep.c > gcc -o oo-spd/i386/preprep.exe \ > -mno-cygwin oo-spd/i386/preprep.o > > Essential Lisp files seem to be missing. You should either > do `make bootstrap' or create `lisp/abbrev.elc' somehow. > > mingw32-make: *** [maybe-bootstrap-SH] Error 255 > > Yes, sorry, I'm on Windows. That's an important piece of knowledge ;-) What version of Make do you have? What does "mingw32-make --version" print? The --jobs option started to work correctly on Windows only in the latest version 3.81 of GNU Make. If you don't have that version of Make, --jobs will be very unreliable on Windows. Also, note that, since the jobserver feature in Make is currently not suppoorted on Windows, the sub-Make's are invoked with the implied "--jobs=1" option, in effect defeating parallelism in recusrive Make invocations. You will have to tweak the recursive Make invocations in the Makefile's to explicitly pass the --jobs=2 option to recursive Make's, in order to see parallelism in subdirectories. Also, what port of sh.exe do you have? > You mentioned build log, what else information can I provide to help > track this problem down? Instead of just "mingw32-make --jobs=4 bootstrap" run this command (from the CMD prompt): mingw32-make --jobs=4 --debug=j bootstrap 2>&1 | tee build.txt The file build.txt will then capture all the output of the build session; post that file here. (I assume that you have a port of GNU `tee'; if not, you will have to install it, or capture the session output in some other way.) I see already a few problems with --jobs related to Windows, but I'd like to see the information requested above to make sure I don't miss anything. TIA ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2006-12-20 23:27 ` Eli Zaretskii @ 2006-12-21 0:14 ` Eric Lilja 2006-12-22 21:50 ` Eli Zaretskii 0 siblings, 1 reply; 29+ messages in thread From: Eric Lilja @ 2006-12-21 0:14 UTC (permalink / raw) [-- Attachment #1: Type: text/plain, Size: 2717 bytes --] Eli Zaretskii wrote: >> From: Eric Lilja <mindcooler@gmail.com> >> Date: Wed, 20 Dec 2006 22:03:56 +0100 >> >> Ok, here's where it errors out (same with --jobs=2 and --jobs=4): >> gcc -I. -DWIN32_LEAN_AND_MEAN -D_WIN32_WINNT=0x0400 -D_X86_=1 -c >> -mno-cygwin -mtune=pentium4 -O2 -Di386 -D_CRTAPI1 >> =_cdecl -o oo-spd/i386/preprep.o preprep.c >> gcc -o oo-spd/i386/preprep.exe \ >> -mno-cygwin oo-spd/i386/preprep.o >> >> Essential Lisp files seem to be missing. You should either >> do `make bootstrap' or create `lisp/abbrev.elc' somehow. >> >> mingw32-make: *** [maybe-bootstrap-SH] Error 255 >> >> Yes, sorry, I'm on Windows. > > That's an important piece of knowledge ;-) > > What version of Make do you have? What does "mingw32-make --version" > print? The --jobs option started to work correctly on Windows only in > the latest version 3.81 of GNU Make. If you don't have that version > of Make, --jobs will be very unreliable on Windows. $ mingw32-make --version GNU Make 3.81 Copyright (C) 2006 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. This program built for i686-pc-mingw32 > > Also, note that, since the jobserver feature in Make is currently not > suppoorted on Windows, the sub-Make's are invoked with the implied > "--jobs=1" option, in effect defeating parallelism in recusrive Make > invocations. You will have to tweak the recursive Make invocations in > the Makefile's to explicitly pass the --jobs=2 option to recursive > Make's, in order to see parallelism in subdirectories. > > Also, what port of sh.exe do you have? $ sh.exe --version GNU bash, version 3.2.9(10)-release (i686-pc-cygwin) Copyright (C) 2005 Free Software Foundation, Inc. > >> You mentioned build log, what else information can I provide to help >> track this problem down? > > Instead of just "mingw32-make --jobs=4 bootstrap" run this command > (from the CMD prompt): > > mingw32-make --jobs=4 --debug=j bootstrap 2>&1 | tee build.txt > > The file build.txt will then capture all the output of the build > session; post that file here. > > (I assume that you have a port of GNU `tee'; if not, you will have to > install it, or capture the session output in some other way.) > > I see already a few problems with --jobs related to Windows, but I'd > like to see the information requested above to make sure I don't miss > anything. I ran it and I'm attaching build.txt, but I ran it from TCI running bash instead of a regular cmd prompt, hope that I still was able to provide you with the information you needed. > > TIA - Eric [-- Attachment #2: build.txt --] [-- Type: text/plain, Size: 2621 bytes --] GNU Make 3.81 Copyright (C) 2006 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. This program built for i686-pc-mingw32 CreateProcess(C:\cygwin\bin\echo.exe,echo,...) CreateProcess(C:\cygwin\bin\sh.exe,C:/cygwin/bin/sh.exe -c "echo [Please ignore a syntax error on the next line - it is intentional] 1>&2",...) [Please ignore a syntax error on the next line - it is intentional] CreateProcess(C:\cygwin\bin\sh.exe,C:/cygwin/bin/sh.exe -c "echo foo\"",...) /usr/bin/sh: -c: line 0: unexpected EOF while looking for matching `"' /usr/bin/sh: -c: line 1: syntax error: unexpected end of file mkdir "oo-spd" CreateProcess(C:\cygwin\bin\sh.exe,C:/cygwin/bin/sh.exe -c "mkdir \"oo-spd\"",...) Putting child 0x009e7808 (oo-spd) PID 10413920 on the chain. CreateProcess(C:\cygwin\bin\echo.exe,echo Using C:/cygwin/bin/sh.exe as shell.,...) Putting child 0x009ef2c8 (which-sh) PID 10425384 on the chain. Using C:/cygwin/bin/sh.exe as shell. CreateProcess(C:\cygwin\bin\sh.exe,C:/cygwin/bin/sh.exe -c "if [ ! -f ../lisp/abbrev.elc ] ; then \ echo; \ echo \"Essential Lisp files seem to be missing. You should either\"; \ echo \"do \`make bootstrap' or create \`lisp/abbrev.elc' somehow.\"; \ echo; \ exit -1; \ fi",...) Putting child 0x009f2df0 (maybe-bootstrap-SH) PID 10441560 on the chain. Live child 0x009f2df0 (maybe-bootstrap-SH) PID 10441560 Live child 0x009ef2c8 (which-sh) PID 10425384 Live child 0x009e7808 (oo-spd) PID 10413920 Reaping winning child 0x009ef2c8 PID 10425384 Removing child 0x009ef2c8 PID 10425384 from chain. Live child 0x009f2df0 (maybe-bootstrap-SH) PID 10441560 Live child 0x009e7808 (oo-spd) PID 10413920 Essential Lisp files seem to be missing. You should either do `make bootstrap' or create `lisp/abbrev.elc' somehow. Reaping winning child 0x009e7808 PID 10413920 Removing child 0x009e7808 PID 10413920 from chain. mkdir "oo-spd/i386" CreateProcess(C:\cygwin\bin\sh.exe,C:/cygwin/bin/sh.exe -c "mkdir \"oo-spd/i386\"",...) Putting child 0x009f0f98 (oo-spd/i386) PID 10413920 on the chain. Live child 0x009f0f98 (oo-spd/i386) PID 10413920 Live child 0x009f2df0 (maybe-bootstrap-SH) PID 10441560 Reaping losing child 0x009f2df0 PID 10441560 mingw32-make: *** [maybe-bootstrap-SH] Error 255 Removing child 0x009f2df0 PID 10441560 from chain. mingw32-make: *** Waiting for unfinished jobs.... Live child 0x009f0f98 (oo-spd/i386) PID 10413920 Reaping winning child 0x009f0f98 PID 10413920 Removing child 0x009f0f98 PID 10413920 from chain. [-- Attachment #3: Type: text/plain, Size: 142 bytes --] _______________________________________________ Emacs-devel mailing list Emacs-devel@gnu.org http://lists.gnu.org/mailman/listinfo/emacs-devel ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2006-12-21 0:14 ` Eric Lilja @ 2006-12-22 21:50 ` Eli Zaretskii 2006-12-22 23:30 ` Eric Lilja 2006-12-23 1:54 ` Juanma Barranquero 0 siblings, 2 replies; 29+ messages in thread From: Eli Zaretskii @ 2006-12-22 21:50 UTC (permalink / raw) Cc: emacs-devel > From: Eric Lilja <mindcooler@gmail.com> > Date: Thu, 21 Dec 2006 01:14:45 +0100 > > Eli Zaretskii wrote: > >> From: Eric Lilja <mindcooler@gmail.com> > >> Date: Wed, 20 Dec 2006 22:03:56 +0100 > >> > >> Ok, here's where it errors out (same with --jobs=2 and --jobs=4): > >> gcc -I. -DWIN32_LEAN_AND_MEAN -D_WIN32_WINNT=0x0400 -D_X86_=1 -c > >> -mno-cygwin -mtune=pentium4 -O2 -Di386 -D_CRTAPI1 > >> =_cdecl -o oo-spd/i386/preprep.o preprep.c > >> gcc -o oo-spd/i386/preprep.exe \ > >> -mno-cygwin oo-spd/i386/preprep.o > >> > >> Essential Lisp files seem to be missing. You should either > >> do `make bootstrap' or create `lisp/abbrev.elc' somehow. > >> > >> mingw32-make: *** [maybe-bootstrap-SH] Error 255 > >> > >> Yes, sorry, I'm on Windows. > > > > That's an important piece of knowledge ;-) > > > > What version of Make do you have? What does "mingw32-make --version" > > print? The --jobs option started to work correctly on Windows only in > > the latest version 3.81 of GNU Make. If you don't have that version > > of Make, --jobs will be very unreliable on Windows. > > $ mingw32-make --version > GNU Make 3.81 I found and fixed several problems with Windows makefile's which prevented parallel builds. Please resync with CVS and try again, I hope it will work now. Note that, since the native Windows build of Make (including mingw32-make) doesn't yet support the jobserver feature, it by default invokes sub-Make's with --jobs=1 switch, and thus defeats parallelism in recursive Make invocations. I made a provision for working around this; to use it, invoke the top-level Make in the nt/ subdirectory like this: mingw32-make bootstrap --jobs=2 XMFLAGS="--jobs=2" The XMFLAGS variable is used to pass an explicit --jobs option to sub-Make's, overriding the default --jobs=1. (There's no requirement that the argument to --jobs on the command line must be identical to what you put in XMFLAGS, you can use different numbers if you wish.) Please tell if you have any additional problems with parallel builds on Windows. And thanks again for drawing our attention to this. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2006-12-22 21:50 ` Eli Zaretskii @ 2006-12-22 23:30 ` Eric Lilja 2006-12-23 1:50 ` Eli Zaretskii 2006-12-23 1:54 ` Juanma Barranquero 1 sibling, 1 reply; 29+ messages in thread From: Eric Lilja @ 2006-12-22 23:30 UTC (permalink / raw) Eli Zaretskii skrev: >> From: Eric Lilja <mindcooler@gmail.com> >> Date: Thu, 21 Dec 2006 01:14:45 +0100 >> >> Eli Zaretskii wrote: >>>> From: Eric Lilja <mindcooler@gmail.com> >>>> Date: Wed, 20 Dec 2006 22:03:56 +0100 >>>> >>>> Ok, here's where it errors out (same with --jobs=2 and --jobs=4): >>>> gcc -I. -DWIN32_LEAN_AND_MEAN -D_WIN32_WINNT=0x0400 -D_X86_=1 -c >>>> -mno-cygwin -mtune=pentium4 -O2 -Di386 -D_CRTAPI1 >>>> =_cdecl -o oo-spd/i386/preprep.o preprep.c >>>> gcc -o oo-spd/i386/preprep.exe \ >>>> -mno-cygwin oo-spd/i386/preprep.o >>>> >>>> Essential Lisp files seem to be missing. You should either >>>> do `make bootstrap' or create `lisp/abbrev.elc' somehow. >>>> >>>> mingw32-make: *** [maybe-bootstrap-SH] Error 255 >>>> >>>> Yes, sorry, I'm on Windows. >>> That's an important piece of knowledge ;-) >>> >>> What version of Make do you have? What does "mingw32-make --version" >>> print? The --jobs option started to work correctly on Windows only in >>> the latest version 3.81 of GNU Make. If you don't have that version >>> of Make, --jobs will be very unreliable on Windows. >> $ mingw32-make --version >> GNU Make 3.81 > > I found and fixed several problems with Windows makefile's which > prevented parallel builds. Please resync with CVS and try again, I > hope it will work now. > > Note that, since the native Windows build of Make (including > mingw32-make) doesn't yet support the jobserver feature, it by default > invokes sub-Make's with --jobs=1 switch, and thus defeats parallelism > in recursive Make invocations. I made a provision for working around > this; to use it, invoke the top-level Make in the nt/ subdirectory > like this: > > mingw32-make bootstrap --jobs=2 XMFLAGS="--jobs=2" > > The XMFLAGS variable is used to pass an explicit --jobs option to > sub-Make's, overriding the default --jobs=1. (There's no requirement > that the argument to --jobs on the command line must be identical to > what you put in XMFLAGS, you can use different numbers if you wish.) > > Please tell if you have any additional problems with parallel builds > on Windows. > > And thanks again for drawing our attention to this. I tried your changes on a fresh checkout (sorry for using time but I thought it would work and wanted to see how big decrease in compile time I would get with your fixes): $ time mingw32-make bootstrap --jobs=2 XMFLAGS="--jobs=2" [Please ignore a syntax error on the next line - it is intentional] /usr/bin/sh: -c: line 0: unexpected EOF while looking for matching `"' /usr/bin/sh: -c: line 1: syntax error: unexpected end of file mkdir "oo-spd" mkdir "oo-spd/i386" echo oo-spd/i386 > stamp_BLD gcc -I. -DWIN32_LEAN_AND_MEAN -D_WIN32_WINNT=0x0400 -D_X86_=1 -c -mno-cygwin -mtune=pentium4 -O2 -Di386 -D_CRTAPI1 =_cdecl -o oo-spd/i386/addsection.o addsection.c gcc -o oo-spd/i386/addsection.exe \ -mno-cygwin oo-spd/i386/addsection.o -luser32 c:/mingw32-make-3.81-1/bin/mingw32-make -j 1 --jobs=2 -C ../lisp bootstrap-clean [Please ignore a syntax error on the next line - it is intentional] /usr/bin/sh: -c: line 0: unexpected EOF while looking for matching `"' /usr/bin/sh: -c: line 1: syntax error: unexpected end of file mingw32-make[1]: Entering directory `c:/cvsemacs/emacs/lisp' cp c:/cvsemacs/emacs/lisp/ldefs-boot.el c:/cvsemacs/emacs/lisp/loaddefs.el c:/mingw32-make-3.81-1/bin/mingw32-make -wj 1 loaddefs.el-SH for dir in . calc calendar emacs-lisp emulation erc eshell gnus international language mail mh-e net play progmodes term textmodes url obsolete; do rm -f $dir/*.elc; done [Please ignore a syntax error on the next line - it is intentional] /usr/bin/sh: -c: line 0: unexpected EOF while looking for matching `"' /usr/bin/sh: -c: line 1: syntax error: unexpected end of file mingw32-make[2]: Entering directory `c:/cvsemacs/emacs/lisp' echo ";;; loaddefs.el --- automatically extracted autoloads" > loaddefs.el-SH echo ";;" >> loaddefs.el-SH; echo ";;; Code:" >> loaddefs.el-SH echo "(autoload 'define-minor-mode \"easy-mmode\")" >>loaddefs.el-SH echo "(autoload 'define-ccl-program \"ccl\")" >>loaddefs.el-SH echo "(autoload 'regexp-opt \"regexp-opt\")" >>loaddefs.el-SH echo "(autoload 'string-to-list \"mule-util\")" >>loaddefs.el-SH echo "(autoload 'define-derived-mode \"derived\")" >>loaddefs.el-SH echo "(autoload 'encoded-kbd-mode \"encoded-kb\")" >>loaddefs.el-SH echo "(defvar cvs-global-menu nil)" >>loaddefs.el-SH echo "♀" >> loaddefs.el-SH echo ";;; Local Variables:" >> loaddefs.el-SH echo ";;; version-control: never" >> loaddefs.el-SH echo ";;; no-byte-compile: t" >> loaddefs.el-SH echo ";;; no-update-autoloads: t" >> loaddefs.el-SH echo ";;; End:" >> loaddefs.el-SH echo ";;; loaddefs.el ends here" >> loaddefs.el-SH mingw32-make[2]: Leaving directory `c:/cvsemacs/emacs/lisp' cp loaddefs.el-SH c:/cvsemacs/emacs/lisp/loaddefs.el rm loaddefs.el-SH mingw32-make[1]: Leaving directory `c:/cvsemacs/emacs/lisp' c:/mingw32-make-3.81-1/bin/mingw32-make -j 1 --jobs=2 -C ../src clean [Please ignore a syntax error on the next line - it is intentional] /usr/bin/sh: -c: line 0: unexpected EOF while looking for matching `"' /usr/bin/sh: -c: line 1: syntax error: unexpected end of file mingw32-make[1]: Entering directory `c:/cvsemacs/emacs/src' rm *~ "s/*~" "m/*~" rm: cannot remove `*~': No such file or directory rm: cannot remove `s/*~': No such file or directory rm: cannot remove `m/*~': No such file or directory mingw32-make[1]: [clean] Error 1 (ignored) rm rm: missing operand Try `rm --help' for more information. mingw32-make[1]: [clean] Error 1 (ignored) rm -r oo-spd rm: cannot remove `oo-spd': No such file or directory mingw32-make[1]: [clean] Error 1 (ignored) rm stamp_BLD rm: cannot remove `stamp_BLD': No such file or directory mingw32-make[1]: [clean] Error 1 (ignored) mingw32-make[1]: Leaving directory `c:/cvsemacs/emacs/src' c:/mingw32-make-3.81-1/bin/mingw32-make -j 1 --jobs=2 -C ../lib-src clean [Please ignore a syntax error on the next line - it is intentional] /usr/bin/sh: -c: line 0: unexpected EOF while looking for matching `"' /usr/bin/sh: -c: line 1: syntax error: unexpected end of file mingw32-make[1]: Entering directory `c:/cvsemacs/emacs/lib-src' rm *~ DOC* rm: cannot remove `*~': No such file or directory rm: cannot remove `DOC*': No such file or directory mingw32-make[1]: [clean] Error 1 (ignored) rm ctags.c rm: cannot remove `ctags.c': No such file or directory mingw32-make[1]: [clean] Error 1 (ignored) rm getopt.h rm: cannot remove `getopt.h': No such file or directory mingw32-make[1]: [clean] Error 1 (ignored) rm -r oo-spd rm: cannot remove `oo-spd': No such file or directory mingw32-make[1]: [clean] Error 1 (ignored) rm stamp_BLD rm: cannot remove `stamp_BLD': No such file or directory mingw32-make[1]: [clean] Error 1 (ignored) mingw32-make[1]: Leaving directory `c:/cvsemacs/emacs/lib-src' c:/mingw32-make-3.81-1/bin/mingw32-make -j 1 --jobs=2 XMFLAGS="--jobs=2" -C ../src bootstrap [Please ignore a syntax error on the next line - it is intentional] /usr/bin/sh: -c: line 0: unexpected EOF while looking for matching `"' /usr/bin/sh: -c: line 1: syntax error: unexpected end of file mingw32-make[1]: Entering directory `c:/cvsemacs/emacs/nt' mingw32-make[1]: Nothing to be done for `../src'. c:/mingw32-make-3.81-1/bin/mingw32-make -wj 1 --jobs=2 -C -C ../lisp bootstrap-clean mingw32-make: Entering an unknown directory mingw32-make: *** -C: No such file or directory. Stop. mingw32-make: Leaving an unknown directory mingw32-make[1]: *** [bootstrap-gmake] Error 2 mingw32-make[1]: Leaving directory `c:/cvsemacs/emacs/nt' mingw32-make: *** [bootstrap-gmake] Error 2 real 0m5.093s user 0m0.015s sys 0m0.000s hivemind@mindcooler /cygdrive/c/cvsemacs/emacs/nt $ Also, doing an ordinary bootstrap without specifying jobs or XMFLAGS doesn't seem to work right now either. - Eric ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2006-12-22 23:30 ` Eric Lilja @ 2006-12-23 1:50 ` Eli Zaretskii 2006-12-23 2:04 ` Eric Lilja 0 siblings, 1 reply; 29+ messages in thread From: Eli Zaretskii @ 2006-12-23 1:50 UTC (permalink / raw) Cc: emacs-devel > From: Eric Lilja <mindcooler@gmail.com> > Date: Sat, 23 Dec 2006 00:30:54 +0100 > > c:/mingw32-make-3.81-1/bin/mingw32-make -wj 1 --jobs=2 -C -C ../lisp bootstrap-clean > mingw32-make: Entering an unknown directory > mingw32-make: *** -C: No such file or directory. Stop. Fascinating... where did that second -C come from? Can you try to find out? > Also, doing an ordinary bootstrap without specifying jobs or XMFLAGS > doesn't seem to work right now either. Please show a the error message in that case. Bootstrap works for me, both with -j 2 and without it, but I don't have a dual core machine, only a hyper-threaded one. Also, could you please invoke Make from CMD prompt, not from the Cygwin shell? Ordinary bootstrap works for me, I just verified that again. Did you try in a clean tree, or in the one where the -j 2 bootstrap failed? If the latter, perhaps the cleaning targets don't do a good enough job; please try in a clean tree. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2006-12-23 1:50 ` Eli Zaretskii @ 2006-12-23 2:04 ` Eric Lilja 2006-12-23 10:42 ` Eli Zaretskii 0 siblings, 1 reply; 29+ messages in thread From: Eric Lilja @ 2006-12-23 2:04 UTC (permalink / raw) Eli Zaretskii skrev: >> From: Eric Lilja <mindcooler@gmail.com> >> Date: Sat, 23 Dec 2006 00:30:54 +0100 >> >> c:/mingw32-make-3.81-1/bin/mingw32-make -wj 1 --jobs=2 -C -C ../lisp bootstrap-clean >> mingw32-make: Entering an unknown directory >> mingw32-make: *** -C: No such file or directory. Stop. > > Fascinating... where did that second -C come from? Can you try to > find out? > >> Also, doing an ordinary bootstrap without specifying jobs or XMFLAGS >> doesn't seem to work right now either. > > Please show a the error message in that case. Bootstrap works for me, > both with -j 2 and without it, but I don't have a dual core machine, > only a hyper-threaded one. > > Also, could you please invoke Make from CMD prompt, not from the > Cygwin shell? > > Ordinary bootstrap works for me, I just verified that again. Did you > try in a clean tree, or in the one where the -j 2 bootstrap failed? > If the latter, perhaps the cleaning targets don't do a good enough > job; please try in a clean tree. I did a fresh checkout again. I cannot build emacs anymore. I didn't specify jobs at all. I tried both cmd.com and cygwin's bash. cmd.com yields: C:\cvsemacs\emacs\nt>mingw32-make bootstrap mkdir "oo-spd" mkdir "oo-spd/i386" echo oo-spd/i386 > stamp_BLD gcc -I. -DWIN32_LEAN_AND_MEAN -D_WIN32_WINNT=0x0400 -D_X86_=1 -c -mno-cygwin -mtune=pentium4 -O2 -Di386 -D_CRTAPI1=_cdecl -o oo-spd/i386/addsection.o addsection.c process_begin: CreateProcess(NULL, gcc -I. -DWIN32_LEAN_AND_MEAN -D_WIN32_WINNT=0x0400 -D_X86_=1 -c -mno-cygwin -mtune=pentium4 -O2 -Di386 -D_CRTAPI1=_cdecl -o oo-spd/i386/addsection.o addsection.c, ...) failed. make (e=2): The system cannot find the file specified. mingw32-make: *** [oo-spd/i386/addsection.o] Error 2 (I had to configure from bash because configure.bat said it needed "cp") From bash: $ mingw32-make bootstrap [Please ignore a syntax error on the next line - it is intentional] /usr/bin/sh: -c: line 0: unexpected EOF while looking for matching `"' /usr/bin/sh: -c: line 1: syntax error: unexpected end of file gcc -I. -DWIN32_LEAN_AND_MEAN -D_WIN32_WINNT=0x0400 -D_X86_=1 -c -mno-cygwin -mtune=pentium4 -O2 -Di386 -D_CRTAPI1 =_cdecl -o oo-spd/i386/addsection.o addsection.c gcc -o oo-spd/i386/addsection.exe \ -mno-cygwin oo-spd/i386/addsection.o -luser32 c:/mingw32-make-3.81-1/bin/mingw32-make -C ../lisp bootstrap-clean [Please ignore a syntax error on the next line - it is intentional] /usr/bin/sh: -c: line 0: unexpected EOF while looking for matching `"' /usr/bin/sh: -c: line 1: syntax error: unexpected end of file mingw32-make[1]: Entering directory `c:/cvsemacs/emacs/lisp' cp c:/cvsemacs/emacs/lisp/ldefs-boot.el c:/cvsemacs/emacs/lisp/loaddefs.el for dir in . calc calendar emacs-lisp emulation erc eshell gnus international language mail mh-e net play progmodes term textmodes url obsolete; do rm -f $dir/*.elc; done mingw32-make[1]: Leaving directory `c:/cvsemacs/emacs/lisp' c:/mingw32-make-3.81-1/bin/mingw32-make -C ../src clean [Please ignore a syntax error on the next line - it is intentional] /usr/bin/sh: -c: line 0: unexpected EOF while looking for matching `"' /usr/bin/sh: -c: line 1: syntax error: unexpected end of file mingw32-make[1]: Entering directory `c:/cvsemacs/emacs/src' rm *~ "s/*~" "m/*~" rm: cannot remove `*~': No such file or directory rm: cannot remove `s/*~': No such file or directory rm: cannot remove `m/*~': No such file or directory mingw32-make[1]: [clean] Error 1 (ignored) rm rm: missing operand Try `rm --help' for more information. mingw32-make[1]: [clean] Error 1 (ignored) rm -r oo-spd rm: cannot remove `oo-spd': No such file or directory mingw32-make[1]: [clean] Error 1 (ignored) rm stamp_BLD rm: cannot remove `stamp_BLD': No such file or directory mingw32-make[1]: [clean] Error 1 (ignored) mingw32-make[1]: Leaving directory `c:/cvsemacs/emacs/src' c:/mingw32-make-3.81-1/bin/mingw32-make -C ../lib-src clean [Please ignore a syntax error on the next line - it is intentional] /usr/bin/sh: -c: line 0: unexpected EOF while looking for matching `"' /usr/bin/sh: -c: line 1: syntax error: unexpected end of file mingw32-make[1]: Entering directory `c:/cvsemacs/emacs/lib-src' rm *~ DOC* rm: cannot remove `*~': No such file or directory rm: cannot remove `DOC*': No such file or directory mingw32-make[1]: [clean] Error 1 (ignored) rm ctags.c rm: cannot remove `ctags.c': No such file or directory mingw32-make[1]: [clean] Error 1 (ignored) rm getopt.h rm: cannot remove `getopt.h': No such file or directory mingw32-make[1]: [clean] Error 1 (ignored) rm -r oo-spd rm: cannot remove `oo-spd': No such file or directory mingw32-make[1]: [clean] Error 1 (ignored) rm stamp_BLD rm: cannot remove `stamp_BLD': No such file or directory mingw32-make[1]: [clean] Error 1 (ignored) mingw32-make[1]: Leaving directory `c:/cvsemacs/emacs/lib-src' c:/mingw32-make-3.81-1/bin/mingw32-make XMFLAGS="" -C ../src bootstrap [Please ignore a syntax error on the next line - it is intentional] /usr/bin/sh: -c: line 0: unexpected EOF while looking for matching `"' /usr/bin/sh: -c: line 1: syntax error: unexpected end of file mingw32-make[1]: Entering directory `c:/cvsemacs/emacs/src' c:/mingw32-make-3.81-1/bin/mingw32-make -w \ temacs CFLAGS='-I. -DWIN32_LEAN_AND_MEAN -D_WIN32_WINNT=0x0400 -D_X86_= 1 -c -mno-cygwin -mtune=pentium4 -O2 -Di386 -D_CRTAPI1=_cdecl -Demacs=1 -DWINDOWSNT -DDOS_NT -DHAVE_CONFIG_H -I. ./nt/inc -D_UCHAR_T -DHAVE_NTGUI=1 -DUSE_CRT_DLL=1 -DPURESIZE=5000000' [Please ignore a syntax error on the next line - it is intentional] /usr/bin/sh: -c: line 0: unexpected EOF while looking for matching `"' /usr/bin/sh: -c: line 1: syntax error: unexpected end of file mingw32-make[2]: Entering directory `c:/cvsemacs/emacs/src' mingw32-make[2]: *** No rule to make target ` temacs'. Stop. mingw32-make[2]: Leaving directory `c:/cvsemacs/emacs/src' mingw32-make[1]: *** [bootstrap-temacs] Error 2 mingw32-make[1]: Leaving directory `c:/cvsemacs/emacs/src' mingw32-make: *** [bootstrap-gmake] Error 2 Hope this helps, I won't be able to respond for a few hours, it's past 3 AM here and I going to visist my parents for holidays tomorrow. - Eric ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2006-12-23 2:04 ` Eric Lilja @ 2006-12-23 10:42 ` Eli Zaretskii 2006-12-23 10:59 ` Eric Lilja 0 siblings, 1 reply; 29+ messages in thread From: Eli Zaretskii @ 2006-12-23 10:42 UTC (permalink / raw) Cc: emacs-devel > From: Eric Lilja <mindcooler@gmail.com> > Date: Sat, 23 Dec 2006 03:04:30 +0100 > > I cannot build emacs anymore. Sorry to hear that, but I cannot reproduce any of the failures on my machine, it still bootstraps and builds for me, both with and without "--jobs". > I didn't specify jobs at all. I tried both cmd.com and cygwin's > bash. cmd.com yields: > C:\cvsemacs\emacs\nt>mingw32-make bootstrap > mkdir "oo-spd" > mkdir "oo-spd/i386" > echo oo-spd/i386 > stamp_BLD > gcc -I. -DWIN32_LEAN_AND_MEAN -D_WIN32_WINNT=0x0400 -D_X86_=1 -c > -mno-cygwin -mtune=pentium4 -O2 -Di386 -D_CRTAPI1=_cdecl -o > oo-spd/i386/addsection.o addsection.c > process_begin: CreateProcess(NULL, gcc -I. -DWIN32_LEAN_AND_MEAN > -D_WIN32_WINNT=0x0400 -D_X86_=1 -c -mno-cygwin -mtune=pentium4 -O2 > -Di386 -D_CRTAPI1=_cdecl -o > oo-spd/i386/addsection.o addsection.c, ...) failed. > make (e=2): The system cannot find the file specified. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ I feel terribly stupid, because I cannot figure out what is the file it doesn't find here. It cannot be the oo-spd/i386 directory since it was just created before the compilation command. And it cannot be addsection.c, because you are in the nt/ subdirectory, where addsection.c lives. Could you please try to find out what file is missing? > From bash: > $ mingw32-make bootstrap > [...] > mingw32-make[1]: Leaving directory `c:/cvsemacs/emacs/lib-src' > c:/mingw32-make-3.81-1/bin/mingw32-make XMFLAGS="" -C ../src bootstrap > [Please ignore a syntax error on the next line - it is intentional] > /usr/bin/sh: -c: line 0: unexpected EOF while looking for matching `"' > /usr/bin/sh: -c: line 1: syntax error: unexpected end of file > mingw32-make[1]: Entering directory `c:/cvsemacs/emacs/src' > c:/mingw32-make-3.81-1/bin/mingw32-make -w \ temacs CFLAGS='-I. ^^^^^^^^ > -DWIN32_LEAN_AND_MEAN -D_WIN32_WINNT=0x0400 -D_X86_= > 1 -c -mno-cygwin -mtune=pentium4 -O2 -Di386 -D_CRTAPI1=_cdecl > -Demacs=1 -DWINDOWSNT -DDOS_NT -DHAVE_CONFIG_H -I. > ./nt/inc -D_UCHAR_T -DHAVE_NTGUI=1 -DUSE_CRT_DLL=1 -DPURESIZE=5000000' > [Please ignore a syntax error on the next line - it is intentional] > /usr/bin/sh: -c: line 0: unexpected EOF while looking for matching `"' > /usr/bin/sh: -c: line 1: syntax error: unexpected end of file > mingw32-make[2]: Entering directory `c:/cvsemacs/emacs/src' > mingw32-make[2]: *** No rule to make target ` temacs'. Stop. ^^^^^^^^ I think I can understand this one; sounds like some bug in the MinGW make and/or in its interaction with the Cygwin Bash. Note the lone backslash above and the leading blank in ` temacs'. I made some changes to work around this; please resync and try again. (Needless to say, it still bootstraps for me, at least without --jobs. Will try with --jobs shortly.) ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2006-12-23 10:42 ` Eli Zaretskii @ 2006-12-23 10:59 ` Eric Lilja 2006-12-23 11:24 ` Eric Lilja 0 siblings, 1 reply; 29+ messages in thread From: Eric Lilja @ 2006-12-23 10:59 UTC (permalink / raw) Eli Zaretskii skrev: >> From: Eric Lilja <mindcooler@gmail.com> >> Date: Sat, 23 Dec 2006 03:04:30 +0100 >> >> I cannot build emacs anymore. > > Sorry to hear that, but I cannot reproduce any of the failures on my > machine, it still bootstraps and builds for me, both with and without > "--jobs". > >> I didn't specify jobs at all. I tried both cmd.com and cygwin's >> bash. cmd.com yields: >> C:\cvsemacs\emacs\nt>mingw32-make bootstrap >> mkdir "oo-spd" >> mkdir "oo-spd/i386" >> echo oo-spd/i386 > stamp_BLD >> gcc -I. -DWIN32_LEAN_AND_MEAN -D_WIN32_WINNT=0x0400 -D_X86_=1 -c >> -mno-cygwin -mtune=pentium4 -O2 -Di386 -D_CRTAPI1=_cdecl -o >> oo-spd/i386/addsection.o addsection.c >> process_begin: CreateProcess(NULL, gcc -I. -DWIN32_LEAN_AND_MEAN >> -D_WIN32_WINNT=0x0400 -D_X86_=1 -c -mno-cygwin -mtune=pentium4 -O2 >> -Di386 -D_CRTAPI1=_cdecl -o >> oo-spd/i386/addsection.o addsection.c, ...) failed. >> make (e=2): The system cannot find the file specified. > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > I feel terribly stupid, because I cannot figure out what is the file > it doesn't find here. It cannot be the oo-spd/i386 directory since it > was just created before the compilation command. And it cannot be > addsection.c, because you are in the nt/ subdirectory, where > addsection.c lives. Could you please try to find out what file is > missing? > >> From bash: >> $ mingw32-make bootstrap >> [...] >> mingw32-make[1]: Leaving directory `c:/cvsemacs/emacs/lib-src' >> c:/mingw32-make-3.81-1/bin/mingw32-make XMFLAGS="" -C ../src bootstrap >> [Please ignore a syntax error on the next line - it is intentional] >> /usr/bin/sh: -c: line 0: unexpected EOF while looking for matching `"' >> /usr/bin/sh: -c: line 1: syntax error: unexpected end of file >> mingw32-make[1]: Entering directory `c:/cvsemacs/emacs/src' >> c:/mingw32-make-3.81-1/bin/mingw32-make -w \ temacs CFLAGS='-I. > ^^^^^^^^ >> -DWIN32_LEAN_AND_MEAN -D_WIN32_WINNT=0x0400 -D_X86_= >> 1 -c -mno-cygwin -mtune=pentium4 -O2 -Di386 -D_CRTAPI1=_cdecl >> -Demacs=1 -DWINDOWSNT -DDOS_NT -DHAVE_CONFIG_H -I. >> ./nt/inc -D_UCHAR_T -DHAVE_NTGUI=1 -DUSE_CRT_DLL=1 -DPURESIZE=5000000' >> [Please ignore a syntax error on the next line - it is intentional] >> /usr/bin/sh: -c: line 0: unexpected EOF while looking for matching `"' >> /usr/bin/sh: -c: line 1: syntax error: unexpected end of file >> mingw32-make[2]: Entering directory `c:/cvsemacs/emacs/src' >> mingw32-make[2]: *** No rule to make target ` temacs'. Stop. > ^^^^^^^^ > I think I can understand this one; sounds like some bug in the MinGW > make and/or in its interaction with the Cygwin Bash. Note the lone > backslash above and the leading blank in ` temacs'. > > I made some changes to work around this; please resync and try again. > (Needless to say, it still bootstraps for me, at least without --jobs. > Will try with --jobs shortly.) I just performed a fresh checkout and managed to complete the entire build bootstrap process without problem (this one without specifying --jobs=2). The only thing different from last night was that I had upgraded TCI (an alternative to rxvt) to the latest beta but that shouldn't make a difference I think. I did this checkout before I saw the message I'm replying to now, don't know if I got the changes or not but it worked now. I will do a rebuild again and time it this time and then time another rebuild this time with --jobs=2 and see if both can complete successfully (and their respective times will be interesting for me to compare). - Eric ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2006-12-23 10:59 ` Eric Lilja @ 2006-12-23 11:24 ` Eric Lilja 2006-12-23 12:46 ` Eli Zaretskii 0 siblings, 1 reply; 29+ messages in thread From: Eric Lilja @ 2006-12-23 11:24 UTC (permalink / raw) Eric Lilja skrev: > Eli Zaretskii skrev: >>> From: Eric Lilja <mindcooler@gmail.com> >>> Date: Sat, 23 Dec 2006 03:04:30 +0100 >>> >>> I cannot build emacs anymore. >> >> Sorry to hear that, but I cannot reproduce any of the failures on my >> machine, it still bootstraps and builds for me, both with and without >> "--jobs". >> >>> I didn't specify jobs at all. I tried both cmd.com and cygwin's >>> bash. cmd.com yields: >>> C:\cvsemacs\emacs\nt>mingw32-make bootstrap >>> mkdir "oo-spd" >>> mkdir "oo-spd/i386" >>> echo oo-spd/i386 > stamp_BLD >>> gcc -I. -DWIN32_LEAN_AND_MEAN -D_WIN32_WINNT=0x0400 -D_X86_=1 -c >>> -mno-cygwin -mtune=pentium4 -O2 -Di386 -D_CRTAPI1=_cdecl -o >>> oo-spd/i386/addsection.o addsection.c >>> process_begin: CreateProcess(NULL, gcc -I. -DWIN32_LEAN_AND_MEAN >>> -D_WIN32_WINNT=0x0400 -D_X86_=1 -c -mno-cygwin -mtune=pentium4 -O2 >>> -Di386 -D_CRTAPI1=_cdecl -o >>> oo-spd/i386/addsection.o addsection.c, ...) failed. >>> make (e=2): The system cannot find the file specified. >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> I feel terribly stupid, because I cannot figure out what is the file >> it doesn't find here. It cannot be the oo-spd/i386 directory since it >> was just created before the compilation command. And it cannot be >> addsection.c, because you are in the nt/ subdirectory, where >> addsection.c lives. Could you please try to find out what file is >> missing? >> >>> From bash: >>> $ mingw32-make bootstrap >>> [...] >>> mingw32-make[1]: Leaving directory `c:/cvsemacs/emacs/lib-src' >>> c:/mingw32-make-3.81-1/bin/mingw32-make XMFLAGS="" -C ../src bootstrap >>> [Please ignore a syntax error on the next line - it is intentional] >>> /usr/bin/sh: -c: line 0: unexpected EOF while looking for matching `"' >>> /usr/bin/sh: -c: line 1: syntax error: unexpected end of file >>> mingw32-make[1]: Entering directory `c:/cvsemacs/emacs/src' >>> c:/mingw32-make-3.81-1/bin/mingw32-make -w \ temacs CFLAGS='-I. >> ^^^^^^^^ >>> -DWIN32_LEAN_AND_MEAN -D_WIN32_WINNT=0x0400 -D_X86_= >>> 1 -c -mno-cygwin -mtune=pentium4 -O2 -Di386 -D_CRTAPI1=_cdecl >>> -Demacs=1 -DWINDOWSNT -DDOS_NT -DHAVE_CONFIG_H -I. >>> ./nt/inc -D_UCHAR_T -DHAVE_NTGUI=1 -DUSE_CRT_DLL=1 -DPURESIZE=5000000' >>> [Please ignore a syntax error on the next line - it is intentional] >>> /usr/bin/sh: -c: line 0: unexpected EOF while looking for matching `"' >>> /usr/bin/sh: -c: line 1: syntax error: unexpected end of file >>> mingw32-make[2]: Entering directory `c:/cvsemacs/emacs/src' >>> mingw32-make[2]: *** No rule to make target ` temacs'. Stop. >> ^^^^^^^^ >> I think I can understand this one; sounds like some bug in the MinGW >> make and/or in its interaction with the Cygwin Bash. Note the lone >> backslash above and the leading blank in ` temacs'. >> >> I made some changes to work around this; please resync and try again. >> (Needless to say, it still bootstraps for me, at least without --jobs. >> Will try with --jobs shortly.) > > I just performed a fresh checkout and managed to complete the entire > build bootstrap process without problem (this one without specifying > --jobs=2). The only thing different from last night was that I had > upgraded TCI (an alternative to rxvt) to the latest beta but that > shouldn't make a difference I think. I did this checkout before I saw > the message I'm replying to now, don't know if I got the changes or not > but it worked now. I will do a rebuild again and time it this time and > then time another rebuild this time with --jobs=2 and see if both can > complete successfully (and their respective times will be interesting > for me to compare). Another rebuild: Without jobs, ~11 minutes With jobs: Will not complete. :-( Oh well, at least I can build myself a working emacs now. > > - Eric - Eric ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2006-12-23 11:24 ` Eric Lilja @ 2006-12-23 12:46 ` Eli Zaretskii 2007-01-03 20:56 ` Eric Lilja 0 siblings, 1 reply; 29+ messages in thread From: Eli Zaretskii @ 2006-12-23 12:46 UTC (permalink / raw) Cc: emacs-devel > From: Eric Lilja <mindcooler@gmail.com> > Date: Sat, 23 Dec 2006 12:24:21 +0100 > > Another rebuild: Without jobs, ~11 minutes Same here. I get between 11 and 13 minutes, depending on whether I use "-j 2" or not. This is a single-core system with a 3-GHz hyper-threaded CPU. > With jobs: Will not complete. :-( Please show me the error messages which prevented it from finishing. For repeatability's sake, please always start with a fresh CVS tree when you try another bootstrap, until we resolve the problems. To avoid the need to checkout the entire tree each time, what I do is immediately after a checkout copy the fresh tree to another place (you can compress it if you want to preserve space), and then, before each new build, I remove the old tree, where I built Emacs the previous time, and copy the fresh tree in its place. This makes every build go through the same steps, and makes debugging easier. Could you please adopt a similar scheme? FWIW, I have now finished testing the latest changes in the various makefile.w32-in files. I tested all the combinations I could try on this system: . with sh.exe and with CMD . with and without '-j 2 XMFLAGS="-j 2"' options . both CVS HEAD and the 22.0.92 tarball (with makefile.w32-in files and gmake.defs replaced by the ones in CVS HEAD) All combinations of these were tested, and they all succeeded. So for further testing, I must rely on your data. We use different development tools (e.g., I don't have Cygwin installed, and my gmake is compiled locally from the official sources, not downloaded from the MinGW site), so perhaps some problems only rear their ugly heads with your choice of tools. TIA ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2006-12-23 12:46 ` Eli Zaretskii @ 2007-01-03 20:56 ` Eric Lilja 2007-01-04 4:15 ` Eli Zaretskii 2007-01-04 20:17 ` Randal L. Schwartz 0 siblings, 2 replies; 29+ messages in thread From: Eric Lilja @ 2007-01-03 20:56 UTC (permalink / raw) Eli Zaretskii wrote: >> From: Eric Lilja [snip raw email address quote] >> Date: Sat, 23 Dec 2006 12:24:21 +0100 >> >> Another rebuild: Without jobs, ~11 minutes > > Same here. I get between 11 and 13 minutes, depending on whether I > use "-j 2" or not. This is a single-core system with a 3-GHz > hyper-threaded CPU. > >> With jobs: Will not complete. :-( > > Please show me the error messages which prevented it from finishing. Sorry for the delay, but I have been gone for most of the holidays. Hope you notice my update to this old thread (easy to miss if you use gmane like me). Anyway, I just performed a full checkout and was able to build emacs using the following script: #!/bin/bash export CVS_RSH="ssh" touch ~/.cvspass cvs -z3 -d:pserver:anonymous@cvs.savannah.gnu.org:/sources/emacs co emacs rm ~/.cvspass cd emacs/nt/ ./configure.bat --with-gcc --no-cygwin --no-debug time mingw32-make bootstrap --jobs=2 XMFLAGS="--jobs=2" mingw32-make install Same system as before. So with the changes you made I can now perform a full bootstrap when specifying the jobs parameter. :-) I didn't try to build without specifying jobs, but I'm sure it works if this script works (and it does). Thanks for fixing this! As you mentioned, specifying jobs provide no real benefit under windows at the moment but it still feels good that this has been fixed. - Eric ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2007-01-03 20:56 ` Eric Lilja @ 2007-01-04 4:15 ` Eli Zaretskii 2007-01-04 20:17 ` Randal L. Schwartz 1 sibling, 0 replies; 29+ messages in thread From: Eli Zaretskii @ 2007-01-04 4:15 UTC (permalink / raw) Cc: emacs-devel > From: Eric Lilja <mindcooler@gmail.com> > Date: Wed, 03 Jan 2007 21:56:00 +0100 > > Anyway, I just performed a full checkout and was able to build emacs > using the following script: > > #!/bin/bash > export CVS_RSH="ssh" > touch ~/.cvspass > cvs -z3 -d:pserver:anonymous@cvs.savannah.gnu.org:/sources/emacs co emacs > rm ~/.cvspass > cd emacs/nt/ > ./configure.bat --with-gcc --no-cygwin --no-debug > time mingw32-make bootstrap --jobs=2 XMFLAGS="--jobs=2" > mingw32-make install > > Same system as before. So with the changes you made I can now perform a > full bootstrap when specifying the jobs parameter. :-) Thanks for testing. > As you mentioned, specifying jobs provide no real benefit under > windows at the moment but it still feels good that this has been > fixed. Yes, it means that at least our Makefile's are now more correct in stating dependencies between various targets. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2007-01-03 20:56 ` Eric Lilja 2007-01-04 4:15 ` Eli Zaretskii @ 2007-01-04 20:17 ` Randal L. Schwartz 2007-01-04 21:59 ` Eli Zaretskii 1 sibling, 1 reply; 29+ messages in thread From: Randal L. Schwartz @ 2007-01-04 20:17 UTC (permalink / raw) >>>>> "Eric" == Eric Lilja <mindcooler@gmail.com> writes: Eric> Same system as before. So with the changes you made I can now perform a Eric> full bootstrap when specifying the jobs parameter. :-) I didn't try to Eric> build without specifying jobs, but I'm sure it works if this script Eric> works (and it does). Thanks for fixing this! As you mentioned, Eric> specifying jobs provide no real benefit under windows at the moment but Eric> it still feels good that this has been fixed. I just did: cvs -q up make clean # to start from a known base time make -j3 bootstrap The C compiles at the beginning are much faster, although the .el => .elc phase appears to be single threaded, but worked fine. Oddly enough, it seems to have taken twice as long! Wow. -- Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095 <merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/> Perl/Unix/security consulting, Technical writing, Comedy, etc. etc. See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training! ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2007-01-04 20:17 ` Randal L. Schwartz @ 2007-01-04 21:59 ` Eli Zaretskii 0 siblings, 0 replies; 29+ messages in thread From: Eli Zaretskii @ 2007-01-04 21:59 UTC (permalink / raw) Cc: emacs-devel > From: merlyn@stonehenge.com (Randal L. Schwartz) > Date: 04 Jan 2007 12:17:35 -0800 > > I just did: > cvs -q up > make clean # to start from a known base Is "make clean" indeed enough to remove everything generated by bootstrap? I think you need maintainer-clean. > time make -j3 bootstrap > > The C compiles at the beginning are much faster, although > the .el => .elc phase appears to be single threaded, but worked fine. The byte compilation runs in a single Emacs, but Make should run several other commands in parallel to byte compilation, such as autoloads, finder-data, and custom-deps. So at least the first part of byte compilation runs in parallel with another process > Oddly enough, it seems to have taken twice as long! Wow. What took twice as long, exactly? The byte compilation stage? ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2006-12-22 21:50 ` Eli Zaretskii 2006-12-22 23:30 ` Eric Lilja @ 2006-12-23 1:54 ` Juanma Barranquero 2006-12-23 9:53 ` Eli Zaretskii 1 sibling, 1 reply; 29+ messages in thread From: Juanma Barranquero @ 2006-12-23 1:54 UTC (permalink / raw) Cc: Eric Lilja, emacs-devel On 12/22/06, Eli Zaretskii <eliz@gnu.org> wrote: > I found and fixed several problems with Windows makefile's which > prevented parallel builds. BTW, C:\...\HEAD> cvs -q update ? lib-src/stamp_BLD ? nt/stamp_BLD ? src/stamp_BLD Should we add stamp_BLD to a few .cvsignore files? /L/e/k/t/u ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2006-12-23 1:54 ` Juanma Barranquero @ 2006-12-23 9:53 ` Eli Zaretskii 0 siblings, 0 replies; 29+ messages in thread From: Eli Zaretskii @ 2006-12-23 9:53 UTC (permalink / raw) Cc: mindcooler, emacs-devel > Date: Sat, 23 Dec 2006 02:54:57 +0100 > From: "Juanma Barranquero" <lekktu@gmail.com> > Cc: "Eric Lilja" <mindcooler@gmail.com>, emacs-devel@gnu.org > > C:\...\HEAD> cvs -q update > ? lib-src/stamp_BLD > ? nt/stamp_BLD > ? src/stamp_BLD > > Should we add stamp_BLD to a few .cvsignore files? Definitely. Fixed. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2006-12-20 11:47 Building cvs emacs on a dual-core machine Eric Lilja 2006-12-20 14:40 ` Kim F. Storm 2006-12-20 18:54 ` Eli Zaretskii @ 2006-12-20 22:19 ` Richard Stallman 2006-12-26 19:22 ` Randal L. Schwartz 2 siblings, 1 reply; 29+ messages in thread From: Richard Stallman @ 2006-12-20 22:19 UTC (permalink / raw) Cc: emacs-devel Maybe because it tries to build something that depends on something else that has not been quite finished yet because when multiple jobs are specified things doesn't happen in serial anymore? That is quite likely. We could fix this if we knew precisely what the unrecorded dependency is. But we have to depend on you to figure out what it is. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2006-12-20 22:19 ` Richard Stallman @ 2006-12-26 19:22 ` Randal L. Schwartz 2006-12-27 21:16 ` Richard Stallman 0 siblings, 1 reply; 29+ messages in thread From: Randal L. Schwartz @ 2006-12-26 19:22 UTC (permalink / raw) >>>>> "Richard" == Richard Stallman <rms@gnu.org> writes: Richard> Maybe because it tries to build something that depends on Richard> something else that has not been quite finished yet because when Richard> multiple jobs are specified things doesn't happen in serial anymore? Richard> That is quite likely. We could fix this if we knew precisely what Richard> the unrecorded dependency is. Richard> But we have to depend on you to figure out what it is. Yeah, and that's tough, because it's a timing issue, and those are tricky. For the record, *I* also tried make clean make -j3 bootstrap and it also failed, midway through the build, because some things weren't ready "in time". So, there's clearly some dependency not being marked properly in the Makefiles, but the only solution is to painstakingly go through each and every Makefile and see where -j3 would have gone parallel, and make sure there's no sequential items, or any items that use a "temp file" with a fixed name. It's hard. Unless the Makefile was designed with -j3 in mind, it generally *doesn't* work, in my experience. -- Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095 <merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/> Perl/Unix/security consulting, Technical writing, Comedy, etc. etc. See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training! ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Building cvs emacs on a dual-core machine 2006-12-26 19:22 ` Randal L. Schwartz @ 2006-12-27 21:16 ` Richard Stallman 0 siblings, 0 replies; 29+ messages in thread From: Richard Stallman @ 2006-12-27 21:16 UTC (permalink / raw) Cc: emacs-devel So, there's clearly some dependency not being marked properly in the Makefiles, but the only solution is to painstakingly go through each and every Makefile and see where -j3 would have gone parallel, and make sure there's no sequential items, or any items that use a "temp file" with a fixed name. First of all, which compilation command failed? That will give us one side of the problem. Second, which targets have not yet been made? That will narrow down the other side, down to a few dozen possibilities at most, maybe less. You could probably then figure out by hand which one it is. ^ permalink raw reply [flat|nested] 29+ messages in thread
end of thread, other threads:[~2007-01-04 21:59 UTC | newest] Thread overview: 29+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2006-12-20 11:47 Building cvs emacs on a dual-core machine Eric Lilja 2006-12-20 14:40 ` Kim F. Storm 2006-12-20 19:05 ` Eli Zaretskii 2006-12-21 11:04 ` Kim F. Storm 2006-12-22 14:39 ` Chris Moore 2006-12-22 21:37 ` Eli Zaretskii 2006-12-22 23:39 ` Chris Moore 2006-12-20 18:54 ` Eli Zaretskii 2006-12-20 19:48 ` Eric Lilja 2006-12-20 21:03 ` Eric Lilja 2006-12-20 23:27 ` Eli Zaretskii 2006-12-21 0:14 ` Eric Lilja 2006-12-22 21:50 ` Eli Zaretskii 2006-12-22 23:30 ` Eric Lilja 2006-12-23 1:50 ` Eli Zaretskii 2006-12-23 2:04 ` Eric Lilja 2006-12-23 10:42 ` Eli Zaretskii 2006-12-23 10:59 ` Eric Lilja 2006-12-23 11:24 ` Eric Lilja 2006-12-23 12:46 ` Eli Zaretskii 2007-01-03 20:56 ` Eric Lilja 2007-01-04 4:15 ` Eli Zaretskii 2007-01-04 20:17 ` Randal L. Schwartz 2007-01-04 21:59 ` Eli Zaretskii 2006-12-23 1:54 ` Juanma Barranquero 2006-12-23 9:53 ` Eli Zaretskii 2006-12-20 22:19 ` Richard Stallman 2006-12-26 19:22 ` Randal L. Schwartz 2006-12-27 21:16 ` Richard Stallman
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).