emacs bootstrap hangs on Windows XP / mingw

emacs bootstrap hangs on Windows XP / mingw

[Dieter Deyke]

I am trying to build the CVS version of emacs Windows XP / mingw, but
the build process stops at this point without making any further
progress:

Compiling C:/Users/deyke/emacs-build/subdir/work/lisp/image-file.el
Wrote C:/Users/deyke/emacs-build/subdir/work/lisp/image-file.elc
Compiling C:/Users/deyke/emacs-build/subdir/work/lisp/image-mode.el
Wrote C:/Users/deyke/emacs-build/subdir/work/lisp/image-mode.elc
Compiling C:/Users/deyke/emacs-build/subdir/work/lisp/image.el
Wrote C:/Users/deyke/emacs-build/subdir/work/lisp/image.elc
Compiling C:/Users/deyke/emacs-build/subdir/work/lisp/imenu.el
Wrote C:/Users/deyke/emacs-build/subdir/work/lisp/imenu.elc
Compiling C:/Users/deyke/emacs-build/subdir/work/lisp/indent.el
Wrote C:/Users/deyke/emacs-build/subdir/work/lisp/indent.elc
Compiling C:/Users/deyke/emacs-build/subdir/work/lisp/info-look.el

Looking at the process list I see that the following command is
executing:

C:\Users\deyke\emacs-build\subdir\work\bin\emacs.exe -batch
--no-init-file --no-site-file --multibyte -l loaddefs -f
batch-byte-compile-if-not-done
C:/Users/deyke/emacs-build/subdir/work/lisp/info-look.el

emacs gets no CPU time at all.

If I kill emacs the build process starts compiling the next file, and
then hangs there to.

--
Dieter Deyke

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?

Re: emacs bootstrap hangs on Windows XP / mingw

[Eli Zaretskii]

> From: Dieter Deyke <Dieter_Deyke@CoCreate.com>
> Date: Tue, 05 Feb 2008 10:01:01 -0700
> 
> Looking at the process list I see that the following command is
> executing:
> 
> C:\Users\deyke\emacs-build\subdir\work\bin\emacs.exe -batch
> --no-init-file --no-site-file --multibyte -l loaddefs -f
> batch-byte-compile-if-not-done
> C:/Users/deyke/emacs-build/subdir/work/lisp/info-look.el
> 
> emacs gets no CPU time at all.

Can you attach GDB to the running Emacs, and see where it is hung?

Re: emacs bootstrap hangs on Windows XP / mingw

[Dieter Deyke]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Dieter Deyke <Dieter_Deyke@CoCreate.com>
>> Date: Tue, 05 Feb 2008 10:01:01 -0700
>>
>> Looking at the process list I see that the following command is
>> executing:
>>
>> C:\Users\deyke\emacs-build\subdir\work\bin\emacs.exe -batch
>> --no-init-file --no-site-file --multibyte -l loaddefs -f
>> batch-byte-compile-if-not-done
>> C:/Users/deyke/emacs-build/subdir/work/lisp/info-look.el
>>
>> emacs gets no CPU time at all.
>
> Can you attach GDB to the running Emacs, and see where it is hung?

I tried, but gdb does not seem to be able to interrupt emacs. I see the
message:

Attaching to program
`/c/Users/deyke/emacs-build/subdir/work/bin/emacs.exe', process 18884

but then get no gdb command line, even after hitting C-c, or Break.

Hints?
--
Dieter Deyke

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?

Re: emacs bootstrap hangs on Windows XP / mingw

[Eli Zaretskii]

> From: Dieter Deyke <Dieter_Deyke@CoCreate.com>
> Date: Tue, 05 Feb 2008 14:21:37 -0700
> 
> > Can you attach GDB to the running Emacs, and see where it is hung?
> 
> I tried, but gdb does not seem to be able to interrupt emacs. I see the
> message:
> 
> Attaching to program
> `/c/Users/deyke/emacs-build/subdir/work/bin/emacs.exe', process 18884

This looks like you have a Cygwin GDB, while your Emacs is a native
MinGW build.  Maybe such a combination cannot work here?

Re: emacs bootstrap hangs on Windows XP / mingw

[Dieter Deyke]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Dieter Deyke <Dieter_Deyke@CoCreate.com>
>> Date: Tue, 05 Feb 2008 14:21:37 -0700
>>
>> > Can you attach GDB to the running Emacs, and see where it is hung?
>>
>> I tried, but gdb does not seem to be able to interrupt emacs. I see the
>> message:
>>
>> Attaching to program
>> `/c/Users/deyke/emacs-build/subdir/work/bin/emacs.exe', process 18884
>
> This looks like you have a Cygwin GDB, while your Emacs is a native
> MinGW build.  Maybe such a combination cannot work here?

You were right, but using the mingw gdb results in the same problem, the
debugger cannot break into emacs:

C:\Users\deyke\emacs-build\subdir\work>gdb bin\emacs.exe 22936
GNU gdb 5.1.1 (mingw experimental)
Copyright 2002 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "mingw32"...
C:\Users\deyke\emacs-build\subdir\work/22936: No such file or directory.
Attaching to program `C:\Users\deyke\emacs-build\subdir\work/bin\emacs.exe', process 22936

Then gdb hangs too, and cannot be interrupted by C-c.

For what is worth, process explorer shows the folloing threads in emacs
at that point:

!CreateThread+0x2e
!toascii+0x15d0
ntdll.dll!DbgUiRemoteBreakin
ntdll.dll!ConvertUiListtoApiList+0x276
ntdll.dll!ConvertUiListtoApiList+0x276

If I try to close the cmd window running the emacs build, I get:

"Windows cannot end thos program. It may need more time to complete an
operation."

-- 
Dieter Deyke

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?

Re: emacs bootstrap hangs on Windows XP / mingw

[Eli Zaretskii]

> From: Dieter Deyke <Dieter_Deyke@CoCreate.com>
> Date: Wed, 06 Feb 2008 09:15:43 -0700
> 
> >> Attaching to program
> >> `/c/Users/deyke/emacs-build/subdir/work/bin/emacs.exe', process 18884
> >
> > This looks like you have a Cygwin GDB, while your Emacs is a native
> > MinGW build.  Maybe such a combination cannot work here?
> 
> You were right, but using the mingw gdb results in the same problem, the
> debugger cannot break into emacs:
> 
> C:\Users\deyke\emacs-build\subdir\work>gdb bin\emacs.exe 22936
> GNU gdb 5.1.1 (mingw experimental)
> Copyright 2002 Free Software Foundation, Inc.

That's a VERY old version of GDB!  Not that I think it must be the
reason for the original problem, but you can upgrade to GDB 6.7 on the
MinGW site.

> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for details.
> This GDB was configured as "mingw32"...
> C:\Users\deyke\emacs-build\subdir\work/22936: No such file or directory.
> Attaching to program `C:\Users\deyke\emacs-build\subdir\work/bin\emacs.exe', process 22936
> 
> Then gdb hangs too, and cannot be interrupted by C-c.
> 
> For what is worth, process explorer shows the folloing threads in emacs
> at that point:
> 
> !CreateThread+0x2e
> !toascii+0x15d0
> ntdll.dll!DbgUiRemoteBreakin
> ntdll.dll!ConvertUiListtoApiList+0x276
> ntdll.dll!ConvertUiListtoApiList+0x276

Well, how about if you describe your build environment: what kind of
shell and Make programs do you use, which compiler, what versions of
ported rm and mv, etc.  Could it be that your are hitting one of the
combinations mentioned in nt/INSTALL that are known to fail?

Also, did you ever build the CVS version on this machine with these
development tools, and if so, when was the last time you succeeded?

Re: emacs bootstrap hangs on Windows XP / mingw

[Dieter Deyke]

Eli Zaretskii <eliz@gnu.org> writes:

> Well, how about if you describe your build environment: what kind of
> shell and Make programs do you use, which compiler, what versions of
> ported rm and mv, etc.  Could it be that your are hitting one of the
> combinations mentioned in nt/INSTALL that are known to fail?
>
> Also, did you ever build the CVS version on this machine with these
> development tools, and if so, when was the last time you succeeded?

I did build the CVS version of emacs every week for at least a year with
the same setup, and never had any problems before. The last good build
was 1/25/2008, the build last Friday was the first to fail. I think that
this was just after the Unicode merge.  My build script looks like this:

set PATH=C:\mingw\bin;C:\mingw\mingw\bin;C:\mingw\mingw32\bin;%PATH%

if not exist subdir mkdir subdir
cd subdir

if not exist emacs cvs -d :pserver:anonymous@cvs.sv.gnu.org:/sources/emacs checkout -P -kb emacs
cd emacs
cvs update -P -d -kb
cd ..

if not exist work mkdir work
syncdir emacs work >NUL 2>&1
syncdir emacs work

cd work\nt
call configure.bat --with-gcc --prefix C:/emacs
make bootstrap
make info
rmdir /q /s C:\emacs
make install
copy C:\mingw\bin\*.dll C:\emacs\bin
cd ..\..\..
python clean-emacs-directory.py
cd $HOME
emacs -Q -nw --eval "(display-about-screen)"

The tools I am using are

  c:/mingw/tools-gnuwin32:
  total used in directory 41388 available 94475288
  drwxrwxrwx   1 deyke    root             0 01-23 09:04 .
  drwxrwxrwx   1 deyke    root             0 01-24 15:49 ..
  -rw-rw-rw-   1 deyke    root            71 2006-03-04  GnuWin32 Packages.URL
  -rw-rw-rw-   1 deyke    root       5176996 2005-12-22  coreutils-5.3.0-bin.zip
  -rw-rw-rw-   1 deyke    root        706641 2005-12-22  coreutils-5.3.0-dep.zip
  -rw-rw-rw-   1 deyke    root       4540924 2005-12-22  coreutils-5.3.0-doc.zip
  -rw-rw-rw-   1 deyke    root       9371720 2005-12-22  coreutils-5.3.0-src.zip
  -rw-rw-rw-   1 deyke    root        509555 2005-12-22  findutils-4.2.20-2-bin.zip
  -rw-rw-rw-   1 deyke    root        706592 2005-12-22  findutils-4.2.20-2-dep.zip
  -rw-rw-rw-   1 deyke    root       1062192 2005-12-22  findutils-4.2.20-2-doc.zip
  -rw-rw-rw-   1 deyke    root       1701165 2005-12-22  findutils-4.2.20-2-src.zip
  -rw-rw-rw-   1 deyke    root        286723 2007-03-09  giflib-4.1.4-1-bin.zip
  -rw-rw-rw-   1 deyke    root         35051 2007-03-09  giflib-4.1.4-1-dep.zip
  -rw-rw-rw-   1 deyke    root        319807 2007-03-09  giflib-4.1.4-1-doc.zip
  -rw-rw-rw-   1 deyke    root         12254 2007-03-09  giflib-4.1.4-1-lib.zip
  -rw-rw-rw-   1 deyke    root       1303779 2007-03-09  giflib-4.1.4-1-src.zip
  -rw-rw-rw-   1 deyke    root        189242 2005-12-21  jpeg-6b-4-bin.zip
  -rw-rw-rw-   1 deyke    root         30149 2005-12-21  jpeg-6b-4-dep.zip
  -rw-rw-rw-   1 deyke    root        223380 2005-12-21  jpeg-6b-4-doc.zip
  -rw-rw-rw-   1 deyke    root         38186 2005-12-21  jpeg-6b-4-lib.zip
  -rw-rw-rw-   1 deyke    root        910109 2005-12-21  jpeg-6b-4-src.zip
  -rw-rw-rw-   1 deyke    root        266839 2005-12-22  libpng-1.2.8-bin.zip
  -rw-rw-rw-   1 deyke    root         39518 2005-12-22  libpng-1.2.8-dep.zip
  -rw-rw-rw-   1 deyke    root        322849 2005-12-22  libpng-1.2.8-doc.zip
  -rw-rw-rw-   1 deyke    root        347852 2005-12-22  libpng-1.2.8-lib.zip
  -rw-rw-rw-   1 deyke    root        698344 2005-12-22  libpng-1.2.8-src.zip
  -rw-rw-rw-   1 deyke    root        939884 2005-12-22  texinfo-4.8-bin.zip
  -rw-rw-rw-   1 deyke    root        706556 2005-12-22  texinfo-4.8-dep.zip
  -rw-rw-rw-   1 deyke    root       2795628 2005-12-22  texinfo-4.8-doc.zip
  -rw-rw-rw-   1 deyke    root       2879564 2005-12-22  texinfo-4.8-src.zip
  -rw-rw-rw-   1 deyke    root        687244 2006-04-05  tiff-3.8.2-1-bin.zip
  -rw-rw-rw-   1 deyke    root        307483 2006-04-05  tiff-3.8.2-1-dep.zip
  -rw-rw-rw-   1 deyke    root       1217010 2006-04-05  tiff-3.8.2-1-doc.zip
  -rw-rw-rw-   1 deyke    root         38372 2006-04-05  tiff-3.8.2-1-lib.zip
  -rw-rw-rw-   1 deyke    root       2199761 2006-04-05  tiff-3.8.2-1-src.zip
  -rw-rw-rw-   1 deyke    root         35884 2005-12-21  xpm-3.5.1-1-bin.zip
  -rw-rw-rw-   1 deyke    root        238012 2005-12-21  xpm-3.5.1-1-doc.zip
  -rw-rw-rw-   1 deyke    root         42484 2005-12-21  xpm-3.5.1-1-lib.zip
  -rw-rw-rw-   1 deyke    root        622191 2005-12-21  xpm-3.5.1-1-src.zip
  -rw-rw-rw-   1 deyke    root         99777 2005-12-21  zlib-1.2.3-bin.zip
  -rw-rw-rw-   1 deyke    root         51689 2005-12-21  zlib-1.2.3-doc.zip
  -rw-rw-rw-   1 deyke    root         71569 2005-12-21  zlib-1.2.3-lib.zip
  -rw-rw-rw-   1 deyke    root        647392 2005-12-21  zlib-1.2.3-src.zip

and

  c:/mingw/tools-mingw:
  total used in directory 15669 available 94475288
  drwxrwxrwx   1 deyke    root             0 01-23 09:04 .
  drwxrwxrwx   1 deyke    root             0 01-24 15:49 ..
  -rw-rw-rw-   1 deyke    root            61 2006-03-05  MinGW - Download.URL
  -rw-rw-rw-   1 deyke    root       7442656 11-15 17:48 binutils-2.17.50-20070129-1.tar.gz
  -rw-rw-rw-   1 deyke    root       3464344 2007-06-29  gcc-core-3.4.5-20060117-1.tar.gz
  -rw-rw-rw-   1 deyke    root       1388840 2006-03-26  gdb-5.1.1-1-mingw.tar.gz
  -rw-rw-rw-   1 deyke    root        440158 08-10 09:12 mingw-runtime-3.13.tar.gz
  -rw-rw-rw-   1 deyke    root       1585762 2006-03-26  mingw-utils-0.3.tar.gz
  -rw-rw-rw-   1 deyke    root         97305 08-10 09:13 mingw32-make-3.81-2.tar.gz
  -rw-rw-rw-   1 deyke    root       1625611 08-10 09:11 w32api-3.10.tar.gz

I also have cugwin installed, but it should be later in the PATH, and
therefore not be used for the emacs build.

BTW, I have the same setup on 2 machines, and they both fails at the
same place.

--
Dieter Deyke

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?

Re: emacs bootstrap hangs on Windows XP / mingw

[Eli Zaretskii]

> From: Dieter Deyke <Dieter_Deyke@CoCreate.com>
> Date: Wed, 06 Feb 2008 16:47:50 -0700
> 
> I did build the CVS version of emacs every week for at least a year with
> the same setup, and never had any problems before. The last good build
> was 1/25/2008, the build last Friday was the first to fail. I think that
> this was just after the Unicode merge.

I have no idea what could have causing this, or how to debug this
without being able to run the failing command under GDB.  Can you
start the failing Emacs invocation from GDB?

> I also have cugwin installed, but it should be later in the PATH, and
> therefore not be used for the emacs build.

Perhaps remove it from the PATH, and see if that changes anything.

> BTW, I have the same setup on 2 machines, and they both fails at the
> same place.

Which suggests that something in your setup triggers this, but what?

Do you have some antivirus installed?  If so, how about if you disable
it and see if that helps?

Re: emacs bootstrap hangs on Windows XP / mingw

[Dieter Deyke]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Dieter Deyke <Dieter_Deyke@CoCreate.com>
>> Date: Wed, 06 Feb 2008 16:47:50 -0700
>>
>> I did build the CVS version of emacs every week for at least a year with
>> the same setup, and never had any problems before. The last good build
>> was 1/25/2008, the build last Friday was the first to fail. I think that
>> this was just after the Unicode merge.
>
> I have no idea what could have causing this, or how to debug this
> without being able to run the failing command under GDB.  Can you
> start the failing Emacs invocation from GDB?
>
>> I also have cygwin installed, but it should be later in the PATH, and
>> therefore not be used for the emacs build.
>
> Perhaps remove it from the PATH, and see if that changes anything.
>
>> BTW, I have the same setup on 2 machines, and they both fail at the
>> same place.
>
> Which suggests that something in your setup triggers this, but what?
>
> Do you have some antivirus installed?  If so, how about if you disable
> it and see if that helps?

Quick update:

- I did update all mingw tools to the latest versions:

  binutils-2.18.50-20080109.tar.gz
  gcc-core-4.2.1-sjlj-2.tar.gz
  gdb-6.7.50.20071127-mingw.tar.bz2
  mingw-runtime-3.14.tar.gz
  mingw-utils-0.3.tar.gz
  mingw32-make-3.81-2.tar.gz
  w32api-3.11.tar.gz

- I removed all cygwin directories from PATH

  The side effect of this now is:

  Using C:\WINDOWS\system32\cmd.exe as shell.

- I did not change anything about antivirus, because I get the same
  problems on 2 systems which have different antivirus software
  installed

The result of those changes is that I now see the same problem as other
people in this group, which is

make[2]: Entering directory `C:/Users/deyke/emacs-build/subdir/work/admin/unidata'
"../../src/oo-spd/i386/emacs.exe" -Q --multibyte -batch -f batch-byte-compile unidata-gen.el
Cannot open load file: encoded-kb
make[2]: *** [unidata-gen.elc] Error -1

Instead of trying to get my setup to work, may I ask what setups other
people are using which do work?

Thanks,
-- 
Dieter Deyke

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?

Re: emacs bootstrap hangs on Windows XP / mingw

[Eli Zaretskii]

> From: Dieter Deyke <Dieter_Deyke@CoCreate.com>
> Date: Fri, 08 Feb 2008 10:52:46 -0700
> 
> The result of those changes is that I now see the same problem as other
> people in this group, which is
> 
> make[2]: Entering directory `C:/Users/deyke/emacs-build/subdir/work/admin/unidata'
> "../../src/oo-spd/i386/emacs.exe" -Q --multibyte -batch -f batch-byte-compile unidata-gen.el
> Cannot open load file: encoded-kb
> make[2]: *** [unidata-gen.elc] Error -1

Thank you.  At least we now have only one problem to solve.  I'm
working on it as we speak.

Re: emacs bootstrap hangs on Windows XP / mingw

[Eli Zaretskii]

> From: Dieter Deyke <Dieter_Deyke@CoCreate.com>
> Date: Fri, 08 Feb 2008 10:52:46 -0700
> 
> make[2]: Entering directory `C:/Users/deyke/emacs-build/subdir/work/admin/unidata'
> "../../src/oo-spd/i386/emacs.exe" -Q --multibyte -batch -f batch-byte-compile unidata-gen.el
> Cannot open load file: encoded-kb
> make[2]: *** [unidata-gen.elc] Error -1

Please try again, but with a freshly checked-out CVS tree.

Thanks.

Re: emacs bootstrap hangs on Windows XP / mingw

[Dieter Deyke]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Dieter Deyke <Dieter_Deyke@CoCreate.com>
>> Date: Fri, 08 Feb 2008 10:52:46 -0700
>>
>> make[2]: Entering directory `C:/Users/deyke/emacs-build/subdir/work/admin/unidata'
>> "../../src/oo-spd/i386/emacs.exe" -Q --multibyte -batch -f batch-byte-compile unidata-gen.el
>> Cannot open load file: encoded-kb
>> make[2]: *** [unidata-gen.elc] Error -1
>
> Please try again, but with a freshly checked-out CVS tree.
>
> Thanks.

I did, and this time the bootstrap process completed with no obvious
errors. I had some problems though to run the newly built emacs,
because it would pick up cygwin's version of libXpm.dll whenever cygwin
was in the PATH, and that made emacs hang. I could resolve this by
copying mingw's version into emacs's bin directory.

On another note, reading the bootstrap logfile would be much easier if
rm and mkdir error messages would be suppressed. On my system rm -f and
mkdir -p do not generate error messages if the file is not there, or the
directory already exists.

Also, I had to install sed for the bootstrap to work. The need for sed
is not mentioned in the readme files I found.

Thank you for your help, it is much appreciated,
-- 
Dieter Deyke
mailto:deyke@comcast.net mailto:deyke@gmx.net mailto:deyke@yahoo.com
Vs lbh pna ernq guvf, lbh unir jnl gbb zhpu gvzr.

Re: emacs bootstrap hangs on Windows XP / mingw

[Eli Zaretskii]

> From: Dieter Deyke <deyke@yahoo.com>
> Date: Sun, 10 Feb 2008 09:28:03 -0700
> Posted-To: gmane.emacs.devel
> 
> >> Cannot open load file: encoded-kb
> >> make[2]: *** [unidata-gen.elc] Error -1
> >
> > Please try again, but with a freshly checked-out CVS tree.
> >
> > Thanks.
> 
> I did, and this time the bootstrap process completed with no obvious
> errors.

Thanks for testing this.

> I had some problems though to run the newly built emacs,
> because it would pick up cygwin's version of libXpm.dll whenever cygwin
> was in the PATH, and that made emacs hang.

Yes, that's a known problem.  Mixing Cygwin and MinGW DLLs is
considered harmful.

> Also, I had to install sed for the bootstrap to work. The need for sed
> is not mentioned in the readme files I found.

We intend to remove the need for Sed in the near future, so we hope we
will never need to mention it.