Re: [h-e-w] Current word on binaries

Re: [h-e-w] Current word on binaries

[Harald Maier]

Harald Maier <harald@maierh.de> writes:

> "David Vanderschel" <DvdS@Austin.RR.com> writes:
>
>> On February 4 <Eric.Belpaire@equant.com> wrote in message
>> news:<mailman.1868.1075904614.928.help-gnu-emacs@gnu.org>...
>> Hello,
>>
>> I've installed the binary packages (fullbin and barebin) of emacs
>> 21.3 compiled for Windows, available at
>> http://ftp.gnu.org/gnu/emacs/windows/ The binary packages don't
>> work neither on Windows 2000, nor on Windows 98.  They crash.
>>
>> Emacs remains an outstanding editor.  Thank you for it.
>> Best regards,
>
> I tried it too and I found too that something goes wrong. 'emacs -q'
> works fine, but when I tried to load my .emacs it displays the 'Emacs
> Abort Dialog' and complains that 'A fatal error has occured!'

I did some additional tests and it looks that the problem occurs only
if emacs-21.3 is compiled with a latest cygwin gcc. I did two test
with the emacs-21.3.tar.gz from March last year and the actual tarball
from ftp.gnu.org. On both sources the same happens, so it does not
look that something has changed since the compromise of the server.
Afterwards I compiled emacs with MSVC and the result is that all works
as expected. Currently I have no clue why the problem with gcc
happens.  On my system I have also the latest cvs emacs and that is
always compiled by the cygwin gcc and it works very well. Maybe other
people have an idea why the problem occurs only on the emacs-21.3
sources. I would suggest that we change the current distribution from
a mingw compilation to a msvc compilation.

Harald

Re: [h-e-w] Current word on binaries

[Jason Rumney]

Harald Maier <harald@maierh.de> writes:

> I would suggest that we change the current distribution from a mingw
> compilation to a msvc compilation.

I will do that. It may take some time though, because I think the GNU
admins will have to delete the existing files before I can upload new
ones, and they are still busy with changes to CVS after the most
recent attack on GNU servers. Until then, please use binaries from
other sources.

Re: Current word on binaries

[Eli Zaretskii]

> From: Harald Maier <harald@maierh.de>
> Date: Sat, 14 Feb 2004 10:33:50 +0100
> >
> > I tried it too and I found too that something goes wrong. 'emacs -q'
> > works fine, but when I tried to load my .emacs it displays the 'Emacs
> > Abort Dialog' and complains that 'A fatal error has occured!'
> 
> I did some additional tests and it looks that the problem occurs only
> if emacs-21.3 is compiled with a latest cygwin gcc. I did two test
> with the emacs-21.3.tar.gz from March last year and the actual tarball
> from ftp.gnu.org. On both sources the same happens, so it does not
> look that something has changed since the compromise of the server.
> Afterwards I compiled emacs with MSVC and the result is that all works
> as expected. Currently I have no clue why the problem with gcc
> happens.

What version of GCC is the one that produces a bad binary?  What does
"gcc --version" says?

Can someone run Emacs under GDB and see where exactly does it abort?

One possible idea to check is to use lower level of optimizations when
building Emacs, like -O0 or -O1.  Can someone see if that produces a
good binary?

Re: [h-e-w] Current word on binaries

[Jason Rumney]

"Eli Zaretskii" <eliz@elta.co.il> writes:

> Can someone run Emacs under GDB and see where exactly does it abort?

I have done that. It aborts on line 1739 of bytecode.c.

#ifdef BYTE_CODE_SAFE
    error ("binding stack not balanced (serious byte compiler bug)");
#else
    abort ();
#endif

Since it is deep within the byte code interpreter it is difficult to
find what happened to get get the binding stack in that state.

Re: [h-e-w] Current word on binaries

[Harald Maier]

"Eli Zaretskii" <eliz@elta.co.il> writes:

>> From: Harald Maier <harald@maierh.de>
>> Date: Sat, 14 Feb 2004 10:33:50 +0100
>> >
>> > I tried it too and I found too that something goes wrong. 'emacs -q'
>> > works fine, but when I tried to load my .emacs it displays the 'Emacs
>> > Abort Dialog' and complains that 'A fatal error has occured!'
>> 
>> I did some additional tests and it looks that the problem occurs only
>> if emacs-21.3 is compiled with a latest cygwin gcc. I did two test
>> with the emacs-21.3.tar.gz from March last year and the actual tarball
>> from ftp.gnu.org. On both sources the same happens, so it does not
>> look that something has changed since the compromise of the server.
>> Afterwards I compiled emacs with MSVC and the result is that all works
>> as expected. Currently I have no clue why the problem with gcc
>> happens.
>
> What version of GCC is the one that produces a bad binary?  What does
> "gcc --version" says?
>
> Can someone run Emacs under GDB and see where exactly does it abort?
>
> One possible idea to check is to use lower level of optimizations when
> building Emacs, like -O0 or -O1.  Can someone see if that produces a
> good binary?

It's gcc-3.3.1. I debugged the problem with insight and I got the
following backtrace in insight:

#1  0x01165be7 in w32_abort () at w32fns.c:14239
#2  0x01124db1 in Fbyte_code (bytestr=831842164, vector=1100249856, maxdepth=4) at bytecode.c:1710
#3  0x010234a7 in funcall_lambda (fun=1100290880, nargs=0, arg_vector=0x82ecf8) at eval.c:2851
#4  0x01022e7b in Ffuncall (nargs=1, args=0x82ecf4) at eval.c:2707
#5  0x01122e57 in Fbyte_code (bytestr=831842132, vector=1100249920, maxdepth=3) at bytecode.c:716
#6  0x01021cbc in Feval (form=1367136764) at eval.c:2019


The problem occured with the following command line:

$ emacs -q --no-site-file -l f:/home/maierh/local/elisp/html-helper-mode.el

,----
| ;;; html-helper-mode.el --- Major mode for composing html files.
| ;;; v 3.0.4jolly
`----

html-helper-mode is not the only example where it happens. As I
mentioned before with the current cvs emacs all works fine.

Harald

Re: [h-e-w] Current word on binaries

[Jason Rumney]

Harald Maier <harald@maierh.de> writes:

> It's gcc-3.3.1.

GCC 3.2 has the same problem. -O1 and -O0 do not help.

Re: Current word on binaries

[Harald Maier]

Jason Rumney <jasonr@gnu.org> writes:

> Harald Maier <harald@maierh.de> writes:
>
>> It's gcc-3.3.1.
>
> GCC 3.2 has the same problem. -O1 and -O0 do not help.

Ok, I remember it again. Last year there was a patch about _fmode and
O_BINARY for gcc -mno-cygwin. See herefor the the messages in
emacs-devel with:

,----
| Subject: [david.ponce@wanadoo.fr: Re: Issues with Windows gcc -mno-cygwin
|       (Mingw)]
`----

,----[ snippet from the emacs.c patch ]
| ! #if defined (MSDOS) || defined (WINDOWSNT)
|     /* We do all file input/output as binary files.  When we need to translate
|        newlines, we do that manually.  */
|     _fmode = O_BINARY;
| + #endif /* MSDOS || WINDOWSNT */
|   
| + #ifdef MSDOS
`----

I tried it and then it works fine. Is it not possible to add an file
to the /pub/gnu/emacs sources that describes the restriction or
mentinones the patch.

Harald

Re: [h-e-w] Current word on binaries

[Eli Zaretskii]

> From: Harald Maier <harald@maierh.de>
> Date: Sat, 14 Feb 2004 16:41:29 +0100
> 
> Ok, I remember it again. Last year there was a patch about _fmode and
> O_BINARY for gcc -mno-cygwin. See herefor the the messages in
> emacs-devel with:
> 
> ,----
> | Subject: [david.ponce@wanadoo.fr: Re: Issues with Windows gcc -mno-cygwin
> |       (Mingw)]
> `----

Sorry, I'm confused: what does this have to do with the problem at
hand?  Did you verify that the crash is related to this patch?

> ,----[ snippet from the emacs.c patch ]
> | ! #if defined (MSDOS) || defined (WINDOWSNT)
> |     /* We do all file input/output as binary files.  When we need to translate
> |        newlines, we do that manually.  */
> |     _fmode = O_BINARY;
> | + #endif /* MSDOS || WINDOWSNT */
> |   
> | + #ifdef MSDOS
> `----
> 
> I tried it and then it works fine. Is it not possible to add an file
> to the /pub/gnu/emacs sources that describes the restriction or
> mentinones the patch.

Why not simply patch the sources and be done with it?

Re: [h-e-w] Current word on binaries

[Harald Maier]

"Eli Zaretskii" <eliz@elta.co.il> writes:

>> From: Harald Maier <harald@maierh.de>
>> Date: Sat, 14 Feb 2004 16:41:29 +0100
>> 
>> Ok, I remember it again. Last year there was a patch about _fmode and
>> O_BINARY for gcc -mno-cygwin. See herefor the the messages in
>> emacs-devel with:
>> 
>> ,----
>> | Subject: [david.ponce@wanadoo.fr: Re: Issues with Windows gcc -mno-cygwin
>> |       (Mingw)]
>> `----
>
> Sorry, I'm confused: what does this have to do with the problem at
> hand?  Did you verify that the crash is related to this patch?

Yes, I added the patch and all worked fine. emacs-21.3.tar.gz was made
before that change of gcc. If you want to build emacs with newer gccs
you need that fix, otherwise you will get the errors as discussed.
IIRC that patch was added to the RC branch.

> Why not simply patch the sources and be done with it?

If this is possible, I don't have a problem with that. My only problem
is that the current emacs-21.3.tar.gz _sources_ in a actual cygwin
environment are useless. You will always get discussions about the
problem. Are there any plans to distribute the RC branch. That would
fix the problem too.

Harald 

Re: [h-e-w] Current word on binaries

[Eli Zaretskii]

> From: Harald Maier <harald@maierh.de>
> Date: Sat, 14 Feb 2004 18:24:08 +0100
> 
> > Why not simply patch the sources and be done with it?
> 
> If this is possible, I don't have a problem with that. My only problem
> is that the current emacs-21.3.tar.gz _sources_ in a actual cygwin
> environment are useless. You will always get discussions about the
> problem. Are there any plans to distribute the RC branch. That would
> fix the problem too.

I don't think there are plans to make another release from the branch,
but I didn't see any final word by Richard on this, so I guess it's
possible.

In any case, we could replace emacs-21.3.tar.gz with a tarball under
the same name where emacs.c is patched.  (We could call it
emacs-21.3a.tar.gz, if using the same name would confuse people.)

Re: [h-e-w] Current word on binaries

[Harald Maier]

"Eli Zaretskii" <eliz@elta.co.il> writes:

> In any case, we could replace emacs-21.3.tar.gz with a tarball under
> the same name where emacs.c is patched.  (We could call it
> emacs-21.3a.tar.gz, if using the same name would confuse people.)

That sounds really good to me.

Harald

Re: Current word on binaries

[Richard Stallman]

    Since it is deep within the byte code interpreter it is difficult to
    find what happened to get get the binding stack in that state.

You could disassemble the byte code and see if it is valid.
If it is valid, then debugging the byte code interpreter
might not be too hard.  One could determine what the stack
level should be at certain points in the function, and add
debugging code to test the actual stack level at those points.

One could also do this by adding a bytecode instruction to test
the binding stack level, and adding a flag to make the compiler
generate these instructions.  That would be a new debugging 
feature in the compiler.

Re: Current word on binaries

[Jason Rumney]

Richard Stallman <rms@gnu.org> writes:

>     Since it is deep within the byte code interpreter it is difficult to
>     find what happened to get get the binding stack in that state.
> 
> You could disassemble the byte code and see if it is valid.

I think we have pinpointed the cause. The .elc files were being
loaded as text files (on Windows this means dropping \r and
truncating on Ctrl-Z) due to a change in the defaults for GCC on
Windows sometime between 2.95 and 3.2. We already have a fix in CVS,
but it was not in 21.3.

We can either build the 21.3 release with an older version of GCC, or
patch the sources and release a 21.3a for Windows as suggested by Eli.

Re: [h-e-w] Current word on binaries

[Eli Zaretskii]

> From: Jason Rumney <jasonr@gnu.org>
> Date: 16 Feb 2004 20:38:06 +0000
> 
> I think we have pinpointed the cause. The .elc files were being
> loaded as text files (on Windows this means dropping \r and
> truncating on Ctrl-Z) due to a change in the defaults for GCC on
> Windows sometime between 2.95 and 3.2.

What change in the GCC defaults is that, and how can it affect the
text/binary issue in Emacs?

Re: [h-e-w] Current word on binaries

[Jason Rumney]

Eli Zaretskii <eliz@elta.co.il> writes:

> > From: Jason Rumney <jasonr@gnu.org>
> > Date: 16 Feb 2004 20:38:06 +0000
> > 
> > I think we have pinpointed the cause. The .elc files were being
> > loaded as text files (on Windows this means dropping \r and
> > truncating on Ctrl-Z) due to a change in the defaults for GCC on
> > Windows sometime between 2.95 and 3.2.
> 
> What change in the GCC defaults is that, and how can it affect the
> text/binary issue in Emacs?

The Emacs patch has been posted to this thread already. You commented
on it.

Re: [h-e-w] Current word on binaries

[Eli Zaretskii]

> From: Jason Rumney <jasonr@gnu.org>
> Date: 17 Feb 2004 08:56:15 +0000
> > 
> > What change in the GCC defaults is that, and how can it affect the
> > text/binary issue in Emacs?
> 
> The Emacs patch has been posted to this thread already. You commented
> on it.

I remember the patch, I just don't understand how this issue could be
uncovered by some change in the compiler.  To the best of my
understanding, the original code should have bitten us a long time
ago.

What am I missing?

Re: [h-e-w] Current word on binaries

[Jason Rumney]

"Eli Zaretskii" <eliz@elta.co.il> writes:

> I remember the patch, I just don't understand how this issue could be
> uncovered by some change in the compiler.  To the best of my
> understanding, the original code should have bitten us a long time
> ago.
> 
> What am I missing?

Linking with binmode.obj has been the way we have avoided being bitten
by MSVC in the past, but earlier versions of GCC defaulted to binary
mode and did not contain a binmode.obj (or .o) so that was made
conditional in the makefile. More recent versions of GCC now act more
like Windows compilers, defaulting to text mode and providing a
binmode.o to link with if you want binary mode as the default.

Re: [h-e-w] Current word on binaries

[Eli Zaretskii]

> From: Jason Rumney <jasonr@gnu.org>
> Date: 18 Feb 2004 00:13:32 +0000
> 
> 
> Linking with binmode.obj has been the way we have avoided being bitten
> by MSVC in the past, but earlier versions of GCC defaulted to binary
> mode and did not contain a binmode.obj (or .o) so that was made
> conditional in the makefile. More recent versions of GCC now act more
> like Windows compilers, defaulting to text mode and providing a
> binmode.o to link with if you want binary mode as the default.

So you are saying that the startup module supplied with GCC switched
from binary to text mode as the default?

Re: Current word on binaries

[Richard Stallman]

    We can either build the 21.3 release with an older version of GCC, or
    patch the sources and release a 21.3a for Windows as suggested by Eli.

We could make a 21.4 release from the release branch.
That might be a good idea anyway.

Re: [h-e-w] Current word on binaries

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Wed, 18 Feb 2004 03:38:33 -0500
> 
> We could make a 21.4 release from the release branch.
> That might be a good idea anyway.

I agree.  IIRC, 21.3 had a couple of annoying last-minute bugs.

Re: Current word on binaries

[Harald Maier]

Eli Zaretskii <eliz@elta.co.il> writes:

>> From: Richard Stallman <rms@gnu.org>
>> Date: Wed, 18 Feb 2004 03:38:33 -0500
>> 
>> We could make a 21.4 release from the release branch.
>> That might be a good idea anyway.
>
> I agree.  IIRC, 21.3 had a couple of annoying last-minute bugs.

Hello Eli,

I am not sure if I took the right version (EMACS_21_1_RC), but if this
is the current RC branch, then the patch for the discussed problem is
_not_ in the branch. Please, let me know the right RC tag if I'm wrong
and then I will double check again.

Harald

Re: [h-e-w] Current word on binaries

[Jason Rumney]

Harald Maier wrote:

>Eli Zaretskii <eliz@elta.co.il> writes:
>
>  
>
>>>From: Richard Stallman <rms@gnu.org>
>>>Date: Wed, 18 Feb 2004 03:38:33 -0500
>>>
>>>We could make a 21.4 release from the release branch.
>>>That might be a good idea anyway.
>>>      
>>>
>>I agree.  IIRC, 21.3 had a couple of annoying last-minute bugs.
>>    
>>
>
>Hello Eli,
>
>I am not sure if I took the right version (EMACS_21_1_RC), but if this
>is the current RC branch, then the patch for the discussed problem is
>_not_ in the branch. Please, let me know the right RC tag if I'm wrong
>and then I will double check again.
>  
>

The patch is not in there yet, because we had not decided to continue 
releasing from the
EMACS_21_1_RC branch until now. But that is the correct branch.

Re: Current word on binaries

[Jason Rumney]

[-- Attachment #1: Type: text/plain, Size: 679 bytes --]

Eli Zaretskii wrote:

>>From: Richard Stallman <rms@gnu.org>
>>Date: Wed, 18 Feb 2004 03:38:33 -0500
>>
>>We could make a 21.4 release from the release branch.
>>That might be a good idea anyway.
>>    
>>
>
>I agree.  IIRC, 21.3 had a couple of annoying last-minute bugs.
>  
>
Its a good idea if we can stick to a quick pretest cycle (a couple of 
weeks at the most). Otherwise the dragging out of the pretest will cause 
disillusioned users and distract the developers from getting CVS trunk 
into a releasable state. There are only a couple of minor bugfixes, so 
it should be possible to keep the pretest short if we purposely set out 
to do that and don't get distracted.


[-- Attachment #2: Type: text/html, Size: 1136 bytes --]

Re: [h-e-w] Current word on binaries

[Eli Zaretskii]

> From: Harald Maier <harald@maierh.de>
> Date: Wed, 18 Feb 2004 13:30:23 +0100
> 
> I am not sure if I took the right version (EMACS_21_1_RC), but if this
> is the current RC branch, then the patch for the discussed problem is
> _not_ in the branch. Please, let me know the right RC tag if I'm wrong
> and then I will double check again.

EMACS_21_1_RC is the right tag, as far as I remember.

Re: [h-e-w] Current word on binaries

[Eli Zaretskii]

> Date: Wed, 18 Feb 2004 13:43:24 +0000
> From: Jason Rumney <jasonr@gnu.org>
> >
> >I agree.  IIRC, 21.3 had a couple of annoying last-minute bugs.
> >
> Its a good idea if we can stick to a quick pretest cycle (a couple of
> weeks at the most).

Agreed.

I think this should be possible because, as you said, the release
branch didn't see commits for a long time.

The hardest thing would be to find a victim^H^H^H^H^H^Hvolunteer to
do package the tarballs and deltas and generally supervise the
release process.

Re: [h-e-w] Current word on binaries

[Jason Rumney]

Harald Maier <harald@maierh.de> writes:

> I am not sure if I took the right version (EMACS_21_1_RC), but if this
> is the current RC branch, then the patch for the discussed problem is
> _not_ in the branch.

The fix has been committed to EMACS_21_1_RC now, so a cvs update
should pick it up.