Re: Issues with Windows gcc -mno-cygwin (Mingw)

Re: Issues with Windows gcc -mno-cygwin (Mingw)

[David PONCE]

> The default runtime _fmode is text with all Windows compilers. Why is
> this a problem only for mingw32's 20030208 build of GCC 3.2.2?

I am sorry, it seems my messages wasn't clear enough.  The problem is
not related to the compiler (I got the same results with gcc 3.1 and
3.2).  It is related to changes made in mingw-runtime between version
2.2 and 2.3.  With mingw-runtime 2.2, Emacs compiles and run OOTB.
With runtime 2.3 and 2.4, Emacs compiles OOTB, but fails at execution
time with random errors, because of invalid data read.

Here is an excerpt of the change log of mingw-runtime 2.3, which I
think is related to IO mode change:

2002-10-19  Danny Smith  

	* crt1.c: Define new macro __IN_MINGW_RUNTIME before including
	stdlib.h.
	Define WIN32_MEAN_AND_LEAN before including windows.h
	* include/stdlib.h (_fmode): Protect declaration as dllimported
	variable with __IN_MINGW_RUNTIME.

2002-10-19  Igor Pechtchanski  

	* crt1.c: Include stdlib.h.

2002-10-19  Danny Smith  

	* Makefile.in (CRT0S): Add txtmode.o binmode.o.
	(MINGW_OBJS): Add txtmode.o.
	(SRCDIST_FILES): Add txtmode.c binmode.c.
	crt1.c: Don't include fcntrl.h, stdlib.h.
	(_fmode): Declare, without dllimport attribute.
	(__p__fmode): Declare access function for dll's _fmode.
	(_mingw32_init_fmode): Sync dll _fmode with staticly linked
	_fmode for app.
	* txtmode.c: New file.
	* binmode.c: New file.
	* samples/fmode/test2.c: New file.
	* samples/fmode/jamfile: Add test2.exe target.

Re: Issues with Windows gcc -mno-cygwin (Mingw)

[David PONCE]

Hi All,

Would it be possible to commit this patch from Benjamin Riefenstahl:

<http://mail.gnu.org/archive/html/emacs-devel/2003-03/msg00440.html>

that fixes errors in NTEmacs when run with latest versions of mingw
runtime?

Notice that I had to apply a similar patch to 21.3 for the same
reason:

<http://mail.gnu.org/archive/html/help-emacs-windows/2003-03/msg00067.html>

Sincerely,
David

Re: Issues with Windows gcc -mno-cygwin (Mingw)

[David PONCE]

Hi Jason,

>>- Problem with _fmode (global MSC variable set to O_BINARY as the
>>  default file mode).  For a simple patch see below.  Strictly
>>  speaking I think the real problem is that this is not handled
>>  without the _fmode hack.  The code should just use its own global
>>  variable (or even more simply just add O_BINARY everywhere) instead
>>  of using this brittle compiler/runtime dependent solution.
> 
> 
> I don't understand what problem this is fixing. Can you explain some
> more?

I use latest versions of native mingw tools (not through cygwin):

- gcc 3.2.2 (mingw special 20030208-1)
- binutils 2.13.90-20030111-1
- mingw-runtime 2.4
- w32api 2.2

I also encountered that problem, with latest versions of the
mingw-runtime (2.3, 2.4).

Since these versions, the default runtime _fmode is text mode.
Because of that, I noticed that Emacs incorrectly reads some data,
and I got runtime errors, particularly in the function
"get_doc_string" in doc.c.

I followed the examples given by the mingw team, and did the
following patch, that I put in sysdep.c.  Since that, my Emacs works
like a charm!

Notice that I had to do the same thing with the latest pretest
21.2.95, to fix similar problems.

Hope it will help.
David

Index: src/sysdep.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/sysdep.c,v
retrieving revision 1.248
diff -c -r1.248 sysdep.c
*** src/sysdep.c	4 Feb 2003 14:03:13 -0000	1.248
--- src/sysdep.c	19 Mar 2003 07:29:14 -0000
***************
*** 3231,3236 ****
--- 3231,3248 ----
  #endif /* not WINDOWSNT */
  #endif /* ! HAVE_STRERROR */
  \f
+ /*
+   Since version 2.3 mingw-runtime default IO mode is textmode.
+   That causes invalid data to be read by Emacs, which in turn causes
+   various execution failures.  Changing default runtime IO mode to
+   binary, when mingw-runtime is > 2.2, avoid that.
+ */
+ #if defined(WINDOWSNT) && defined (__MINGW32__) \
+   && __MINGW32_MAJOR_VERSION >= 2 && __MINGW32_MINOR_VERSION > 2
+ #include <fcntl.h>              /* _O_BINARY */
+ int _CRT_fmode = _O_BINARY;
+ #endif
+ 
  int
  emacs_open (path, oflag, mode)
       const char *path;

Re: Issues with Windows gcc -mno-cygwin (Mingw)

[Jason Rumney]

David PONCE <david.ponce@wanadoo.fr> writes:

> Hi Jason,
> 
> I use latest versions of native mingw tools (not through cygwin):
> 
> - gcc 3.2.2 (mingw special 20030208-1)
> - binutils 2.13.90-20030111-1
> - mingw-runtime 2.4
> - w32api 2.2
> 
> I also encountered that problem, with latest versions of the
> mingw-runtime (2.3, 2.4).
> 
> Since these versions, the default runtime _fmode is text mode.
> Because of that, I noticed that Emacs incorrectly reads some data,
> and I got runtime errors, particularly in the function
> "get_doc_string" in doc.c.

The default runtime _fmode is text with all Windows compilers. Why is
this a problem only for mingw32's 20030208 build of GCC 3.2.2?

The usual way to set _fmode to binary globally is to link with
binmode.obj, but I'm not absolutely sure that we want _fmode to be
binary globally (otherwise we would have already done this for MSVC).

Re: Issues with Windows gcc -mno-cygwin (Mingw)

[Benjamin Riefenstahl]

Hi Jason,


Jason Rumney <jasonr@gnu.org> writes:
> [...] I'm not absolutely sure that we want _fmode to be binary
> globally (otherwise we would have already done this for MSVC).

If I see this right you actually did.  It's just that it doesn't work
with current versions of Mingw.  And as this is a compiler extension
of MSVC, you don't get much garantees how it works, so there is little
use in complaining about this feature.


so long, benny

Re: Issues with Windows gcc -mno-cygwin (Mingw)

[Benjamin Riefenstahl]

Hi David,


David PONCE <david.ponce@wanadoo.fr> writes:
> diff -c -r1.248 sysdep.c

I'm not sure this is the right place.  _fmode is set in emacs.c and
unexw32.c, this issue should probably not be spread out over even more
files. 

> + int _CRT_fmode = _O_BINARY;

_CRT_mode is documented to change stdin/stdout/stderr only.  While it
*does* also set _fmode with the current Mingw startup code, I'm not
sure if that is not a bug.

IMO all these problems and confusion just highlight, that these
compiler specific and badly documented options probably shouldn't be
used at all.


so long, benny

Issues with Windows gcc -mno-cygwin (Mingw)

[Benjamin Riefenstahl]

[-- Attachment #1: Type: text/plain, Size: 978 bytes --]

Hi all,


I have compiled the CVS HEAD with the Cygwin Mingw cross-environment
(gcc -mno-cygwin).  I have had two problems:

- The cygpath issue.  The Makefiles have the necessary code to use the
  Cygwin cygpath utility, but it's commented out.  I appreciate that
  the situation is not very stable in that area, but could we consider
  a conditional and a configure item for this?  We do need that code
  as long as we want Emacs to support Cygwin at compile-time, but not
  at run-time, because without that code the Makefile will provide
  Cygwin paths to ELisp.

- Problem with _fmode (global MSC variable set to O_BINARY as the
  default file mode).  For a simple patch see below.  Strictly
  speaking I think the real problem is that this is not handled
  without the _fmode hack.  The code should just use its own global
  variable (or even more simply just add O_BINARY everywhere) instead
  of using this brittle compiler/runtime dependent solution.


so long, benny


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: fmode.diff --]
[-- Type: text/x-patch, Size: 600 bytes --]

Index: unexw32.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/unexw32.c,v
retrieving revision 1.22
diff -c -r1.22 unexw32.c
*** unexw32.c	4 Feb 2003 14:03:13 -0000	1.22
--- unexw32.c	18 Mar 2003 16:19:04 -0000
***************
*** 83,88 ****
--- 83,92 ----
  
  PIMAGE_SECTION_HEADER heap_section;
  
+ /* Needed by the Mingw CRT to override _fmode, setting down in _start
+    doesn't work, it gets reset somewhere in mainCRTStartup() */
+ int _fmode = O_BINARY;
+ 
  #ifdef HAVE_NTGUI
  HINSTANCE hinst = NULL;
  HINSTANCE hprevinst = NULL;

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/emacs-devel

Re: Issues with Windows gcc -mno-cygwin (Mingw)

[Eli Zaretskii]

> From: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de>
> Date: 18 Mar 2003 18:14:09 +0100
> 
> I have compiled the CVS HEAD with the Cygwin Mingw cross-environment
> (gcc -mno-cygwin).  I have had two problems:
> 
> - The cygpath issue.  The Makefiles have the necessary code to use the
>   Cygwin cygpath utility, but it's commented out.  I appreciate that
>   the situation is not very stable in that area, but could we consider
>   a conditional and a configure item for this?  We do need that code
>   as long as we want Emacs to support Cygwin at compile-time, but not
>   at run-time, because without that code the Makefile will provide
>   Cygwin paths to ELisp.

Could you please be more specific?  Where in ELisp will Cygwin paths
be inserted?

> - Problem with _fmode (global MSC variable set to O_BINARY as the
>   default file mode).  For a simple patch see below.  Strictly
>   speaking I think the real problem is that this is not handled
>   without the _fmode hack.  The code should just use its own global
>   variable (or even more simply just add O_BINARY everywhere) instead
>   of using this brittle compiler/runtime dependent solution.

Is this change only for the unexec phase, or for the entire Emacs
operation?  If the former, you could use O_BINARY in unexw32.c and be
done with it.  If the latter (which I suspect is the case), emacs.c
already has similar code fragments for other DOS/Windows ports, so I
think the MinGW port should also use it (I actually thought it already
did, so I wonder how come this was a problem in your case).

Re: Issues with Windows gcc -mno-cygwin (Mingw)

[Benjamin Riefenstahl]

Hi Eli,


"Eli Zaretskii" <eliz@elta.co.il> writes:
> Could you please be more specific?  Where in ELisp will Cygwin paths
> be inserted?

The GNU make from Cygwin automatically predefines the CURDIR variable.
lisp/makefile uses that variable as an absolute path to the lisp code
and passes it to Emacs as a command-line parameter during compilation
of the ELisp code.  Emacs doesn't understand the syntax.

> > From: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de>
> > - Problem with _fmode

> Is this change only for the unexec phase, or for the entire Emacs
> operation?

For the entire operation.  Emacs doesn't use O_BINARY directly to load
files.  The compilation actually works more or less, there are some
crashes, but the compilation just goes ahead.  But the installed Emacs
can't read some of the compiled ELisp code files, because reading a
file in text mode stops at ^Z.

> If the latter (which I suspect is the case), emacs.c already has
> similar code fragments for other DOS/Windows ports, so I think the
> MinGW port should also use it (I actually thought it already did, so
> I wonder how come this was a problem in your case).

Emacs has an assignment to _fmode in main() under #ifdef MSDOS.  We
can add WINDOWSNT there and that would be enough for the specific
problem that I encountered.  Currently the corresponding assignment to
_fmode is in _start(), which is executed before main(), and I thought
there might be a reason why it is done there.  Only it doesn't work
currently, because the runtime code between _start() and main() resets
_fmode for some reason.  A global initialization is basically same
code that binmode.o does, and that works.


so long, benny

Re: Issues with Windows gcc -mno-cygwin (Mingw)

[Eli Zaretskii]

> From: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de>
> Date: 20 Mar 2003 00:31:01 +0100
> 
> > > From: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de>
> > > - Problem with _fmode
> 
> > Is this change only for the unexec phase, or for the entire Emacs
> > operation?
> 
> For the entire operation.  Emacs doesn't use O_BINARY directly to load
> files.

Oh yes, it does--at least in the DOS_NT versions.  Grep the C sources
for _BINARY, and you will see it yourself.

So the only issue with binary I/O I'm aware of is with standard input
and output handles.  That's why the code in emacs.c does what it does.

Note that switching stdin/stdout to binary mode might not be necessary
in the MinGW build; the DOS port does that because otherwise control
characters such as C-c and C-z are interpreted by the OS instead of
being passed directly to the application.

Re: Issues with Windows gcc -mno-cygwin (Mingw)

[Jason Rumney]

Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> writes:

> Currently the corresponding assignment to _fmode is in _start(),
> which is executed before main(), and I thought there might be a
> reason why it is done there.  Only it doesn't work currently,
> because the runtime code between _start() and main() resets _fmode
> for some reason.  A global initialization is basically same code
> that binmode.o does, and that works.

Thank you for that. Until now I've been wondering why Emacs has no
problems with MSVC, but has suddenly started having problems with
MinGW32. Is it reasonable for the runtime to change _fmode between
_start() and main()?  I think it probably is, so unless we really
need _fmode to be set between _start() and main() we should move the
assignment to main().

Re: Issues with Windows gcc -mno-cygwin (Mingw)

[Benjamin Riefenstahl]

[-- Attachment #1: Type: text/plain, Size: 1424 bytes --]

Hi Jason, Eli,


Jason Rumney <jasonr@gnu.org> writes:
> Is it reasonable for the runtime to change _fmode between _start()
> and main()?  I think it probably is,

The _start() routine only gets into it, because the Makefile specifies
an explicit entry point in the call to the linker.  So _start() is a
modification of the runtime, and Mingw can't garantee anything about
code like that.

> so unless we really need _fmode to be set between _start() and
> main() we should move the assignment to main().

Yes, that seems to work, see attached patch. 


> Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> writes:
> > Emacs doesn't use O_BINARY directly to load files.

"Eli Zaretskii" <eliz@elta.co.il> writes:
> Oh yes, it does--at least in the DOS_NT versions.  Grep the C
> sources for _BINARY, and you will see it yourself.

I have.  Emacs does use O_BINARY in some, but not in all places.
Especially Finsert_file_content() doesn't use it in calling
emacs_open() and emacs_open() doesn't add O_BINARY either when it
calls the runtime open().

> So the only issue with binary I/O I'm aware of is with standard
> input and output handles.  That's why the code in emacs.c does what
> it does.

Stdin and stdout are not an issue for NT Emacs.  The C runtime is not
used by the relevant code there, instead the OS API is used directly.
The text/binary distinction is not relevant anymore at that level.


so long, benny


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: fmode.diff --]
[-- Type: text/x-patch, Size: 2184 bytes --]

Index: emacs.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/emacs.c,v
retrieving revision 1.325
diff -c -p -r1.325 emacs.c
*** emacs.c	18 Dec 2002 06:16:28 -0000	1.325
--- emacs.c	20 Mar 2003 16:17:36 -0000
*************** Boston, MA 02111-1307, USA.  */
*** 40,45 ****
--- 40,49 ----
  #include <sys/ioctl.h>
  #endif
  
+ #ifdef WINDOWSNT
+ #include <fcntl.h>
+ #endif
+ 
  #include "lisp.h"
  #include "commands.h"
  #include "intervals.h"
*************** main (argc, argv
*** 954,964 ****
    uninterrupt_malloc ();
  #endif	/* not SYSTEM_MALLOC */
  
! #ifdef MSDOS
    /* We do all file input/output as binary files.  When we need to translate
       newlines, we do that manually.  */
    _fmode = O_BINARY;
  
  #if __DJGPP__ >= 2
    if (!isatty (fileno (stdin)))
      setmode (fileno (stdin), O_BINARY);
--- 958,970 ----
    uninterrupt_malloc ();
  #endif	/* not SYSTEM_MALLOC */
  
! #if defined (MSDOS) || defined (WINDOWSNT)
    /* We do all file input/output as binary files.  When we need to translate
       newlines, we do that manually.  */
    _fmode = O_BINARY;
+ #endif /* MSDOS || WINDOWSNT */
  
+ #ifdef MSDOS
  #if __DJGPP__ >= 2
    if (!isatty (fileno (stdin)))
      setmode (fileno (stdin), O_BINARY);
Index: unexw32.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/unexw32.c,v
retrieving revision 1.22
diff -c -p -r1.22 unexw32.c
*** unexw32.c	4 Feb 2003 14:03:13 -0000	1.22
--- unexw32.c	20 Mar 2003 16:17:50 -0000
*************** Boston, MA 02111-1307, USA.
*** 23,29 ****
  
  #include <config.h>
  
- #include <stdlib.h> 	/* _fmode */
  #include <stdio.h>
  #include <fcntl.h>
  #include <time.h>
--- 23,28 ----
*************** _start (void)
*** 111,120 ****
  
    /* Grab our malloc arena space now, before CRT starts up. */
    init_heap ();
- 
-   /* The default behavior is to treat files as binary and patch up
-      text files appropriately, in accordance with the MSDOS code.  */
-   _fmode = O_BINARY;
  
    /* This prevents ctrl-c's in shells running while we're suspended from
       having us exit.  */
--- 110,115 ----

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/emacs-devel

Re: Issues with Windows gcc -mno-cygwin (Mingw)

[Jason Rumney]

Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> writes:

> - The cygpath issue.  The Makefiles have the necessary code to use the
>   Cygwin cygpath utility, but it's commented out.  I appreciate that
>   the situation is not very stable in that area, but could we consider
>   a conditional and a configure item for this?  We do need that code
>   as long as we want Emacs to support Cygwin at compile-time, but not
>   at run-time, because without that code the Makefile will provide
>   Cygwin paths to ELisp.

Ultimately cygwin should fix their port of make to pass paths that
Windows programs expect, but if you would like to work on a
conditional that does not break the build with mingw32 versions of
make or when older versions of cygpath are installed, go ahead.

> - Problem with _fmode (global MSC variable set to O_BINARY as the
>   default file mode).  For a simple patch see below.  Strictly
>   speaking I think the real problem is that this is not handled
>   without the _fmode hack.  The code should just use its own global
>   variable (or even more simply just add O_BINARY everywhere) instead
>   of using this brittle compiler/runtime dependent solution.

I don't understand what problem this is fixing. Can you explain some
more?

Re: Issues with Windows gcc -mno-cygwin (Mingw)

[Benjamin Riefenstahl]

Hi Jason,


Jason Rumney <jasonr@gnu.org> writes:
> Ultimately cygwin should fix their port of make to pass paths that
> Windows programs expect,

I thought that the option --win32 that is supported by Cygwin helps,
but it doesn't.  In addition the Emacs Makefiles don't work in that
mode, they somehow depend on --unix (the opposite option).  If the
Emacs Makefiles would work in --win32 mode, that mode could probably
be changed to use a Windows style path for CURDIR, I would think that
using Cygwin paths in --win32 mode is a bug. 

At the moment that option seems like more work than I would want to
handle.

> but if you would like to work on a conditional that does not break
> the build with mingw32 versions of make or when older versions of
> cygpath are installed, go ahead.

I'll see what I can do. 

> I don't understand what problem this is fixing. Can you explain some
> more?

See my reply to Eli. 


so long, benny