Building Emacs from a new MinGW environment

Building Emacs from a new MinGW environment

[Dani Moncayo]

Hi,

today, I've noticed that the MinGW developers have recently released a
new graphical installation & setup tool: mingw-get-setup.exe [1].

I've tried it, and I've easily set up a brand new MinGW (+MSYS)
development environment.

Then, I've tried to build Emacs from the new environment (with the same
script I use these days).  The autogen & configure steps go well, but
"make bootstrap" fails here:

gcc  -std=gnu99 -DHAVE_CONFIG_H -I.
-I/c/MinGW/msys/1.0/home/dani/emacs/trunk/lib -I
../src -I/c/MinGW/msys/1.0/home/dani/emacs/trunk/src   -mtune=pentium4
 -DGLYPH_DEBU
G=1 -I/c/usr/include -DUSE_CRT_DLL=1 -I
/c/MinGW/msys/1.0/home/dani/emacs/trunk/nt/i
nc    -O0 -g3 -MT gettimeofday.o -MD -MP -MF .deps/gettimeofday.Tpo -c
-o gettimeofd
ay.o /c/MinGW/msys/1.0/home/dani/emacs/trunk/lib/gettimeofday.c
c:/MinGW/msys/1.0/home/dani/emacs/trunk/lib/gettimeofday.c:102:1:
error: conflicting
 types for 'gettimeofday'
In file included from
c:/MinGW/msys/1.0/home/dani/emacs/trunk/lib/gettimeofday.c:23:
0:
c:/MinGW/msys/1.0/home/dani/emacs/trunk/nt/inc/sys/time.h:45:5: note:
previous decla
ration of 'gettimeofday' was here
make[3]: *** [gettimeofday.o] Error 1
make[3]: Leaving directory `/c/MinGW/msys/1.0/home/dani/emacs/build/lib'
make[2]: *** [all] Error 2
make[2]: Leaving directory `/c/MinGW/msys/1.0/home/dani/emacs/build/lib'
make[1]: *** [lib] Error 2
make[1]: Leaving directory `/c/MinGW/msys/1.0/home/dani/emacs/build'
make: *** [bootstrap] Error 2



--- Footnotes ---

[1] Available at http://sourceforge.net/projects/mingw/files/Installer/


-- 
Dani Moncayo

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Mon, 26 Aug 2013 20:38:23 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> 
> today, I've noticed that the MinGW developers have recently released a
> new graphical installation & setup tool: mingw-get-setup.exe [1].
> 
> I've tried it, and I've easily set up a brand new MinGW (+MSYS)
> development environment.

Don't.  The "brand new MinGW" is binary incompatible with the previous
versions (read the small print in the NEWS file).  Which means that
all of the DLLs you download from 3rd-party sites, including those
ported by me, and also most of those distributed by MinGW themselves,
are suddenly prone to subtle breakage.  In fact, even without any
DLLs, the new startup code pulls in functions like readdir, which are
incompatible with what Emacs defines and uses.  Last time I tried to
link Emacs with the new MinGW runtime, wildcard expansion on the
command line stopped working.

(At the time, I tried, but failed miserably, to convince the MinGW
developers that these binary incompatibilities are a bad idea.)

So I advise you to stay with 3.20 for the time being, as I currently
see no way of reconciling these incompatibilities, and don't intend to
invest any effort in that.

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

> So I advise you to stay with 3.20 for the time being, as I currently
> see no way of reconciling these incompatibilities

Ok, I'll stick for the moment to the old environment.  Thanks for the info.

-- 
Dani Moncayo

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

On Mon, Aug 26, 2013 at 10:08 PM, Dani Moncayo <dmoncayo@gmail.com> wrote:
>> So I advise you to stay with 3.20 for the time being, as I currently
>> see no way of reconciling these incompatibilities
>
> Ok, I'll stick for the moment to the old environment.  Thanks for the info.

Hi Eli,

A couple of days ago, I screwed up my build environment by mistake,
and since currently we cannot create such environment automatically
(with the "mingw-get" package manager), because of the
incompatibilities you pointed out, I've had to re-create the MinGW
environment via the "manual" method.  And I've _almost_ succeeded  :),
but "make bootstrap" fails at some point after dumping "temacs.exe".

The last messages from "make bootstrap" are below.  I've been
searching the internet looking for one solution, but I've failed so
far.  Perhaps you could help me out.  TIA.


Dumping from temacs.tmp

          to temacs.exe

test "no" = "yes" || \
  test "X" = X ||  -r temacs.exe
cd ../lisp; make -w --jobserver-fds=3,4 - --jobserver-fds=3,4 -
--jobserver-fds=3,4 - --jobserver-fds=3,4 -j update-subdirs
make[3]: Entering directory `/home/dani/emacs/build/lisp'
cd /home/dani/emacs/emacs.git/lisp; subdirs=`find . -type d -print`;
for file in $subdirs; do  case $file in */.* | */.*/* | */=* |
*/cedet* ) ;;  *) wins="$wins${wins:+ }$file" ;;  esac;  done; \
for file in $wins; do \
   /home/dani/emacs/emacs.git/build-aux/update-subdirs $file; \
done;
make[3]: Leaving directory `/home/dani/emacs/build/lisp'
if test "no" = "yes"; then \
  rm -f bootstrap-emacs.exe; \
  ln temacs.exe bootstrap-emacs.exe; \
else \
  `/bin/pwd`/temacs --batch --load loadup bootstrap || exit 1; \
  test "X" = X ||  -zex emacs.exe; \
  mv -f emacs.exe bootstrap-emacs.exe; \
fi
Warning: arch-independent data dir
`%emacs_dir%/share/emacs/24.3.50/etc/': Permission denied

Error: charsets directory not found:

c:/msys/home/dani/emacs/build/src/%emacs_dir%/share/emacs/24.3.50/etc/charsets

Emacs will not function correctly without the character map files.

Please check your installation!

Makefile:834: recipe for target `bootstrap-emacs.exe' failed
make[2]: *** [bootstrap-emacs.exe] Error 1
make[2]: Leaving directory `/home/dani/emacs/build/src'
Makefile:381: recipe for target `src' failed
make[1]: *** [src] Error 2
make[1]: Leaving directory `/home/dani/emacs/build'
Makefile:1060: recipe for target `bootstrap' failed
make: *** [bootstrap] Error 2



-- 
Dani Moncayo

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Fri, 13 Sep 2013 16:31:41 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
> A couple of days ago, I screwed up my build environment by mistake,
> and since currently we cannot create such environment automatically
> (with the "mingw-get" package manager), because of the
> incompatibilities you pointed out, I've had to re-create the MinGW
> environment via the "manual" method.  And I've _almost_ succeeded  :),
> but "make bootstrap" fails at some point after dumping "temacs.exe".

What versions of GCC, Binutils, MinGW runtime, and w32api do you have
installed?

> Warning: arch-independent data dir
> `%emacs_dir%/share/emacs/24.3.50/etc/': Permission denied
> 
> Error: charsets directory not found:
> 
> c:/msys/home/dani/emacs/build/src/%emacs_dir%/share/emacs/24.3.50/etc/charsets
> 
> Emacs will not function correctly without the character map files.
> 
> Please check your installation!

The %emacs_dir% part was supposed to be expanded.  If nothing else
comes up from checking the installed packages, try to run the command

  temacs --batch --load loadup bootstrap

under GDB, and see why the expansion doesn't happen.

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

> What versions of GCC, Binutils, MinGW runtime, and w32api do you have
> installed?

GCC 4.7.2
Binutils 2.23.1
mingw-rt 3.20
w32api 3.17


-- 
Dani Moncayo

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Sat, 14 Sep 2013 11:41:20 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
> > What versions of GCC, Binutils, MinGW runtime, and w32api do you have
> > installed?
> 
> GCC 4.7.2
> Binutils 2.23.1
> mingw-rt 3.20
> w32api 3.17

Looks fine, but do make sure you don't have any binaries or libraries
lying around from the previous installations.

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

> The %emacs_dir% part was supposed to be expanded.  If nothing else
> comes up from checking the installed packages, try to run the command
>
>   temacs --batch --load loadup bootstrap
>
> under GDB, and see why the expansion doesn't happen.

Caveat: my knowledge of gdb and debugging emacs is almost zero :).
However, after a few hours, I think I've found out something.

I've looked for the first call to "Fexpand_file_name" which returns a
"bad" file name (bad = containing a literal "%emacs_dir%").  It is
this one:

  #0  Fexpand_file_name (name=57476769, default_directory=57476785)
      at C:/msys/home/dani/emacs/emacs.git/src/fileio.c:1437
  #1  0x01119643 in Fexpand_file_name (name=57476753,
default_directory=57422161)
      at C:/msys/home/dani/emacs/emacs.git/src/fileio.c:930
  #2  0x011bf86f in init_callproc ()
      at C:/msys/home/dani/emacs/emacs.git/src/callproc.c:1616
  #3  0x010dad7c in main (argc=5, argv=0x972af8)
      at C:/msys/home/dani/emacs/emacs.git/src/emacs.c:1325

The parameters of the call in frame #0 were:
  (gdb) p SSDATA(name)
  $1 = 0x36c4d50 <__register_frame_info+57429328>
"%emacs_dir%/share/emacs/24.3.50/etc/"
  (gdb) p SSDATA(default_directory)
  $3 = 0x36c4d7c <__register_frame_info+57429372>
"c:/msys/home/dani/emacs/build/src/"

The parameters of the call in frame #1 were:
  (gdb) p SSDATA(name)
  $4 = 0x36c4d48 <__register_frame_info+57429320> "GNU"
  (gdb) p SSDATA(default_directory)
  $5 = 0x36c4204 <__register_frame_info+57426436>
"%emacs_dir%/share/emacs/24.3.50/etc/"

IIUC, the function call at frame 0 failed to expand the default
directory passed to the function call at frame 1.

Now, It seems to me that the code responsible for the expansion of
 "%emacs_dir%" is this snippet of "Fexpand_file_name":

  /* If the file name has special constructs in it,
     call the corresponding file handler.  */
  handler = Ffind_file_name_handler (name, Qexpand_file_name);
  if (!NILP (handler))
    {
      handled_name = call3 (handler, Qexpand_file_name,
   name, default_directory);
      if (STRINGP (handled_name))
return handled_name;
      error ("Invalid handler in `file-name-handler-alist'");
    }

But I observe that the call to "Ffind_file_name_handler" returns
"Qnil", and therefore the expansion (the call to "call3") is not
carried out.

Debugging inside "Ffind_file_name_handler", I see that the "for" loop
is never entered.

Does this gives you a clue of what is failing?  If you need more info,
just tell me what commands should I run.


-- 
Dani Moncayo

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Sat, 14 Sep 2013 16:25:03 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
> I've looked for the first call to "Fexpand_file_name" which returns a
> "bad" file name (bad = containing a literal "%emacs_dir%").  It is
> this one:
> 
>   #0  Fexpand_file_name (name=57476769, default_directory=57476785)
>       at C:/msys/home/dani/emacs/emacs.git/src/fileio.c:1437
>   #1  0x01119643 in Fexpand_file_name (name=57476753,
> default_directory=57422161)
>       at C:/msys/home/dani/emacs/emacs.git/src/fileio.c:930
>   #2  0x011bf86f in init_callproc ()
>       at C:/msys/home/dani/emacs/emacs.git/src/callproc.c:1616
>   #3  0x010dad7c in main (argc=5, argv=0x972af8)
>       at C:/msys/home/dani/emacs/emacs.git/src/emacs.c:1325
> 
> The parameters of the call in frame #0 were:
>   (gdb) p SSDATA(name)
>   $1 = 0x36c4d50 <__register_frame_info+57429328>
> "%emacs_dir%/share/emacs/24.3.50/etc/"
>   (gdb) p SSDATA(default_directory)
>   $3 = 0x36c4d7c <__register_frame_info+57429372>
> "c:/msys/home/dani/emacs/build/src/"
> 
> The parameters of the call in frame #1 were:
>   (gdb) p SSDATA(name)
>   $4 = 0x36c4d48 <__register_frame_info+57429320> "GNU"
>   (gdb) p SSDATA(default_directory)
>   $5 = 0x36c4204 <__register_frame_info+57426436>
> "%emacs_dir%/share/emacs/24.3.50/etc/"
> 
> IIUC, the function call at frame 0 failed to expand the default
> directory passed to the function call at frame 1.
> 
> Now, It seems to me that the code responsible for the expansion of
>  "%emacs_dir%" is this snippet of "Fexpand_file_name":
> 
>   /* If the file name has special constructs in it,
>      call the corresponding file handler.  */
>   handler = Ffind_file_name_handler (name, Qexpand_file_name);
>   if (!NILP (handler))
>     {
>       handled_name = call3 (handler, Qexpand_file_name,
>    name, default_directory);
>       if (STRINGP (handled_name))
> return handled_name;
>       error ("Invalid handler in `file-name-handler-alist'");
>     }
> 
> But I observe that the call to "Ffind_file_name_handler" returns
> "Qnil", and therefore the expansion (the call to "call3") is not
> carried out.
> 
> Debugging inside "Ffind_file_name_handler", I see that the "for" loop
> is never entered.
> 
> Does this gives you a clue of what is failing?  If you need more info,
> just tell me what commands should I run.

You need to step further down into init_callproc.  It should correctly
initialize Vdata_directory here:

  if (data_dir == 0)
    {
      Lisp_Object tem, tem1, srcdir;

      srcdir = Fexpand_file_name (build_string ("../src/"),
				  build_string (PATH_DUMPLOADSEARCH));
      tem = Fexpand_file_name (build_string ("GNU"), Vdata_directory);
      tem1 = Ffile_exists_p (tem);
      if (!NILP (Fequal (srcdir, Vinvocation_directory)) || NILP (tem1))
	{
	  Lisp_Object newdir;
	  newdir = Fexpand_file_name (build_string ("../etc/"),
				      build_string (PATH_DUMPLOADSEARCH));
	  tem = Fexpand_file_name (build_string ("GNU"), newdir);
	  tem1 = Ffile_exists_p (tem);
	  if (!NILP (tem1))
	    Vdata_directory = newdir;  <<<<<<<<<<<<<<<<<<<<<<<<<<<<<
	}
    }

If that doesn't work, perhaps PATH_DUMPLOADSEARCH has the wrong value
(it comes from src/epaths.h).  If PATH_DUMPLOADSEARCH looks OK (it
should not have any %emacs_dir% in it, then look at the value of 'tem'
3 lines above the line I marked:

  (gdb) p tem
  (gdb) xtype
  (gdb) xstring

If the result of 'xstring' indeed shows the etc subdirectory of your
Emacs source tree, then perhaps the trouble happens inside
Ffile_exists_p: it should return non-nil in this case.  You can
display its result:

  (gdb) p tem1
  (gdb) xtype
  (gdb) xsymbol

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

> You need to step further down into init_callproc.  It should correctly
> initialize Vdata_directory here:
>
>   if (data_dir == 0)
>     {
>       Lisp_Object tem, tem1, srcdir;
>
>       srcdir = Fexpand_file_name (build_string ("../src/"),
>                                   build_string (PATH_DUMPLOADSEARCH));
>       tem = Fexpand_file_name (build_string ("GNU"), Vdata_directory);
>       tem1 = Ffile_exists_p (tem);
>       if (!NILP (Fequal (srcdir, Vinvocation_directory)) || NILP (tem1))
>         {
>           Lisp_Object newdir;
>           newdir = Fexpand_file_name (build_string ("../etc/"),
>                                       build_string (PATH_DUMPLOADSEARCH));
>           tem = Fexpand_file_name (build_string ("GNU"), newdir);
>           tem1 = Ffile_exists_p (tem);
>           if (!NILP (tem1))
>             Vdata_directory = newdir;  <<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>         }
>     }
>

The sentence you marked doesn't get executed, because "NILP(tem1)" is 1.

> If that doesn't work, perhaps PATH_DUMPLOADSEARCH has the wrong value
> (it comes from src/epaths.h).

It's defined like this:
  #define PATH_DUMPLOADSEARCH "/home/dani/emacs/emacs.git/lisp"

Although that path doesn't have a literal "%emacs_dir%", I don't know
if it is correct, because it's relative to my MSYS (which is installed
in "c:\msys").

And BTW, looking at the file, I see many strings including a literal
"%emacs_dir%", e.g.:
  #define PATH_LOADSEARCH
"%emacs_dir%/share/emacs/24.3.50/lisp;%emacs_dir%/share/emacs/24.3.50/leim"

I don't know if that is correct either.

>  If PATH_DUMPLOADSEARCH looks OK (it
> should not have any %emacs_dir% in it, then look at the value of 'tem'
> 3 lines above the line I marked:
>
>   (gdb) p tem
>   (gdb) xtype
>   (gdb) xstring

(gdb) p tem
$1 = 57503793
(gdb) xtype
Undefined command: "xtype".  Try "help".
(gdb) xstring
Undefined command: "xstring".  Try "help".

I glanced over etc/DEBUG, and saw this:

  Some GDB versions by default do not automatically load .gdbinit files
  in the directory where you invoke GDB.  With those versions of GDB,
  you will see a warning when GDB starts, like this:

    warning: File ".../src/.gdbinit" auto-loading has been declined by
your `auto-load safe-path' set to "$debugdir:$datadir/auto-load".

  There are several ways to overcome that difficulty, they are all
  described in the node "Auto-loading safe path" in the GDB user manual.

and after looking at that node in the GDB manual, I created a file
~/.gdbinit with this line:
  add-auto-load-safe-path C:\msys\home\dani

That removes the warning issued by gdb at startup, but the "xtype" and
"xstring" commands remain unknown to gdb.  What am I doing wrong?

> If the result of 'xstring' indeed shows the etc subdirectory of your
> Emacs source tree, then perhaps the trouble happens inside
> Ffile_exists_p: it should return non-nil in this case.  You can
> display its result:
>
>   (gdb) p tem1
>   (gdb) xtype
>   (gdb) xsymbol





-- 
Dani Moncayo

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Sat, 14 Sep 2013 17:42:28 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
> >           if (!NILP (tem1))
> >             Vdata_directory = newdir;  <<<<<<<<<<<<<<<<<<<<<<<<<<<<<
> >         }
> >     }
> >
> 
> The sentence you marked doesn't get executed, because "NILP(tem1)" is 1.
> 
> > If that doesn't work, perhaps PATH_DUMPLOADSEARCH has the wrong value
> > (it comes from src/epaths.h).
> 
> It's defined like this:
>   #define PATH_DUMPLOADSEARCH "/home/dani/emacs/emacs.git/lisp"

This is wrong, it should be the full Windows file name starting with a
drive letter.  Looks like the epaths-force-w32 target in the top-level
Makefile is not working correctly for some reason.  It should convert
the value of srcdir into a Windows d:/foo/bar format, and then replace
@SRC@ in nt/epaths.nt with that value.

> And BTW, looking at the file, I see many strings including a literal
> "%emacs_dir%", e.g.:
>   #define PATH_LOADSEARCH
> "%emacs_dir%/share/emacs/24.3.50/lisp;%emacs_dir%/share/emacs/24.3.50/leim"
> 
> I don't know if that is correct either.

That is correct.

> (gdb) xtype
> Undefined command: "xtype".  Try "help".
> (gdb) xstring
> Undefined command: "xstring".  Try "help".
> 
> I glanced over etc/DEBUG, and saw this:
> 
>   Some GDB versions by default do not automatically load .gdbinit files
>   in the directory where you invoke GDB.  With those versions of GDB,
>   you will see a warning when GDB starts, like this:
> 
>     warning: File ".../src/.gdbinit" auto-loading has been declined by
> your `auto-load safe-path' set to "$debugdir:$datadir/auto-load".
> 
>   There are several ways to overcome that difficulty, they are all
>   described in the node "Auto-loading safe path" in the GDB user manual.
> 
> and after looking at that node in the GDB manual, I created a file
> ~/.gdbinit with this line:
>   add-auto-load-safe-path C:\msys\home\dani
> 
> That removes the warning issued by gdb at startup, but the "xtype" and
> "xstring" commands remain unknown to gdb.  What am I doing wrong?

I'd try this instead:

  set auto-load safe-path c:/msys/home/dani

(I always disable this nuisance by using "/" as the argument.)

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

>> > If that doesn't work, perhaps PATH_DUMPLOADSEARCH has the wrong value
>> > (it comes from src/epaths.h).
>>
>> It's defined like this:
>>   #define PATH_DUMPLOADSEARCH "/home/dani/emacs/emacs.git/lisp"
>
> This is wrong, it should be the full Windows file name starting with a
> drive letter.  Looks like the epaths-force-w32 target in the top-level
> Makefile is not working correctly for some reason.  It should convert
> the value of srcdir into a Windows d:/foo/bar format, and then replace
> @SRC@ in nt/epaths.nt with that value.

I see, but alas, right now I don't know how to investigate this further.

>> and after looking at that node in the GDB manual, I created a file
>> ~/.gdbinit with this line:
>>   add-auto-load-safe-path C:\msys\home\dani
>>
>> That removes the warning issued by gdb at startup, but the "xtype" and
>> "xstring" commands remain unknown to gdb.  What am I doing wrong?
>
> I'd try this instead:
>
>   set auto-load safe-path c:/msys/home/dani
>
> (I always disable this nuisance by using "/" as the argument.)

I just tried that too.  The warning disappears, but the xtype/xstring
commands remain undefined.


It seems that today is not my day :(

-- 
Dani Moncayo

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Sat, 14 Sep 2013 18:34:45 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
> >> > If that doesn't work, perhaps PATH_DUMPLOADSEARCH has the wrong value
> >> > (it comes from src/epaths.h).
> >>
> >> It's defined like this:
> >>   #define PATH_DUMPLOADSEARCH "/home/dani/emacs/emacs.git/lisp"
> >
> > This is wrong, it should be the full Windows file name starting with a
> > drive letter.  Looks like the epaths-force-w32 target in the top-level
> > Makefile is not working correctly for some reason.  It should convert
> > the value of srcdir into a Windows d:/foo/bar format, and then replace
> > @SRC@ in nt/epaths.nt with that value.
> 
> I see, but alas, right now I don't know how to investigate this further.

Add 'echo' to display the values.

What is the value of srcdir in top-level Makefile?

> >   set auto-load safe-path c:/msys/home/dani
> >
> > (I always disable this nuisance by using "/" as the argument.)
> 
> I just tried that too.  The warning disappears, but the xtype/xstring
> commands remain undefined.

"source ./gdbinit" inside GDB should do the trick.

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

[-- Attachment #1: Type: text/plain, Size: 1985 bytes --]

> What is the value of srcdir in top-level Makefile?

it is
  /home/dani/emacs/emacs.git

and FWIW, msys_to_w32 is
  sed -e 's,\\,/,g' -e 's,^/\([A-Za-z]\)/,\1:/,'

and strangely the printed value of w32srcdir is an empty string.  It
strange because:
* The output of the command employed to produce its value (echo
"${srcdir}" | ${msys_to_w32}) is "/home/dani/emacs/emacs.git".
* The occurrences of "@SRC@" in epaths.nt are actually replaced by
that string ("/home/dani/emacs/emacs.git") when written to epaths.h.
I'm attaching the diff between these two files.

BTW, I don't understand why there are two "$" in
"/^.*#/s|@SRC@|$${w32srcdir}|g", unlike the two previous lines (which
should be similar, I think).

>> >   set auto-load safe-path c:/msys/home/dani
>> >
>> > (I always disable this nuisance by using "/" as the argument.)
>>
>> I just tried that too.  The warning disappears, but the xtype/xstring
>> commands remain undefined.
>
> "source ./gdbinit" inside GDB should do the trick.

Doing "source ./.gdbinit" didn't solve the problem, but I saw that the
contents of that file ("src/.gdbinit" from the build directory) are
this single line:
  source /home/dani/emacs/emacs.git/src/.gdbinit

so it occurred to me that perhaps giving that command to gdb could do
the trick.  And yes, it did.  So here we go:

> If that doesn't work, perhaps PATH_DUMPLOADSEARCH has the wrong value
> (it comes from src/epaths.h).  If PATH_DUMPLOADSEARCH looks OK (it
> should not have any %emacs_dir% in it, then look at the value of 'tem'
> 3 lines above the line I marked:
>
>   (gdb) p tem

$1 = 57476881

>   (gdb) xtype

Lisp_String

>   (gdb) xstring

$3 = (struct Lisp_String *) 0x36d0710 <__register_frame_info+57476880>
"c:/msys/home/dani/emacs/build/src/%emacs_dir%/share/emacs/24.3.50/etc/gnu"

> If the result of 'xstring' indeed shows the etc subdirectory of your
> Emacs source tree, then ....

No, the problem is already there: that path is wrong, obviously.

-- 
Dani Moncayo

[-- Attachment #2: epaths.diff --]
[-- Type: application/octet-stream, Size: 2472 bytes --]

--- epaths.nt	2013-09-14 20:59:50 +0200
+++ epaths.h	2013-09-14 20:59:50 +0200
@@ -38,7 +38,7 @@
    <datadir>/emacs/VERSION/lisp:<datadir>/emacs/VERSION/leim
    where datadir is eg /usr/local/share.
 */
-#define PATH_LOADSEARCH "%emacs_dir%/share/emacs/@VER@/lisp;%emacs_dir%/share/emacs/@VER@/leim"
+#define PATH_LOADSEARCH "%emacs_dir%/share/emacs/24.3.50/lisp;%emacs_dir%/share/emacs/24.3.50/leim"
 
 /* Like PATH_LOADSEARCH, but contains the non-standard pieces.
    These are the site-lisp directories.  Configure sets this to
@@ -48,25 +48,25 @@
    This is combined with PATH_LOADSEARCH to make the default load-path.
    If the --no-site-lisp option is used, this piece is excluded.
 */
-#define PATH_SITELOADSEARCH "%emacs_dir%/share/emacs/@VER@/site-lisp;%emacs_dir%/share/emacs/site-lisp"
+#define PATH_SITELOADSEARCH "%emacs_dir%/share/emacs/24.3.50/site-lisp;%emacs_dir%/share/emacs/site-lisp"
 
 /* Like PATH_LOADSEARCH, but used only during the build process
    when Emacs is dumping.  Configure (using "make epaths-force") sets
    this to $buildlisppath, which normally has the value: <srcdir>/lisp.
 */
-#define PATH_DUMPLOADSEARCH "@SRC@/lisp"
+#define PATH_DUMPLOADSEARCH "/home/dani/emacs/emacs.git/lisp"
 
 /* The extra search path for programs to invoke.  This is appended to
    whatever the PATH environment variable says to set the Lisp
    variable exec-path and the first file name in it sets the Lisp
    variable exec-directory.  exec-directory is used for finding
    executables and other architecture-dependent files.  */
-#define PATH_EXEC "%emacs_dir%/libexec/emacs/@VER@/@CFG@"
+#define PATH_EXEC "%emacs_dir%/libexec/emacs/24.3.50/i686-pc-mingw32"
 
 /* Where Emacs should look for its architecture-independent data
    files, like the NEWS file.  The lisp variable data-directory
    is set to this value.  */
-#define PATH_DATA "%emacs_dir%/share/emacs/@VER@/etc"
+#define PATH_DATA "%emacs_dir%/share/emacs/24.3.50/etc"
 
 /* Where Emacs should look for X bitmap files.
    The lisp variable x-bitmap-file-path is set based on this value.  */
@@ -74,7 +74,7 @@
 
 /* Where Emacs should look for its docstring file.  The lisp variable
    doc-directory is set to this value.  */
-#define PATH_DOC "%emacs_dir%/share/emacs/@VER@/etc"
+#define PATH_DOC "%emacs_dir%/share/emacs/24.3.50/etc"
 
 /* Where the configuration process believes the info tree lives.  The
    lisp variable configure-info-directory gets its value from this

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Sat, 14 Sep 2013 21:57:26 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
> > What is the value of srcdir in top-level Makefile?
> 
> it is
>   /home/dani/emacs/emacs.git

That's the source of the problem, it should be something like
/c/msys/home/dani/emacs/emacs.git.  So it now looks like configure
and/or config.status didn't edit Makefile.in as it should have.

How did you invoke the configure script?

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

>> > What is the value of srcdir in top-level Makefile?
>>
>> it is
>>   /home/dani/emacs/emacs.git
>
> That's the source of the problem, it should be something like
> /c/msys/home/dani/emacs/emacs.git.  So it now looks like configure
> and/or config.status didn't edit Makefile.in as it should have.
>
> How did you invoke the configure script?

Like this:
  $ cd ~/emacs/build
  $ rm -fr *
  $ CPPFLAGS='-DGLYPH_DEBUG=1' CFLAGS='-O0 -g3'
../emacs.git/nt/msysconfig.sh --enable-checking

And apparently, everything was fine:

  Configured for `i686-pc-mingw32'.

    Where should the build process find the source code?
/home/dani/emacs/emacs.git
    What compiler should emacs be built with?               gcc
-std=gnu99 -O0 -g3
    Should Emacs use the GNU version of malloc?             yes
    Should Emacs use a relocating allocator for buffers?    yes
    Should Emacs use mmap(2) for buffer allocation?         no
    What window system should Emacs use?                    w32
    What toolkit should Emacs use?                          none
    Where do we find X Windows header files?                NONE
    Where do we find X Windows libraries?                   NONE
    Does Emacs use -lXaw3d?                                 no
    Does Emacs use -lXpm?                                   yes
    Does Emacs use -ljpeg?                                  yes
    Does Emacs use -ltiff?                                  yes
    Does Emacs use a gif library?                           yes
    Does Emacs use -lpng?                                   yes
    Does Emacs use -lrsvg-2?                                no
    Does Emacs use imagemagick?                             no
    Does Emacs support sound?                               no
    Does Emacs use -lgpm?                                   no
    Does Emacs use -ldbus?                                  no
    Does Emacs use -lgconf?                                 no
    Does Emacs use GSettings?                               no
    Does Emacs use a file notification library?             yes (w32)
    Does Emacs use access control lists?                    yes
    Does Emacs use -lselinux?                               no
    Does Emacs use -lgnutls?                                yes
    Does Emacs use -lxml2?                                  yes
    Does Emacs use -lfreetype?                              no
    Does Emacs use -lm17n-flt?                              no
    Does Emacs use -lotf?                                   no
    Does Emacs use -lxft?                                   no
    Does Emacs directly use zlib?                           yes
    Does Emacs use toolkit scroll bars?                     yes

-- 
Dani Moncayo

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

>> How did you invoke the configure script?
>
> Like this:
>   $ cd ~/emacs/build
>   $ rm -fr *
>   $ CPPFLAGS='-DGLYPH_DEBUG=1' CFLAGS='-O0 -g3'
> ../emacs.git/nt/msysconfig.sh --enable-checking

I tried to configure again, but this time using a windows-absolute
path for the shell script, i.e.:
  CPPFLAGS='-DGLYPH_DEBUG=1' CFLAGS='-O0 -g3'
/c/msys/home/dani/emacs/emacs.git/nt/msysconfig.sh --enable-checking

and this time the "%emacs_dir%" problem did't appear during "make bootstrap".

So, if it is required to invoke "msysconfig.sh" with a
windows-absolute path, I think that there is something to fix in
Emacs:
* either remove that limitation (the build machinery should be smart
enough to handle this use case correctly)
* or failing that, at least detect these use-cases at configure time
and error out giving a description of the problem.

Aside from that, unfortunately the bootstrap didn't complete
successfully this time either.  I got 3 crashes, where a message box
popped out with this text:

  GNU Emacs: The extensible ... has stopped working
  A problem caused the program to ... bla bla bla.
  [Close program]

and then the bootstrap stopped.

First crash:
  Compiling /c/msys/home/dani/emacs/emacs.git/lisp/ffap.el
  Wrote c:/msys/home/dani/emacs/emacs.git/lisp/ffap.elc
  <<<<<<< HERE
  Makefile:252: recipe for target `ffap.elc' failed
  make[3]: *** [ffap.elc] Error 255

Second crash:
Compiling /c/msys/home/dani/emacs/emacs.git/lisp/gnus/deuglify.el
  Wrote c:/msys/home/dani/emacs/emacs.git/lisp/gnus/deuglify.elc
  <<<<<<< HERE
  Makefile:252: recipe for target `gnus/deuglify.elc' failed
  make[3]: *** [gnus/deuglify.elc] Error 3

Third crash and abort:
  Compiling /c/msys/home/dani/emacs/emacs.git/lisp/obsolete/xesam.el
  Wrote c:/msys/home/dani/emacs/emacs.git/lisp/obsolete/xesam.elc
  <<<<<<< HERE
  Makefile:252: recipe for target `obsolete/xesam.elc' failed
  make[3]: *** [obsolete/xesam.elc] Error 3
  make[3]: Leaving directory `/home/dani/emacs/build/lisp'
  Makefile:280: recipe for target `compile-main' failed
  make[2]: *** [compile-main] Error 2
  make[2]: Leaving directory `/home/dani/emacs/build/lisp'
  Makefile:367: recipe for target `lisp' failed
  make[1]: *** [lisp] Error 2
  make[1]: Leaving directory `/home/dani/emacs/build'
  Makefile:1060: recipe for target `bootstrap' failed
  make: *** [bootstrap] Error 2




-- 
Dani Moncayo

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Sun, 15 Sep 2013 00:30:26 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
> I tried to configure again, but this time using a windows-absolute
> path for the shell script, i.e.:
>   CPPFLAGS='-DGLYPH_DEBUG=1' CFLAGS='-O0 -g3'
> /c/msys/home/dani/emacs/emacs.git/nt/msysconfig.sh --enable-checking
> 
> and this time the "%emacs_dir%" problem did't appear during "make bootstrap".

Again, check that your dirname and sed are MSYS programs.  Maybe also
expr, and pwd -- they are all involved in the configure script
fragment that calculates the source directory.

> So, if it is required to invoke "msysconfig.sh" with a
> windows-absolute path

It is not required, I invoke the configure script using relative file
names all the time, and it works as expected.

> Aside from that, unfortunately the bootstrap didn't complete
> successfully this time either.  I got 3 crashes, where a message box
> popped out with this text:

I just bootstrapped the latest trunk, and didn't see any crashes.

>   GNU Emacs: The extensible ... has stopped working
>   A problem caused the program to ... bla bla bla.
>   [Close program]
> 
> and then the bootstrap stopped.
> 
> First crash:
>   Compiling /c/msys/home/dani/emacs/emacs.git/lisp/ffap.el
>   Wrote c:/msys/home/dani/emacs/emacs.git/lisp/ffap.elc
>   <<<<<<< HERE
>   Makefile:252: recipe for target `ffap.elc' failed
>   make[3]: *** [ffap.elc] Error 255

All I can suggest is attach GDB to the crashing program, type
"continue" at GDB prompt, then click "Debug", and see where it
crashes.

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Sat, 14 Sep 2013 23:19:06 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
> >> > What is the value of srcdir in top-level Makefile?
> >>
> >> it is
> >>   /home/dani/emacs/emacs.git
> >
> > That's the source of the problem, it should be something like
> > /c/msys/home/dani/emacs/emacs.git.  So it now looks like configure
> > and/or config.status didn't edit Makefile.in as it should have.
> >
> > How did you invoke the configure script?
> 
> Like this:
>   $ cd ~/emacs/build
>   $ rm -fr *
>   $ CPPFLAGS='-DGLYPH_DEBUG=1' CFLAGS='-O0 -g3'
> ../emacs.git/nt/msysconfig.sh --enable-checking

This is OK.

> And apparently, everything was fine:
> 
>   Configured for `i686-pc-mingw32'.
> 
>     Where should the build process find the source code?    /home/dani/emacs/emacs.git

No, this is already wrong: the source directory should be something
like /c/msys/home/dani/emacs/emacs.git, i.e. the full Windows file
name in MSYS format.  Something is still not working correctly.

Are you sure you restored the MSYS tools correctly?  E.g., do you have
dirname.exe and sed.exe in C:/MSYS/bin, and are those MSYS
executables?  Also, which version of MSYS Bash do you have?

The configure script computes the source directory in a fragment that
starts with this comment:

  # Find the source files, if location was not specified.

Perhaps add 'echo' there in strategical places to see what is not
working, and why.

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

>>   Configured for `i686-pc-mingw32'.
>>
>>     Where should the build process find the source code?    /home/dani/emacs/emacs.git
>
> No, this is already wrong: the source directory should be something
> like /c/msys/home/dani/emacs/emacs.git, i.e. the full Windows file
> name in MSYS format.  Something is still not working correctly.

Mmm, but I don't see what wrong with a path like
"/home/dani/whatever".  It is a perfectly legitimate MSYS path (like
"/c/msys/home/dani/whatever" or "c:/msys/home/dani/whatever"), isn't
it?

Why that should be the culprit of the lack of expansion of "%emacs_dir%"?

> Are you sure you restored the MSYS tools correctly?

Well, my MSYS is exactly the same I used many times for building my
Emacs binaries: the pre-packaged version from MinGW-builds released on
2013-05-15:
  http://sourceforge.net/projects/mingwbuilds/files/external-binary-packages/

IOW: the only thing that have changed wrt my previous build
environment (which compiled emacs successfully a couple of weeks ago)
is the MinGW part of the environment, which is now "handmade".

>  E.g., do you have
> dirname.exe and sed.exe in C:/MSYS/bin, and are those MSYS
> executables?

I think so:
  $ type -a dirname
  dirname is /bin/dirnam

  $ type -a sed
  sed is /bin/sed

>  Also, which version of MSYS Bash do you have?

$ bash --version
GNU bash, version 3.1.17(1)-release (i686-pc-msys)
Copyright (C) 2005 Free Software Foundation, Inc.

> The configure script computes the source directory in a fragment that
> starts with this comment:
>
>   # Find the source files, if location was not specified.
>
> Perhaps add 'echo' there in strategical places to see what is not
> working, and why.

At the end of that fragment (which begins with "# Find the source
files, if location was not specified."), srcdir holds "../emacs.git".
And looking at the configure script, I think that the fragment
responsible for making that path absolute is a bit later in the file.
It is a fairly simple and short fragment which begins with "#### Make
srcdir absolute, if it isn't already.".  I've seen that, after that
second fragment, "srcdir" holds its definitive value:
"/home/dani/emacs/emacs.git".

If you look at that code, you'll see that the "srcdir" variable can be
updated with either (a) the output of the "pwd" or (b) the contents of
the PWD variable.  But if I try those options in my MSYS bash, both
give me the same MSYS path "/home/dani/emacs/emacs.git".

Perhaps the key factor here is the fact that, in my case, the
directory holding the source code is inside the MSYS tree (under my
MSYS "home" directory).  I guess that in your case that directory is
outside the MSYS tree, so that its absolute path in MSYS has
necessarily the form "/X/some/dir" (where X is the letter of some
windows drive).

-- 
Dani Moncayo

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Mon, 16 Sep 2013 18:48:24 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
> >>   Configured for `i686-pc-mingw32'.
> >>
> >>     Where should the build process find the source code?    /home/dani/emacs/emacs.git
> >
> > No, this is already wrong: the source directory should be something
> > like /c/msys/home/dani/emacs/emacs.git, i.e. the full Windows file
> > name in MSYS format.  Something is still not working correctly.
> 
> Mmm, but I don't see what wrong with a path like
> "/home/dani/whatever".  It is a perfectly legitimate MSYS path (like
> "/c/msys/home/dani/whatever" or "c:/msys/home/dani/whatever"), isn't
> it?

It is, as long as it is used by MSYS programs.  But it is not
legitimate when it is passed to MinGW Emacs, because Emacs will not
find this directory.  Which is exactly what happens in your case.

> Why that should be the culprit of the lack of expansion of "%emacs_dir%"?

Expansion of %emacs_dir% is not the issue here, contrary to what I
originally thought and wrote.  The issue is that temacs does not find
the etc directory where it should, because temacs is a MinGW program,
and doesn't know about the MSYS '/' magic.  That is why MSYS file
names such as /home/dani/whatever should never end up in src/epaths.h.

> > The configure script computes the source directory in a fragment that
> > starts with this comment:
> >
> >   # Find the source files, if location was not specified.
> >
> > Perhaps add 'echo' there in strategical places to see what is not
> > working, and why.
> 
> At the end of that fragment (which begins with "# Find the source
> files, if location was not specified."), srcdir holds "../emacs.git".
> And looking at the configure script, I think that the fragment
> responsible for making that path absolute is a bit later in the file.
> It is a fairly simple and short fragment which begins with "#### Make
> srcdir absolute, if it isn't already.".  I've seen that, after that
> second fragment, "srcdir" holds its definitive value:
> "/home/dani/emacs/emacs.git".
> 
> If you look at that code, you'll see that the "srcdir" variable can be
> updated with either (a) the output of the "pwd" or (b) the contents of
> the PWD variable.  But if I try those options in my MSYS bash, both
> give me the same MSYS path "/home/dani/emacs/emacs.git".

In my case, pwd gives an absolute /d/foo/bar file name.  It does that
even if I chdir to a directory below the MSYS root, _provided_ that I
use an absolute file name when I chdir (/usr is mounted on D:/usr/MSYS
in my case):

 $ cd /d/usr/MSYS && pwd
 /d/usr/MSYS
 $ cd /usr && pwd
 /usr

> Perhaps the key factor here is the fact that, in my case, the
> directory holding the source code is inside the MSYS tree (under my
> MSYS "home" directory).

Are you saying that the previous builds, which worked, were not under
the MSYS home directory?  If they were, then why the builds started
failing?

> I guess that in your case that directory is outside the MSYS tree,
> so that its absolute path in MSYS has necessarily the form
> "/X/some/dir" (where X is the letter of some windows drive).

Yes, I build outside of the MSYS hierarchy.  But does this explain why
your builds stopped working after whatever happened to your MinGW
installation?

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

>> Mmm, but I don't see what wrong with a path like
>> "/home/dani/whatever".  It is a perfectly legitimate MSYS path (like
>> "/c/msys/home/dani/whatever" or "c:/msys/home/dani/whatever"), isn't
>> it?
>
> It is, as long as it is used by MSYS programs.  But it is not
> legitimate when it is passed to MinGW Emacs, because Emacs will not
> find this directory.  Which is exactly what happens in your case.

I see.  But then, why doesn't the configure/build process make the
translation when needed?  That would remove this limitation.

I think I'm not doing anything extraordinary: just move to the build
directory with "cd emacs/build" (which is the easier way to specify
that path) and then invoking to the MSYS configure script with a
relative path (which again is the easier way of doing that).

If that "automatic translation" is difficult to implement, then I
think that Emacs at least should document the limitation (in
"nt/INSTALL") and also add some check to the configure script in order
to catch this use-case and stop the process with a helpful message.

>> Why that should be the culprit of the lack of expansion of "%emacs_dir%"?
>
> Expansion of %emacs_dir% is not the issue here, contrary to what I
> originally thought and wrote.  The issue is that temacs does not find
> the etc directory where it should, because temacs is a MinGW program,
> and doesn't know about the MSYS '/' magic.  That is why MSYS file
> names such as /home/dani/whatever should never end up in src/epaths.h.

Ok, but see below: the nicer solution would be IMO to automate this
translation of paths when necessary.

>> If you look at that code, you'll see that the "srcdir" variable can be
>> updated with either (a) the output of the "pwd" or (b) the contents of
>> the PWD variable.  But if I try those options in my MSYS bash, both
>> give me the same MSYS path "/home/dani/emacs/emacs.git".
>
> In my case, pwd gives an absolute /d/foo/bar file name.  It does that
> even if I chdir to a directory below the MSYS root, _provided_ that I
> use an absolute file name when I chdir (/usr is mounted on D:/usr/MSYS
> in my case):
>
>  $ cd /d/usr/MSYS && pwd
>  /d/usr/MSYS
>  $ cd /usr && pwd
>  /usr

Of course.  These are two _different_ paths which happen to refer to
the same directory.

>> Perhaps the key factor here is the fact that, in my case, the
>> directory holding the source code is inside the MSYS tree (under my
>> MSYS "home" directory).
>
> Are you saying that the previous builds, which worked, were not under
> the MSYS home directory?  If they were, then why the builds started
> failing?

No, in the previous build, both the source and build directories were
outside from the MSYS tree (under "c:\emacs").  That's why I didn't
experience this problem then.

>> I guess that in your case that directory is outside the MSYS tree,
>> so that its absolute path in MSYS has necessarily the form
>> "/X/some/dir" (where X is the letter of some windows drive).
>
> Yes, I build outside of the MSYS hierarchy.  But does this explain why
> your builds stopped working after whatever happened to your MinGW
> installation?

Yes.  As we have seen, the "%emacs_dir%" problem is related to the way
you reference the MSYS configure script: If you use a "long" path
("/c/msys/home/...) then the problem doesn't crop up, but if you
either (a) use a "short" path ("/home/...") or (b) use a relative path
_and_ the current directory is "short", then the problem does arise.

-- 
Dani Moncayo

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Mon, 16 Sep 2013 21:25:29 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
> >> Mmm, but I don't see what wrong with a path like
> >> "/home/dani/whatever".  It is a perfectly legitimate MSYS path (like
> >> "/c/msys/home/dani/whatever" or "c:/msys/home/dani/whatever"), isn't
> >> it?
> >
> > It is, as long as it is used by MSYS programs.  But it is not
> > legitimate when it is passed to MinGW Emacs, because Emacs will not
> > find this directory.  Which is exactly what happens in your case.
> 
> I see.  But then, why doesn't the configure/build process make the
> translation when needed?

Because I don't know how to do that.  Is there any MSYS command that
would add the missing leading directories?  Patches welcome.

> If that "automatic translation" is difficult to implement, then I
> think that Emacs at least should document the limitation

Will do, if no other solution presents itself.

> and also add some check to the configure script in order
> to catch this use-case and stop the process with a helpful message.

If we can identify this use case, we are more than half-way towards
solving it.  IOW, I don't know how to identify it.

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

>> I see.  But then, why doesn't the configure/build process make the
>> translation when needed?
>
> Because I don't know how to do that.  Is there any MSYS command that
> would add the missing leading directories?  Patches welcome.

In an MSYS bash, the "pwd" shell builtin has a "-W" option, which
produces a Windows-native path of the current directory.  That should
allow to translate any MSYS path.

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Mon, 16 Sep 2013 21:44:42 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
> In an MSYS bash, the "pwd" shell builtin has a "-W" option, which
> produces a Windows-native path of the current directory.  That should
> allow to translate any MSYS path.

The question is how to use this feature in the context of this
problem.  We cannot replace all invocations of 'pwd' in configure with
"pwd -W", obviously.

Does it work to replace this line in the top-level Makefile.in

        @(w32srcdir=`echo "${srcdir}" | ${msys_to_w32}` ;       \

with this:

        @(w32srcdir=`(cd "${srcdir}" && pwd -W) | ${msys_to_w32}` ;       \

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Mon, 16 Sep 2013 23:19:41 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> Does it work to replace this line in the top-level Makefile.in
> 
>         @(w32srcdir=`echo "${srcdir}" | ${msys_to_w32}` ;       \
> 
> with this:
> 
>         @(w32srcdir=`(cd "${srcdir}" && pwd -W) | ${msys_to_w32}` ;       \

Actually, a better fix might be in configure.ac, here:


  #### Make srcdir absolute, if it isn't already.  It's important to
  #### avoid running the file name through pwd unnecessarily, since pwd can
  #### give you automounter prefixes, which can go away.  We do all this
  #### so Emacs can find its files when run uninstalled.
  ## Make sure CDPATH doesn't affect cd (in case PWD is relative).
  unset CDPATH
  case "${srcdir}" in
    [[\\/]]* | ?:[[\\/]]*) ;;
    . )
      ## We may be able to use the $PWD environment variable to make this
      ## absolute.  But sometimes PWD is inaccurate.
      ## Note: we used to use $PWD at the end instead of `pwd`,
      ## but that tested only for a well-formed and valid PWD,
      ## it did not object when PWD was well-formed and valid but just wrong.
      if test ".$PWD" != "." && test ".`(cd "$PWD" ; sh -c pwd)`" = ".`pwd`"  ;
      then
	srcdir="$PWD"
      else
	srcdir=`(cd "$srcdir"; pwd)`
      fi
    ;;
    *  ) srcdir=`(cd "$srcdir"; pwd)` ;;
  esac

Change this:

  case "${srcdir}" in
    [[\\/]]* | ?:[[\\/]]*) ;;

into this:

  case "${srcdir}" in
    [[\\/]]* | ?:[[\\/]]*) 
    test "$MSYSTEM" = "MINGW32" && srcdir=`(cd "$srcdir"; pwd -W)`
    ;;

This fix, if it works, is better, since it will fix also the stuff
that gets written into .gdbinit file in the build directory (it didn't
work for you because an MSYS file name was written there, and the
MinGW build of GDB doesn't grok that).

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

> Change this:
>
>   case "${srcdir}" in
>     [[\\/]]* | ?:[[\\/]]*) ;;
>
> into this:
>
>   case "${srcdir}" in
>     [[\\/]]* | ?:[[\\/]]*)
>     test "$MSYSTEM" = "MINGW32" && srcdir=`(cd "$srcdir"; pwd -W)`
>     ;;
>
> This fix, if it works, is better, since it will fix also the stuff
> that gets written into .gdbinit file in the build directory (it didn't
> work for you because an MSYS file name was written there, and the
> MinGW build of GDB doesn't grok that).

This does't work in my case, because at that point, "${srcdir}"
contains the relative path "../emacs.git", not something like
"/c/whatever" or "c:/whatever".

I think that the right fix is to replace *all* the treatment of
patterns like the above ("/c/whatever" or "c:/whatever") with logic
based on the "pwd -W" feature of MSYS.

IOW, an MSYS path doesn't have to match none of the above patterns, so
the only reliable way of getting a Windows-native path is with "pwd
-W".

Therefore, things line "msys_to_w32", "msys_lisppath_to_w32" should be
replaced with the above criteria.

-- 
Dani Moncayo

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Tue, 17 Sep 2013 10:17:38 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
> > Change this:
> >
> >   case "${srcdir}" in
> >     [[\\/]]* | ?:[[\\/]]*) ;;
> >
> > into this:
> >
> >   case "${srcdir}" in
> >     [[\\/]]* | ?:[[\\/]]*)
> >     test "$MSYSTEM" = "MINGW32" && srcdir=`(cd "$srcdir"; pwd -W)`
> >     ;;
> >
> > This fix, if it works, is better, since it will fix also the stuff
> > that gets written into .gdbinit file in the build directory (it didn't
> > work for you because an MSYS file name was written there, and the
> > MinGW build of GDB doesn't grok that).
> 
> This does't work in my case, because at that point, "${srcdir}"
> contains the relative path "../emacs.git", not something like
> "/c/whatever" or "c:/whatever".

OK, then we could put this line

  test "$MSYSTEM" = "MINGW32" && srcdir=`(cd "$srcdir"; pwd -W)`

immediately below the 'esac' in the fragment I've shown.  Does that do
the job for you?

> I think that the right fix is to replace *all* the treatment of
> patterns like the above ("/c/whatever" or "c:/whatever") with logic
> based on the "pwd -W" feature of MSYS.
> 
> IOW, an MSYS path doesn't have to match none of the above patterns, so
> the only reliable way of getting a Windows-native path is with "pwd
> -W".
> 
> Therefore, things line "msys_to_w32", "msys_lisppath_to_w32" should be
> replaced with the above criteria.

I'm not sure I understand what you are suggesting, specifically.  Can
you show a patch that works for you?

In any case, the problem with src/.gdbinit in the build tree still
needs to be solved; no amount of changes in the Makefile's can do
that, because that file is created by config.status.  So we still need
something in configure.ac as well.

The advantage of my suggestion is that it solves this problem in a
single place, once and for all, while what you seem to suggest would
need multiple changes in many places.  Why is that better?

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

[-- Attachment #1: Type: text/plain, Size: 2269 bytes --]

> OK, then we could put this line
>
>   test "$MSYSTEM" = "MINGW32" && srcdir=`(cd "$srcdir"; pwd -W)`
>
> immediately below the 'esac' in the fragment I've shown.  Does that do
> the job for you?

I've tried it but the bootstrap fails.  I'm attaching the output of
the corresponding "make bootstrap" .

>> I think that the right fix is to replace *all* the treatment of
>> patterns like the above ("/c/whatever" or "c:/whatever") with logic
>> based on the "pwd -W" feature of MSYS.
>>
>> IOW, an MSYS path doesn't have to match none of the above patterns, so
>> the only reliable way of getting a Windows-native path is with "pwd
>> -W".
>>
>> Therefore, things line "msys_to_w32", "msys_lisppath_to_w32" should be
>> replaced with the above criteria.
>
> I'm not sure I understand what you are suggesting, specifically.  Can
> you show a patch that works for you?

I'm not yet familiar enough with the logic involved here.  I've tried
something but it didn't work either.

But the general principle is that, it is conceptually wrong to do
conversions of pathnames from MSYS format to native windows format
based on pattern substitution, assuming that the MSYS paths will
always be either in "/X/whatever" format or in "X:/whatever" format.

Therefore, whenever we need to convert pathnames from any
MSYS-compliant format to the windows-native counterpart, the only
reliable way is using the "pwd -W" feature.

> In any case, the problem with src/.gdbinit in the build tree still
> needs to be solved; no amount of changes in the Makefile's can do
> that, because that file is created by config.status.  So we still need
> something in configure.ac as well.

FWIW, with the last change you suggested, the file "src/.gdbinit" in
the build tree now contains this:
   source C:/msys/home/d.moncayo.melgar/emacs/emacs.git/src/.gdbinit

> The advantage of my suggestion is that it solves this problem in a
> single place, once and for all, while what you seem to suggest would
> need multiple changes in many places.  Why is that better?

See above.  Again: doing conversions to windows-native format like
e.g. piping through "msys_to_w32" is conceptually wrong, I think,
because not all MSYS-compliant paths will match the "/X/.." or
"X:/..." patterns.

-- 
Dani Moncayo

[-- Attachment #2: make-bootstrap.zip --]
[-- Type: application/zip, Size: 4962 bytes --]

Re: Building Emacs from a new MinGW environment

[Glenn Morris]

Dani Moncayo wrote:

> Therefore, whenever we need to convert pathnames from any
> MSYS-compliant format to the windows-native counterpart, the only
> reliable way is using the "pwd -W" feature.

You could replace every instance of `pwd` with `pwd ${PWD_OPTS}`, where
configure defines PWD_OPTS=-W on MS, empty elsewhere.

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Tue, 17 Sep 2013 18:09:37 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
> > OK, then we could put this line
> >
> >   test "$MSYSTEM" = "MINGW32" && srcdir=`(cd "$srcdir"; pwd -W)`
> >
> > immediately below the 'esac' in the fragment I've shown.  Does that do
> > the job for you?
> 
> I've tried it but the bootstrap fails.  I'm attaching the output of
> the corresponding "make bootstrap" .

I don't understand where did the
C:/msys/home/d.moncayo.melgar/emacs/emacs.git/ directory come from?
Does directory by such name indeed exist on your disk, and Windows
(not MSYS) can find it?  Did you try this from a directory different
from where you made your previous trials?

If the directory exists, I don't understand the reason for these
failures:

  /bin/makeinfo --force --enable-encoding -I C:/msys/home/d.moncayo.melgar/emacs/emacs.git/doc/lispref/../emacs -I C:/msys/home/d.moncayo.melgar/emacs/emacs.git/doc/lispref --no-split -o C:/msys/home/d.moncayo.melgar/emacs/emacs.git/doc/lispref/../../info/elisp.info C:/msys/home/d.moncayo.melgar/emacs/emacs.git/doc/lispref/elisp.texi
  C:/msys/home/d.moncayo.melgar/emacs/emacs.git/doc/lispref/elisp.texi:58: @include `emacsver.texi': No such file or directory.

Is this some problem with the MSYS port of makeinfo, perhaps?

> But the general principle is that, it is conceptually wrong to do
> conversions of pathnames from MSYS format to native windows format
> based on pattern substitution, assuming that the MSYS paths will
> always be either in "/X/whatever" format or in "X:/whatever" format.

If the change in configure.ac works, we will be able to remove almost
all of that stuff in top-level Makefile.

> Therefore, whenever we need to convert pathnames from any
> MSYS-compliant format to the windows-native counterpart, the only
> reliable way is using the "pwd -W" feature.

That's what my suggestion tries to do, but in a single place.

> > In any case, the problem with src/.gdbinit in the build tree still
> > needs to be solved; no amount of changes in the Makefile's can do
> > that, because that file is created by config.status.  So we still need
> > something in configure.ac as well.
> 
> FWIW, with the last change you suggested, the file "src/.gdbinit" in
> the build tree now contains this:
>    source C:/msys/home/d.moncayo.melgar/emacs/emacs.git/src/.gdbinit

Is this the correct file name, or isn't it?

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

[-- Attachment #1: Type: text/plain, Size: 3949 bytes --]

>> > OK, then we could put this line
>> >
>> >   test "$MSYSTEM" = "MINGW32" && srcdir=`(cd "$srcdir"; pwd -W)`
>> >
>> > immediately below the 'esac' in the fragment I've shown.  Does that do
>> > the job for you?
>>
>> I've tried it but the bootstrap fails.  I'm attaching the output of
>> the corresponding "make bootstrap" .
>
> I don't understand where did the
> C:/msys/home/d.moncayo.melgar/emacs/emacs.git/ directory come from?
> Does directory by such name indeed exist on your disk, and Windows
> (not MSYS) can find it?  Did you try this from a directory different
> from where you made your previous trials?

That directory is simply the root of my local Emacs source code tree.
I guess that what confused you is the "d.moncayo.melgar" part, which
was "dani" in previous messages.  Well, the difference is due that one
is the desktop I use at work, and the other is my personal laptop.
But the build environment is equal in both machines (modulo the user
name).

> If the directory exists, I don't understand the reason for these
> failures:
>
>   /bin/makeinfo --force --enable-encoding -I C:/msys/home/d.moncayo.melgar/emacs/emacs.git/doc/lispref/../emacs -I C:/msys/home/d.moncayo.melgar/emacs/emacs.git/doc/lispref --no-split -o C:/msys/home/d.moncayo.melgar/emacs/emacs.git/doc/lispref/../../info/elisp.info C:/msys/home/d.moncayo.melgar/emacs/emacs.git/doc/lispref/elisp.texi
>   C:/msys/home/d.moncayo.melgar/emacs/emacs.git/doc/lispref/elisp.texi:58: @include `emacsver.texi': No such file or directory.
>
> Is this some problem with the MSYS port of makeinfo, perhaps?

I've always used the makeinfo that comes with MSYS, yes.  But that
version have always worked fine for me.  This problem appears just
when I try your proposed change in "configure.ac".

>> But the general principle is that, it is conceptually wrong to do
>> conversions of pathnames from MSYS format to native windows format
>> based on pattern substitution, assuming that the MSYS paths will
>> always be either in "/X/whatever" format or in "X:/whatever" format.
>
> If the change in configure.ac works, we will be able to remove almost
> all of that stuff in top-level Makefile.

Great.  I hope we will get this to work  ;).

>> Therefore, whenever we need to convert pathnames from any
>> MSYS-compliant format to the windows-native counterpart, the only
>> reliable way is using the "pwd -W" feature.
>
> That's what my suggestion tries to do, but in a single place.

Mmmm but your suggested change doesn't to work for me.

I've tried it again:  autogen.sh + msysconfig.sh + "make bootstrap",
with this single change in my Emacs tree:

  diff --git a/configure.ac b/configure.ac
  index 86a5f30..cb8f6d6 100644
  --- a/configure.ac
  +++ b/configure.ac
  @@ -443,6 +443,8 @@ case "${srcdir}" in
     *  ) srcdir=`(cd "$srcdir"; pwd)` ;;
   esac

  +test "$MSYSTEM" = "MINGW32" && srcdir=`(cd "$srcdir"; pwd -W)`
  +
   ### Canonicalize the configuration name.

   AC_CANONICAL_HOST

The result in my laptop seems to be the same than in my office
desktop: autogen ok, configure ok, but bootstrap failure  (new logfile
attached).

Perhaps you could try this yourself.  It should be pretty easy.  Just
move or copy the emacs source code under your MSYS' home directory,
and try to configure & bootstrap avoiding "long" pathnames.

>> > In any case, the problem with src/.gdbinit in the build tree still
>> > needs to be solved; no amount of changes in the Makefile's can do
>> > that, because that file is created by config.status.  So we still need
>> > something in configure.ac as well.
>>
>> FWIW, with the last change you suggested, the file "src/.gdbinit" in
>> the build tree now contains this:
>>    source C:/msys/home/d.moncayo.melgar/emacs/emacs.git/src/.gdbinit
>
> Is this the correct file name, or isn't it?

Yes, that is correct.

-- 
Dani Moncayo

[-- Attachment #2: make-bootstrap.zip --]
[-- Type: application/zip, Size: 4933 bytes --]

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Tue, 17 Sep 2013 22:29:43 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
> > I don't understand where did the
> > C:/msys/home/d.moncayo.melgar/emacs/emacs.git/ directory come from?
> > Does directory by such name indeed exist on your disk, and Windows
> > (not MSYS) can find it?  Did you try this from a directory different
> > from where you made your previous trials?
> 
> That directory is simply the root of my local Emacs source code tree.
> I guess that what confused you is the "d.moncayo.melgar" part, which
> was "dani" in previous messages.  Well, the difference is due that one
> is the desktop I use at work, and the other is my personal laptop.

Please make a point of telling this in the future.  I have no way of
knowing when you make such changes, and they add more confusion to
this already complicated issue.

> > If the directory exists, I don't understand the reason for these
> > failures:
> >
> >   /bin/makeinfo --force --enable-encoding -I C:/msys/home/d.moncayo.melgar/emacs/emacs.git/doc/lispref/../emacs -I C:/msys/home/d.moncayo.melgar/emacs/emacs.git/doc/lispref --no-split -o C:/msys/home/d.moncayo.melgar/emacs/emacs.git/doc/lispref/../../info/elisp.info C:/msys/home/d.moncayo.melgar/emacs/emacs.git/doc/lispref/elisp.texi
> >   C:/msys/home/d.moncayo.melgar/emacs/emacs.git/doc/lispref/elisp.texi:58: @include `emacsver.texi': No such file or directory.
> >
> > Is this some problem with the MSYS port of makeinfo, perhaps?
> 
> I've always used the makeinfo that comes with MSYS, yes.  But that
> version have always worked fine for me.  This problem appears just
> when I try your proposed change in "configure.ac".

If you invoke the above command from the MSYS Bash prompt, does it
also fail in the same way?

> I've tried it again:  autogen.sh + msysconfig.sh + "make bootstrap",
> with this single change in my Emacs tree:
> 
>   diff --git a/configure.ac b/configure.ac
>   index 86a5f30..cb8f6d6 100644
>   --- a/configure.ac
>   +++ b/configure.ac
>   @@ -443,6 +443,8 @@ case "${srcdir}" in
>      *  ) srcdir=`(cd "$srcdir"; pwd)` ;;
>    esac
> 
>   +test "$MSYSTEM" = "MINGW32" && srcdir=`(cd "$srcdir"; pwd -W)`
>   +
>    ### Canonicalize the configuration name.
> 
>    AC_CANONICAL_HOST
> 
> The result in my laptop seems to be the same than in my office
> desktop: autogen ok, configure ok, but bootstrap failure  (new logfile
> attached).

What about this variant:

 test "$MSYSTEM" = "MINGW32" && srcdir=`(cd "$srcdir"; pwd -W | sed -e 's,^\([A-Za-z]\):,/\1,')`

It should produce srcdir in the /d/foo/bar format.

> Perhaps you could try this yourself.

Sorry, I don't have time for this.  If the above still does not work,
then for now, building under the MSYS root will be unsupported, until
someone figures a way of making it work and submits the necessary
patches.

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

[-- Attachment #1: Type: text/plain, Size: 3071 bytes --]

> Please make a point of telling this in the future.  I have no way of
> knowing when you make such changes, and they add more confusion to
> this already complicated issue.

I'm sorry.  I thought the change in the user name will not confuse you.

>> > If the directory exists, I don't understand the reason for these
>> > failures:
>> >
>> >   /bin/makeinfo --force --enable-encoding -I C:/msys/home/d.moncayo.melgar/emacs/emacs.git/doc/lispref/../emacs -I C:/msys/home/d.moncayo.melgar/emacs/emacs.git/doc/lispref --no-split -o C:/msys/home/d.moncayo.melgar/emacs/emacs.git/doc/lispref/../../info/elisp.info C:/msys/home/d.moncayo.melgar/emacs/emacs.git/doc/lispref/elisp.texi
>> >   C:/msys/home/d.moncayo.melgar/emacs/emacs.git/doc/lispref/elisp.texi:58: @include `emacsver.texi': No such file or directory.
>> >
>> > Is this some problem with the MSYS port of makeinfo, perhaps?
>>
>> I've always used the makeinfo that comes with MSYS, yes.  But that
>> version have always worked fine for me.  This problem appears just
>> when I try your proposed change in "configure.ac".
>
> If you invoke the above command from the MSYS Bash prompt, does it
> also fail in the same way?

Yes, it fails the same way.  I'm attaching the output I get from that
command, when invoked directly from bash.

>> I've tried it again:  autogen.sh + msysconfig.sh + "make bootstrap",
>> with this single change in my Emacs tree:
>>
>>   diff --git a/configure.ac b/configure.ac
>>   index 86a5f30..cb8f6d6 100644
>>   --- a/configure.ac
>>   +++ b/configure.ac
>>   @@ -443,6 +443,8 @@ case "${srcdir}" in
>>      *  ) srcdir=`(cd "$srcdir"; pwd)` ;;
>>    esac
>>
>>   +test "$MSYSTEM" = "MINGW32" && srcdir=`(cd "$srcdir"; pwd -W)`
>>   +
>>    ### Canonicalize the configuration name.
>>
>>    AC_CANONICAL_HOST
>>
>> The result in my laptop seems to be the same than in my office
>> desktop: autogen ok, configure ok, but bootstrap failure  (new logfile
>> attached).
>
> What about this variant:
>
>  test "$MSYSTEM" = "MINGW32" && srcdir=`(cd "$srcdir"; pwd -W | sed -e 's,^\([A-Za-z]\):,/\1,')`
>
> It should produce srcdir in the /d/foo/bar format.

It didn't produce that format, because the configure script generated
by "autogen.sh" a slightly different version of that line: the square
brackets in "[A-Za-z]" were removed.

I managed to get the intended conversion with this line in "configure.ac":

  test "$MSYSTEM" = "MINGW32" && srcdir=`(cd "$srcdir"; pwd -W | sed
-e 's,^\([[A-Za-z]]\):,/\1,')`

With that change, now the only problems remaining are the crashes I
get much later in the bootstraping, when Emacs compiles certain Elisp
files.

I already saw these severals days ago when I invoked msysconfig like this:
  (environment vars)
/c/msys/home/dani/emacs/emacs.git/nt/msysconfig.sh --enable-checking

Perhaps these crashes are due to a different problem.

I'm attaching:
* The "config.log" file.
* The summary of the configure script.
* The output of "make bootstrap".

-- 
Dani Moncayo

[-- Attachment #2: attachments.zip --]
[-- Type: application/zip, Size: 79745 bytes --]

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Wed, 18 Sep 2013 11:32:09 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
>   test "$MSYSTEM" = "MINGW32" && srcdir=`(cd "$srcdir"; pwd -W | sed
> -e 's,^\([[A-Za-z]]\):,/\1,')`
> 
> With that change, now the only problems remaining are the crashes I
> get much later in the bootstraping, when Emacs compiles certain Elisp
> files.

Do the same crashes happen if you build Emacs outside of the MSYS
tree?

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

>>   test "$MSYSTEM" = "MINGW32" && srcdir=`(cd "$srcdir"; pwd -W | sed
>> -e 's,^\([[A-Za-z]]\):,/\1,')`
>>
>> With that change, now the only problems remaining are the crashes I
>> get much later in the bootstraping, when Emacs compiles certain Elisp
>> files.
>
> Do the same crashes happen if you build Emacs outside of the MSYS
> tree?

I've just tried it and yes, I get the same 3 crashes
  Makefile:252: recipe for target `ffap.elc' failed
  Makefile:252: recipe for target `gnus/deuglify.elc' failed
  Makefile:252: recipe for target `obsolete/xesam.elc' failed


-- 
Dani Moncayo

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Wed, 18 Sep 2013 12:38:29 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
> >>   test "$MSYSTEM" = "MINGW32" && srcdir=`(cd "$srcdir"; pwd -W | sed
> >> -e 's,^\([[A-Za-z]]\):,/\1,')`
> >>
> >> With that change, now the only problems remaining are the crashes I
> >> get much later in the bootstraping, when Emacs compiles certain Elisp
> >> files.
> >
> > Do the same crashes happen if you build Emacs outside of the MSYS
> > tree?
> 
> I've just tried it and yes, I get the same 3 crashes
>   Makefile:252: recipe for target `ffap.elc' failed
>   Makefile:252: recipe for target `gnus/deuglify.elc' failed
>   Makefile:252: recipe for target `obsolete/xesam.elc' failed

Probably some other issue, but for a good measure try building from
the source tree, i.e. in-place.

Are these crashes, or just failures to byte-compile with some error
message from the byte compiler?

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

>> I've just tried it and yes, I get the same 3 crashes
>>   Makefile:252: recipe for target `ffap.elc' failed
>>   Makefile:252: recipe for target `gnus/deuglify.elc' failed
>>   Makefile:252: recipe for target `obsolete/xesam.elc' failed
>
> Probably some other issue, but for a good measure try building from
> the source tree, i.e. in-place.

Will do, but before that I'd like to know if you need me to print some
values from the gdb session I have (see my other reply about the
backtrace).

> Are these crashes, or just failures to byte-compile with some error
> message from the byte compiler?

These are crashes.  Looking at the output of "make bootstrap", it
seems that the byte-compilation didn't barfed:

. . .
Compiling /C/msys/home/d.moncayo.melgar/emacs/emacs.git/lisp/ffap.el
Wrote c:/msys/home/d.moncayo.melgar/emacs/emacs.git/lisp/ffap.elc
Makefile:252: recipe for target `ffap.elc' failed
make[3]: *** [ffap.elc] Error 255
make[3]: Leaving directory `/home/d.moncayo.melgar/emacs/build/lisp'
make[3]: Entering directory `/home/d.moncayo.melgar/emacs/build/lisp'
Compiling /C/msys/home/d.moncayo.melgar/emacs/emacs.git/lisp/emacs-lisp/eieio-base.el
Wrote c:/msys/home/d.moncayo.melgar/emacs/emacs.git/lisp/emacs-lisp/eieio-base.elc
. . .



-- 
Dani Moncayo

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

> I've just tried it and yes, I get the same 3 crashes
>   Makefile:252: recipe for target `ffap.elc' failed
>   Makefile:252: recipe for target `gnus/deuglify.elc' failed
>   Makefile:252: recipe for target `obsolete/xesam.elc' failed

I've been able to get a backtrace of the first crash (compiling "ffap.elc").

When the crash message popped up, I attached a gdb session to that
process (gdb --pid=XXX), and then ran "thread apply all bt", getting
the following backtrace:


Thread 3 (Thread 4284.0x1c40):
#0  0x77ce000d in ntdll!DbgBreakPoint () from C:\Windows\SysWOW64\ntdll.dll
#1  0x77d6f896 in ntdll!DbgUiRemoteBreakin ()
   from C:\Windows\SysWOW64\ntdll.dll
#2  0x1b60f584 in ?? ()
#3  0x00000000 in ?? ()

Thread 2 (Thread 4284.0x24c):
#0  0x77cf013d in ntdll!ZwWaitForMultipleObjects ()
   from C:\Windows\SysWOW64\ntdll.dll
#1  0x77cf013d in ntdll!ZwWaitForMultipleObjects ()
   from C:\Windows\SysWOW64\ntdll.dll
#2  0x77d22f51 in ntdll!RtlMoveMemory () from C:\Windows\SysWOW64\ntdll.dll
#3  0x00000001 in ?? ()
#4  0x00000001 in ?? ()
#5  0x00000000 in ?? ()

Thread 1 (Thread 4284.0x2754):
#0  0x77cf013d in ntdll!ZwWaitForMultipleObjects ()
   from C:\Windows\SysWOW64\ntdll.dll
#1  0x77cf013d in ntdll!ZwWaitForMultipleObjects ()
   from C:\Windows\SysWOW64\ntdll.dll
#2  0x75b815e9 in WaitForMultipleObjectsEx ()
   from C:\Windows\syswow64\KernelBase.dll
#3  0x00000002 in ?? ()
#4  0x0088e8cc in ?? ()
#5  0x75921a2c in WaitForMultipleObjectsEx ()
   from C:\Windows\syswow64\kernel32.dll
#6  0x75924220 in WaitForMultipleObjects ()
   from C:\Windows\syswow64\kernel32.dll
#7  0x759480c4 in KERNEL32!GetApplicationRecoveryCallback ()
   from C:\Windows\syswow64\kernel32.dll
#8  0x75947f83 in KERNEL32!GetApplicationRecoveryCallback ()
   from C:\Windows\syswow64\kernel32.dll
#9  0x75947878 in UnhandledExceptionFilter ()
   from C:\Windows\syswow64\kernel32.dll
#10 0x0088eaf8 in ?? ()
#11 0x759477f7 in UnhandledExceptionFilter ()
   from C:\Windows\syswow64\kernel32.dll
#12 0x0088eaf8 in ?? ()
#13 0x776b8f74 in msvcrt!abort () from C:\Windows\syswow64\msvcrt.dll
#14 0x6e956f62 in libgcc_s_dw2-1!__deregister_frame_info_bases ()
   from c:\mingw-emacs\bin\libgcc_s_dw2-1.dll
#15 0x03d5c428 in ?? ()
#16 0x7765c3e9 in msvcrt!isspace () from C:\Windows\syswow64\msvcrt.dll
#17 0x776636bb in msvcrt!exit () from C:\Windows\syswow64\msvcrt.dll
#18 0x010db6e3 in Fkill_emacs (arg=0) at c:/emacs/emacs.git/src/emacs.c:1899
#19 0x0116bd70 in Ffuncall (nargs=2, args=0x88ef4c)
    at c:/emacs/emacs.git/src/eval.c:2856
#20 0x011ac471 in exec_byte_code (bytestr=57861185, vector=61452157,
    maxdepth=40, args_template=1024, nargs=0, args=0x88f280)
    at c:/emacs/emacs.git/src/bytecode.c:905
#21 0x0116c552 in funcall_lambda (fun=61448165, nargs=0, arg_vector=0x88f280)
    at c:/emacs/emacs.git/src/eval.c:3024
#22 0x0116bfac in Ffuncall (nargs=1, args=0x88f27c)
    at c:/emacs/emacs.git/src/eval.c:2905
#23 0x011ac471 in exec_byte_code (bytestr=19538345, vector=19538365,
    maxdepth=88, args_template=1028, nargs=1, args=0x88f57c)
    at c:/emacs/emacs.git/src/bytecode.c:905
#24 0x0116c552 in funcall_lambda (fun=19538325, nargs=1, arg_vector=0x88f578)
    at c:/emacs/emacs.git/src/eval.c:3024
#25 0x0116bfac in Ffuncall (nargs=2, args=0x88f574)
    at c:/emacs/emacs.git/src/eval.c:2905
#26 0x011ac471 in exec_byte_code (bytestr=19525513, vector=19525533,
    maxdepth=68, args_template=0, nargs=0, args=0x88f89c)
    at c:/emacs/emacs.git/src/bytecode.c:905
#27 0x0116c552 in funcall_lambda (fun=19525493, nargs=0, arg_vector=0x88f89c)
    at c:/emacs/emacs.git/src/eval.c:3024
#28 0x0116bfac in Ffuncall (nargs=1, args=0x88f898)
    at c:/emacs/emacs.git/src/eval.c:2905
#29 0x011ac471 in exec_byte_code (bytestr=19523977, vector=19523997,
    maxdepth=32, args_template=0, nargs=0, args=0x88fb00)
    at c:/emacs/emacs.git/src/bytecode.c:905
#30 0x0116c552 in funcall_lambda (fun=19523957, nargs=0, arg_vector=0x88fb00)
    at c:/emacs/emacs.git/src/eval.c:3024
#31 0x0116c271 in apply_lambda (fun=19523957, args=57370650)
    at c:/emacs/emacs.git/src/eval.c:2965
#32 0x0116ac19 in eval_sub (form=59078214)
    at c:/emacs/emacs.git/src/eval.c:2271
#33 0x0116a239 in Feval (form=59078214, lexical=57370650)
    at c:/emacs/emacs.git/src/eval.c:2044
#34 0x010dd455 in top_level_2 () at c:/emacs/emacs.git/src/keyboard.c:1172
#35 0x01168bb8 in internal_condition_case (bfun=0x10dd438 <top_level_2>,
    handlers=57425114, hfun=0x10dcff4 <cmd_error>)
    at c:/emacs/emacs.git/src/eval.c:1339
#36 0x010dd489 in top_level_1 (ignore=57370650)
    at c:/emacs/emacs.git/src/keyboard.c:1180
#37 0x011684d2 in internal_catch (tag=57414994, func=0x10dd457 <top_level_1>,
    arg=57370650) at c:/emacs/emacs.git/src/eval.c:1113
#38 0x010dd3bb in command_loop () at c:/emacs/emacs.git/src/keyboard.c:1141
#39 0x010dcb91 in recursive_edit_1 () at c:/emacs/emacs.git/src/keyboard.c:781
#40 0x010dcd4d in Frecursive_edit () at c:/emacs/emacs.git/src/keyboard.c:845
#41 0x010db01b in main (argc=9, argv=0x3d51500)
    at c:/emacs/emacs.git/src/emacs.c:1570

-- 
Dani Moncayo

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Wed, 18 Sep 2013 14:31:06 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
> #11 0x759477f7 in UnhandledExceptionFilter ()
>    from C:\Windows\syswow64\kernel32.dll
> #12 0x0088eaf8 in ?? ()
> #13 0x776b8f74 in msvcrt!abort () from C:\Windows\syswow64\msvcrt.dll
> #14 0x6e956f62 in libgcc_s_dw2-1!__deregister_frame_info_bases ()
>    from c:\mingw-emacs\bin\libgcc_s_dw2-1.dll
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> #15 0x03d5c428 in ?? ()
> #16 0x7765c3e9 in msvcrt!isspace () from C:\Windows\syswow64\msvcrt.dll
> #17 0x776636bb in msvcrt!exit () from C:\Windows\syswow64\msvcrt.dll
> #18 0x010db6e3 in Fkill_emacs (arg=0) at c:/emacs/emacs.git/src/emacs.c:1899

That libgcc_s_dw2-1.dll is the problem, I think.  I think this might
be the same issue as summarized here:

  http://sourceforge.net/mailarchive/message.php?msg_id=31010191

and in the links in that message.

Alternatively, perhaps you have conflicting versions of
libgcc_s_dw2-1.dll on your system.

Questions:

  . Did you install a different version of GCC, as part of your MinGW
    re-installation?  If so, which GCC version do you have now, and
    which one did you have before?

  . Does temacs.exe you produce depend on libgcc_s_dw2-1.dll?  (Use
    the dependency walker or objdump to find out.)  If not, perhaps
    some libraries that Emacs loads depend on that DLL?

  . If you add -shared-libgcc switch to the temacs link command line,
    does the problem go away?

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

[-- Attachment #1: Type: text/plain, Size: 2172 bytes --]

> Questions:
>
>   . Did you install a different version of GCC, as part of your MinGW
>     re-installation?  If so, which GCC version do you have now, and
>     which one did you have before?

I have now the same version of GCC as before: 4.7.2.

>   . Does temacs.exe you produce depend on libgcc_s_dw2-1.dll?  (Use
>     the dependency walker or objdump to find out.)

I don't see that dependency, but FWIW, I'm attaching its dependencies
(as exported by "dependency walker").

> If not, perhaps
>     some libraries that Emacs loads depend on that DLL?

I don't know.  FWIW, I'm building Emacs with the usual libraries:
  giflib-4.1.4-1
  gnutls-3.0.9
  jpeg-6b-4
  libpng-dev_1.4.3-1
  libxml2-2.7.8
  tiff-3.8.2-1
  libXpm-3.5.11

And their dependency libraries:
  zlib-1.2.8-1
  pkg-config_0.26-1
  glib_2.28.8-1
  p11-kit-0.9
  gettext-runtime_0.18.1.1-2
  libiconv-1.14-2

I could try first bootstrapping without libraries and then add the
libraries one by one, to determine which library triggers the problem
(if any).

>   . If you add -shared-libgcc switch to the temacs link command line,
>     does the problem go away?

Yes, it goes away!  :)

I've been able to do a full bootstrap with the patch below applied to
my source code tree.

diff --git a/configure.ac b/configure.ac
index 86a5f30..1af4f16 100644
--- a/configure.ac
+++ b/configure.ac
@@ -443,6 +443,8 @@ case "${srcdir}" in
   *  ) srcdir=`(cd "$srcdir"; pwd)` ;;
 esac

+test "$MSYSTEM" = "MINGW32" && srcdir=`(cd "$srcdir"; pwd -W | sed -e
's,^\([[A-Za-z]]\):,/\1,')`
+
 ### Canonicalize the configuration name.

 AC_CANONICAL_HOST
diff --git a/src/Makefile.in b/src/Makefile.in
index 254aa17..ff5ee20 100644
--- a/src/Makefile.in
+++ b/src/Makefile.in
@@ -106,7 +106,7 @@ LD_SWITCH_SYSTEM=@LD_SWITCH_SYSTEM@
 LD_SWITCH_SYSTEM_TEMACS=@LD_SWITCH_SYSTEM_TEMACS@

 ## Flags to pass to ld only for temacs.
-TEMACS_LDFLAGS = $(LD_SWITCH_SYSTEM) $(LD_SWITCH_SYSTEM_TEMACS)
+TEMACS_LDFLAGS = $(LD_SWITCH_SYSTEM) $(LD_SWITCH_SYSTEM_TEMACS) -shared-libgcc

 ## If available, the full path to the paxctl program.
 ## On grsecurity/PaX systems, unexec will fail due to a gap between




-- 
Dani Moncayo

[-- Attachment #2: temacs-dependencies.zip --]
[-- Type: application/zip, Size: 723423 bytes --]

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Wed, 18 Sep 2013 18:51:28 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
> >   . Does temacs.exe you produce depend on libgcc_s_dw2-1.dll?  (Use
> >     the dependency walker or objdump to find out.)
> 
> I don't see that dependency, but FWIW, I'm attaching its dependencies
> (as exported by "dependency walker").
> 
> > If not, perhaps
> >     some libraries that Emacs loads depend on that DLL?
> 
> I don't know.  FWIW, I'm building Emacs with the usual libraries:
>   giflib-4.1.4-1
>   gnutls-3.0.9
>   jpeg-6b-4
>   libpng-dev_1.4.3-1
>   libxml2-2.7.8
>   tiff-3.8.2-1
>   libXpm-3.5.11
> 
> And their dependency libraries:
>   zlib-1.2.8-1
>   pkg-config_0.26-1
>   glib_2.28.8-1
>   p11-kit-0.9
>   gettext-runtime_0.18.1.1-2
>   libiconv-1.14-2

gnutls also depends on libintl, and either libintl or libiconv (I
don't remember which one) might depend on libgcc_s_dw2-1.dll.

> >   . If you add -shared-libgcc switch to the temacs link command line,
> >     does the problem go away?
> 
> Yes, it goes away!  :)

Then I'm quite sure you have a dependency on libgcc_s_dw2-1.dll in one
of the other DLLs.  All the symptoms are exactly as described in

  http://sourceforge.net/mailarchive/message.php?msg_id=31010191

The solution is to use libiconv and libintl from the ezwinports site,
they don't have that problem.

Note that building Emacs with -shared-libgcc explicitly makes
temacs.exe and emacs.exe dependent on libgcc_s_dw2-1.dll, so you must
carry that DLL together with the executables.  That's why I don't like
this solution, if it can be avoided.

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

> Then I'm quite sure you have a dependency on libgcc_s_dw2-1.dll in one
> of the other DLLs.  All the symptoms are exactly as described in
>
>   http://sourceforge.net/mailarchive/message.php?msg_id=31010191
>
> The solution is to use libiconv and libintl from the ezwinports site,
> they don't have that problem.

Confirmed: When I use earlier versions of those libraries
(libiconv-1.13.1-1 and libintl-0.17-1), I finally can bootstrap
successfully.

Thank you very much!

-- 
Dani Moncayo

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Fri, 20 Sep 2013 00:56:15 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
> > Then I'm quite sure you have a dependency on libgcc_s_dw2-1.dll in one
> > of the other DLLs.  All the symptoms are exactly as described in
> >
> >   http://sourceforge.net/mailarchive/message.php?msg_id=31010191
> >
> > The solution is to use libiconv and libintl from the ezwinports site,
> > they don't have that problem.
> 
> Confirmed: When I use earlier versions of those libraries
> (libiconv-1.13.1-1 and libintl-0.17-1), I finally can bootstrap
> successfully.

Great.

Andy, could this be your problem as well?

Re: Building Emacs from a new MinGW environment

[Andy Moreton]

On Fri 20 Sep 2013, Eli Zaretskii wrote:

>> Date: Fri, 20 Sep 2013 00:56:15 +0200
>> From: Dani Moncayo <dmoncayo@gmail.com>
>> Cc: Emacs development discussions <emacs-devel@gnu.org>
>> 
>> > Then I'm quite sure you have a dependency on libgcc_s_dw2-1.dll in one
>> > of the other DLLs.  All the symptoms are exactly as described in
>> >
>> >   http://sourceforge.net/mailarchive/message.php?msg_id=31010191> >
>> > The solution is to use libiconv and libintl from the ezwinports site,
>> > they don't have that problem.
>> 
>> Confirmed: When I use earlier versions of those libraries
>> (libiconv-1.13.1-1 and libintl-0.17-1), I finally can bootstrap
>> successfully.
>
> Great.
>
> Andy, could this be your problem as well?

Could be - c:\mingw\bin\libintl-8.dll has a dependency on
libgcc_s_dw2-1.dll on one of the machines, so it's likely to be the same
issue.

    AdnyM

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

>>> > Then I'm quite sure you have a dependency on libgcc_s_dw2-1.dll in one
>>> > of the other DLLs.  All the symptoms are exactly as described in
>>> >
>>> >   http://sourceforge.net/mailarchive/message.php?msg_id=31010191> >
>>> > The solution is to use libiconv and libintl from the ezwinports site,
>>> > they don't have that problem.
>>>
>>> Confirmed: When I use earlier versions of those libraries
>>> (libiconv-1.13.1-1 and libintl-0.17-1), I finally can bootstrap
>>> successfully.
>>
>> Great.
>>
>> Andy, could this be your problem as well?
>
> Could be - c:\mingw\bin\libintl-8.dll has a dependency on
> libgcc_s_dw2-1.dll on one of the machines, so it's likely to be the same
> issue.

For the record: I've just seen another library which have this
problem: "libz-1.dll".

Version "libz-1.2.7-1" depends on "libgcc_s_dw2-1.dll" and therefore
makes Emacs crash.  The problem goes away with the older version
"libz-1.2.3-1".

-- 
Dani Moncayo

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Fri, 20 Sep 2013 13:08:59 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> 
> For the record: I've just seen another library which have this
> problem: "libz-1.dll".
> 
> Version "libz-1.2.7-1" depends on "libgcc_s_dw2-1.dll" and therefore
> makes Emacs crash.  The problem goes away with the older version
> "libz-1.2.3-1".

Is libz-1.dll loaded by Emacs itself, or does some other library
depend on it?

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

>> For the record: I've just seen another library which have this
>> problem: "libz-1.dll".
>>
>> Version "libz-1.2.7-1" depends on "libgcc_s_dw2-1.dll" and therefore
>> makes Emacs crash.  The problem goes away with the older version
>> "libz-1.2.3-1".
>
> Is libz-1.dll loaded by Emacs itself, or does some other library
> depend on it?

I don't know.  How can I find that out?

If I open my recently built "emacs.exe" with "dependency walker", I
don't see a dependency with libz, neither direct nor through another
library, but the dependency tree is too large to be sure, and the
"edit > find" option is disabled.

-- 
Dani Moncayo

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

>>> Version "libz-1.2.7-1" depends on "libgcc_s_dw2-1.dll" and therefore
>>> makes Emacs crash.  The problem goes away with the older version
>>> "libz-1.2.3-1".
>>
>> Is libz-1.dll loaded by Emacs itself, or does some other library
>> depend on it?
>
> I don't know.  How can I find that out?

FWIW: I needed to include zlib in my build environment because it is
required by libpng and libxml (if my notes are correct).

-- 
Dani Moncayo

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Fri, 20 Sep 2013 14:30:02 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
> FWIW: I needed to include zlib in my build environment because it is
> required by libpng and libxml (if my notes are correct).

Use the dependency walker on those -- they should depend on zlib1.dll,
not libz-1.dll.  The former doesn't have the problem of being
dependent on libgcc DLL.

Don't you also have zlib1.dll on your system?  If you do, Emacs
prefers it to libz-1.dll.

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Fri, 20 Sep 2013 14:22:36 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
> > Is libz-1.dll loaded by Emacs itself, or does some other library
> > depend on it?
> 
> I don't know.  How can I find that out?

Use the dependency walker on the DLLs that are loaded by Emacs (image
DLLs, GnuTLS, libxml2).

> If I open my recently built "emacs.exe" with "dependency walker", I
> don't see a dependency with libz

That's expected, since Emacs loads it dynamically when needed.

> the dependency tree is too large to be sure, and the
> "edit > find" option is disabled.

The tree size will be much smaller, once you collapse all the branches
that begin from Windows system DLLs, since none of them cannot
possibly depend on zlib.

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Fri, 20 Sep 2013 13:08:59 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> 
> For the record: I've just seen another library which have this
> problem: "libz-1.dll".
> 
> Version "libz-1.2.7-1" depends on "libgcc_s_dw2-1.dll" and therefore
> makes Emacs crash.  The problem goes away with the older version
> "libz-1.2.3-1".

No need to compromise for an older version.  I have just uploaded to
the ezwinports site the latest zlib 1.2.8 built for Windows without
any dependencies on libgcc DLL.  You can find it here:

  http://sourceforge.net/projects/ezwinports/files/zlib-1.2.8-w32-bin.zip/download

Assuming none of your other libraries explicitly depend on libz-1.dll,
you should be fine, because Emacs prefers zlib1.dll to libz-1.dll
(assuming they are in the same directory), and the optional libraries
we recommend all depend on zlib1.dll also.

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

>> For the record: I've just seen another library which have this
>> problem: "libz-1.dll".
>>
>> Version "libz-1.2.7-1" depends on "libgcc_s_dw2-1.dll" and therefore
>> makes Emacs crash.  The problem goes away with the older version
>> "libz-1.2.3-1".
>
> No need to compromise for an older version.  I have just uploaded to
> the ezwinports site the latest zlib 1.2.8 built for Windows without
> any dependencies on libgcc DLL.  You can find it here:
>
>   http://sourceforge.net/projects/ezwinports/files/zlib-1.2.8-w32-bin.zip/download
>
> Assuming none of your other libraries explicitly depend on libz-1.dll,
> you should be fine, because Emacs prefers zlib1.dll to libz-1.dll
> (assuming they are in the same directory), and the optional libraries
> we recommend all depend on zlib1.dll also.

It works, thank you.

-- 
Dani Moncayo

Re: Building Emacs from a new MinGW environment

[Andy Moreton]

On Wed 18 Sep 2013, Eli Zaretskii wrote:

>> Date: Wed, 18 Sep 2013 11:32:09 +0200
>> From: Dani Moncayo <dmoncayo@gmail.com>
>> Cc: Emacs development discussions <emacs-devel@gnu.org>
>> 
>>   test "$MSYSTEM" = "MINGW32" && srcdir=`(cd "$srcdir"; pwd -W | sed
>> -e 's,^\([[A-Za-z]]\):,/\1,')`
>> 
>> With that change, now the only problems remaining are the crashes I
>> get much later in the bootstraping, when Emacs compiles certain Elisp
>> files.
>
> Do the same crashes happen if you build Emacs outside of the MSYS
> tree?

I have an unchanged mingw toochain I have beeen using successfully for
months, and I also see a crash during bootstrap.

From my build log for bootstrap of r114374, it crashed while compiling
lisp/cedet/semantic/db-ebrowse.el. Attaching gdb and issuing 'thread
apply all bt' did not obtain a backtrace:

(gdb) The program being debugged exited while in a function called from GDB.
Evaluation of the expression containing the function
(backtrace_top) will be abandoned.

The rest of the build continued to completion. This seems to be
repeatable. Anything I can do to help diagnose it ?

    AndyM

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Wed, 18 Sep 2013 11:46:44 +0100
> 
> I have an unchanged mingw toochain I have beeen using successfully for
> months, and I also see a crash during bootstrap.
> 
> >From my build log for bootstrap of r114374, it crashed while compiling
> lisp/cedet/semantic/db-ebrowse.el.

What exactly does "crash" mean here?

Anyway, this is a different file from those mentioned by Dani.

Are you building from the source tree, i.e. in-place?  That's what I
do, and I just bootstrapped today (twice) with no problems.

> Attaching gdb and issuing 'thread apply all bt' did not obtain a
> backtrace:
> 
> (gdb) The program being debugged exited while in a function called from GDB.
> Evaluation of the expression containing the function
> (backtrace_top) will be abandoned.

Rename .gdbinit, so that xbacktrace is not invoked, and try obtaining
the backtrace again.

> The rest of the build continued to completion. This seems to be
> repeatable. Anything I can do to help diagnose it ?

A backtrace would be a good start ;-)

TIA

Re: Building Emacs from a new MinGW environment

[Sean Sieger]

Eli Zaretskii <eliz@gnu.org> writes:

    Are you building from the source tree, i.e. in-place?  That's what I
    do, and I just bootstrapped today (twice) with no problems.

Sorry for butting in.  Eli, is the MinGW set-up you are using different
than the one indicated in /c/trunk/nt/INSTALL?

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> From: Sean Sieger <sean.sieger@gmail.com>
> Date: Wed, 18 Sep 2013 08:44:58 -0400
> 
> Eli, is the MinGW set-up you are using different than the one
> indicated in /c/trunk/nt/INSTALL?

No, it is not different.

Re: Building Emacs from a new MinGW environment

[Sean Sieger]

Eli Zaretskii <eliz@gnu.org> writes:

    > Eli, is the MinGW set-up you are using different than the one
    > indicated in /c/trunk/nt/INSTALL?

    No, it is not different.

But the MinGW Installer installs the mingwrt-4.

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> From: Sean Sieger <sean.sieger@gmail.com>
> Date: Wed, 18 Sep 2013 09:19:43 -0400
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
>     > Eli, is the MinGW set-up you are using different than the one
>     > indicated in /c/trunk/nt/INSTALL?
> 
>     No, it is not different.
> 
> But the MinGW Installer installs the mingwrt-4.

I never used the installer, I always install MinGW and MSYS packages
by hand.

Re: Building Emacs from a new MinGW environment

[Andy Moreton]

On Wed 18 Sep 2013, Eli Zaretskii wrote:

>> From: Andy Moreton <andrewjmoreton@gmail.com>
>> Date: Wed, 18 Sep 2013 11:46:44 +0100
>> 
>> I have an unchanged mingw toochain I have beeen using successfully for
>> months, and I also see a crash during bootstrap.
>> 
>> >From my build log for bootstrap of r114374, it crashed while compiling
>> lisp/cedet/semantic/db-ebrowse.el.
>
> What exactly does "crash" mean here?

Emacs abort dialog box appears during elisp compilation.

> Anyway, this is a different file from those mentioned by Dani.

Agreed.

> Are you building from the source tree, i.e. in-place?  That's what I
> do, and I just bootstrapped today (twice) with no problems.

Bzr branch in   c:\emacs\src\emacs\trunk
Build output in c:\emacs\src\emacs\trunk\build-msys

> Rename .gdbinit, so that xbacktrace is not invoked, and try obtaining
> the backtrace again.

Thanks for the hint - backtrace follows. I've omitted uninteresting
threads that do not have emacs in the call stack.

GNU gdb (GDB) 7.5
...Load .gdbinit
Attaching to process 6828
[New Thread 6828.0x1098]
[New Thread 6828.0x1410]
[New Thread 6828.0x7cc]
[New Thread 6828.0x15bc]
[New Thread 6828.0x1524]
[New Thread 6828.0x1654]
Reading symbols from c:\emacs\src\emacs\trunk\build-msys\src\emacs.exe...done.
(gdb) ./src/.gdbinit: No such file or directory.
(gdb) Continuing.

Program received signal SIGTRAP, Trace/breakpoint trap.
[Switching to Thread 6828.0x1098]
0x75e6321a in KERNELBASE!DeleteAce () from C:\Windows\syswow64\KernelBase.dll
(gdb) thread apply all bt full

... [uninteresting threads omitted] ...

Thread 1 (Thread 6828.0x1098):
#0  0x75e6321a in KERNELBASE!DeleteAce () from C:\Windows\syswow64\KernelBase.dll
No symbol table info available.
#1  0x0118ddad in emacs_abort () at c:/emacs/src/emacs/trunk/src/w32fns.c:7988
        button = <optimized out>
#2  0x010ae31d in terminate_due_to_signal (sig=sig@entry=0xb, backtrace_limit=backtrace_limit@entry=0x28) at c:/emacs/src/emacs/trunk/src/emacs.c:369
No locals.
#3  0x010c730d in handle_fatal_signal (sig=0xb) at c:/emacs/src/emacs/trunk/src/sysdep.c:1626
No locals.
#4  deliver_thread_signal (sig=0xb, handler=<optimized out>) at c:/emacs/src/emacs/trunk/src/sysdep.c:1600
        handler = 0x10c71c2 <handle_fatal_signal>
#5  deliver_fatal_thread_signal (sig=0xb) at c:/emacs/src/emacs/trunk/src/sysdep.c:1638
No locals.
#6  0x010011ea in _gnu_exception_handler@4 ()
No symbol table info available.
#7  0x75d2fffb in KERNEL32!GetQueuedCompletionStatus () from C:\Windows\syswow64\kernel32.dll
No symbol table info available.
#8  0x0088bffc in ?? ()
No symbol table info available.
#9  0x76f474ff in ntdll!AlpcMaxAllowedMessageLength () from C:\Windows\SysWOW64\ntdll.dll
No symbol table info available.
#10 0x0088bffc in ?? ()
No symbol table info available.
#11 0x76f09f45 in ntdll!RtlpNtSetValueKey () from C:\Windows\SysWOW64\ntdll.dll
No symbol table info available.
#12 0x01160b7e in sprintf (__stream=0x0, __format=0x1419ae0 "Copying raw data for %.8s...", __format=0x1419ae0 "Copying raw data for %.8s...") at c:/mingw/bin/../lib/gcc/mingw32/4.7.2/../../../../include/stdio.h:269
        __retval = 0x6
        __local_argv = 0x88ffec ""
#13 0x7efde000 in ?? ()
No symbol table info available.
#14 0x00000000 in ?? ()
No symbol table info available.

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Wed, 18 Sep 2013 14:21:14 +0100
> 
> > Are you building from the source tree, i.e. in-place?  That's what I
> > do, and I just bootstrapped today (twice) with no problems.
> 
> Bzr branch in   c:\emacs\src\emacs\trunk
> Build output in c:\emacs\src\emacs\trunk\build-msys

Can you try building in trunk itself?  I'd like to know if this is in
any way related to the fact that the source and the build trees are
different.

> #6  0x010011ea in _gnu_exception_handler@4 ()
> No symbol table info available.
> #7  0x75d2fffb in KERNEL32!GetQueuedCompletionStatus () from C:\Windows\syswow64\kernel32.dll
> No symbol table info available.
> #8  0x0088bffc in ?? ()
> No symbol table info available.
> #9  0x76f474ff in ntdll!AlpcMaxAllowedMessageLength () from C:\Windows\SysWOW64\ntdll.dll
> No symbol table info available.
> #10 0x0088bffc in ?? ()
> No symbol table info available.
> #11 0x76f09f45 in ntdll!RtlpNtSetValueKey () from C:\Windows\SysWOW64\ntdll.dll
> No symbol table info available.
> #12 0x01160b7e in sprintf (__stream=0x0, __format=0x1419ae0 "Copying raw data for %.8s...", __format=0x1419ae0 "Copying raw data for %.8s...") at c:/mingw/bin/../lib/gcc/mingw32/4.7.2/../../../../include/stdio.h:269

__stream being NULL in the call to sprintf is the problem, I think.
But the only instances of "Copying raw data for %.8s..." I could find
are in src/unexw32.c and nt/addsection.c, and there the first argument
is a local array that cannot possibly be NULL, so I'm unsure how this
could happen.  Moreover, both unexw32.c and addsection.c are unrelated
to byte-compiling, which just adds to the mystery...

Do you see the same crash if you byte-compile db-ebrowse.el by hand?
If so, try running the same command under GDB, you might get a better
backtrace.  Using the latest port of GDB (7.6.1 should be available)
might also help.

Re: Building Emacs from a new MinGW environment

[Andy Moreton]

On Wed 18 Sep 2013, Eli Zaretskii wrote:

[snipped]
>> #12 0x01160b7e in sprintf (__stream=0x0, __format=0x1419ae0 "Copying raw
>> data for %.8s...", __format=0x1419ae0 "Copying raw data for %.8s...") at
>> c:/mingw/bin/../lib/gcc/mingw32/4.7.2/../../../../include/stdio.h:269
>
> __stream being NULL in the call to sprintf is the problem, I think.
> But the only instances of "Copying raw data for %.8s..." I could find
> are in src/unexw32.c and nt/addsection.c, and there the first argument
> is a local array that cannot possibly be NULL, so I'm unsure how this
> could happen.  Moreover, both unexw32.c and addsection.c are unrelated
> to byte-compiling, which just adds to the mystery...

I suspect that gdb is slightly confused. The mingw stdio.h does include
a redirect to mingw supplied versions, but the argument names don't
quite match.

> Do you see the same crash if you byte-compile db-ebrowse.el by hand?
> If so, try running the same command under GDB, you might get a better
> backtrace.  Using the latest port of GDB (7.6.1 should be available)
> might also help.

Byte compiling that file from an uninstalled emacs works properly - it
only seems to fall over when byte compliling during bootstrap. I'll try
to update gdb and report back if that gives a more meaningful backtrace.

    AndyM

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Wed, 18 Sep 2013 11:32:09 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
> >  test "$MSYSTEM" = "MINGW32" && srcdir=`(cd "$srcdir"; pwd -W | sed -e 's,^\([A-Za-z]\):,/\1,')`
> >
> > It should produce srcdir in the /d/foo/bar format.
> 
> It didn't produce that format, because the configure script generated
> by "autogen.sh" a slightly different version of that line: the square
> brackets in "[A-Za-z]" were removed.
> 
> I managed to get the intended conversion with this line in "configure.ac":
> 
>   test "$MSYSTEM" = "MINGW32" && srcdir=`(cd "$srcdir"; pwd -W | sed
> -e 's,^\([[A-Za-z]]\):,/\1,')`
> 
> With that change, now the only problems remaining are the crashes I
> get much later in the bootstraping, when Emacs compiles certain Elisp
> files.

I have now installed this change in configure.ac, with a suitable
commentary.

Thanks.

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

> I have now installed this change in configure.ac, with a suitable
> commentary.

Thank you so much Eli.

Just one comment: what about this:

>> But the general principle is that, it is conceptually wrong to do
>> conversions of pathnames from MSYS format to native windows format
>> based on pattern substitution, assuming that the MSYS paths will
>> always be either in "/X/whatever" format or in "X:/whatever" format.
>
> If the change in configure.ac works, we will be able to remove almost
> all of that stuff in top-level Makefile.

?

-- 
Dani Moncayo

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Thu, 19 Sep 2013 10:56:42 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
> Just one comment: what about this:
> 
> >> But the general principle is that, it is conceptually wrong to do
> >> conversions of pathnames from MSYS format to native windows format
> >> based on pattern substitution, assuming that the MSYS paths will
> >> always be either in "/X/whatever" format or in "X:/whatever" format.
> >
> > If the change in configure.ac works, we will be able to remove almost
> > all of that stuff in top-level Makefile.
> 
> ?

I'd appreciate if someone else could work on this.  I'm trying to
finish up my work on TTY menus, which is dragging for more than a year
now.

In a nutshell, srcdir is now guaranteed to be in /d/foo/bar format,
and should be edited into d:/foo/bar when it is placed in
src/epaths.h.  The job is to remove anything from top-level
Makefile.in that is required for file names in format other than
/d/foo/bar, and still allow using %emacs_dir% in
'--enable-locallisppath=PATH' option to configure.  So I think
msys_to_w32 should stay intact, but msys_lisppath_to_w32 could be
simplified.

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

> In a nutshell, srcdir is now guaranteed to be in /d/foo/bar format,
> and should be edited into d:/foo/bar when it is placed in
> src/epaths.h.  The job is to remove anything from top-level
> Makefile.in that is required for file names in format other than
> /d/foo/bar, and still allow using %emacs_dir% in
> '--enable-locallisppath=PATH' option to configure.  So I think
> msys_to_w32 should stay intact, but msys_lisppath_to_w32 could be
> simplified.

A more ambitious goal, IMO, would be to understand what is the problem
with doing every conversion of MSYS paths to native w32 format with
"pwd -W", so that the pattern-matching technique could be entirely
replaced with that simpler and more elegant one.

-- 
Dani Moncayo

Re: Building Emacs from a new MinGW environment

[Eli Zaretskii]

> Date: Thu, 19 Sep 2013 12:04:06 +0200
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
> > In a nutshell, srcdir is now guaranteed to be in /d/foo/bar format,
> > and should be edited into d:/foo/bar when it is placed in
> > src/epaths.h.  The job is to remove anything from top-level
> > Makefile.in that is required for file names in format other than
> > /d/foo/bar, and still allow using %emacs_dir% in
> > '--enable-locallisppath=PATH' option to configure.  So I think
> > msys_to_w32 should stay intact, but msys_lisppath_to_w32 could be
> > simplified.
> 
> A more ambitious goal, IMO, would be to understand what is the problem
> with doing every conversion of MSYS paths to native w32 format with
> "pwd -W", so that the pattern-matching technique could be entirely
> replaced with that simpler and more elegant one.

That's a possibility, but note that "pwd -W" requires that you cd into
the directory first, which might be a problem in some rare cases.

Also msys_lisppath_to_w32 works on PATH-style directory lists, not on
single directories, and some of those directories might not exist, or
include %emacs_dir%, which will cause pwd to fail.

If these difficulties can be overcome, I have nothing against using
"pwd -W".

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

[-- Attachment #1: Type: text/plain, Size: 1461 bytes --]

>> > In a nutshell, srcdir is now guaranteed to be in /d/foo/bar format,
>> > and should be edited into d:/foo/bar when it is placed in
>> > src/epaths.h.  The job is to remove anything from top-level
>> > Makefile.in that is required for file names in format other than
>> > /d/foo/bar, and still allow using %emacs_dir% in
>> > '--enable-locallisppath=PATH' option to configure.  So I think
>> > msys_to_w32 should stay intact, but msys_lisppath_to_w32 could be
>> > simplified.
>>
>> A more ambitious goal, IMO, would be to understand what is the problem
>> with doing every conversion of MSYS paths to native w32 format with
>> "pwd -W", so that the pattern-matching technique could be entirely
>> replaced with that simpler and more elegant one.
>
> That's a possibility, but note that "pwd -W" requires that you cd into
> the directory first, which might be a problem in some rare cases.
>
> Also msys_lisppath_to_w32 works on PATH-style directory lists, not on
> single directories, and some of those directories might not exist, or
> include %emacs_dir%, which will cause pwd to fail.
>
> If these difficulties can be overcome, I have nothing against using
> "pwd -W".

I'm attaching a patch that works for me, and I think fulfills the
above requirements.

It makes _all_ translations of paths to native MS-Windows format using
the "pwd -W" feature of the MSYS shell, which I think is the only
reliable way of doing such translations.

-- 
Dani Moncayo

[-- Attachment #2: msys-to-w32.diff --]
[-- Type: text/plain, Size: 8856 bytes --]

diff --git a/Makefile.in b/Makefile.in
index 984dcea..ed3aa8b 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -317,22 +317,9 @@ epaths-force: FRC
 	  -e 's;\(#.*PATH_DOC\).*$$;\1 "${etcdocdir}";') &&		\
 	${srcdir}/build-aux/move-if-change epaths.h.$$$$ src/epaths.h
 
-# Convert MSYS-style /x/foo or Windows-style x:\foo file names
-# into x:/foo that Windows can grok.
-msys_to_w32=sed -e 's,\\\\,/,g' -e 's,^/\([A-Za-z]\)/,\1:/,'
-
-# Transform directory search path and its components.  Original can
-# be MSYS or Windows style.  Set path separator to ";", directory
-# separator to "/" and transform MSYS-style "/c/" to "c:/".
-# Remove empty path components and escape semicolons.
-msys_lisppath_to_w32=sed -e 's,\\\\,/,g' 			\
-	-e 's,\(^\|[:;]\)\([A-Za-z]\):/,\1/\2/,g'		\
-	-e 's/:/;/g' -e 's,\(^\|;\)/\([A-Za-z]\)/,\1\2:/,g'	\
-	-e 's/;\+/;/g' -e 's/^;//' -e 's/;$$//' -e 's/;/\\\\;/g'
-
-# Replace "${prefix}" with '%emacs_dir%' (which expands to install
+# Replace "${w32prefix}" with '%emacs_dir%' (which expands to install
 # directory at runtime).
-msys_prefix_subst=sed -e 's!\(^\|;\)'"$${prefixpattern}"'\([;/]\|$$\)!\1%emacs_dir%\2!g'
+msys_w32prefix_subst=sed -e 's!\(^\|;\)'"$${w32prefixpattern}"'\([;/]\|$$\)!\1%emacs_dir%\2!g'
 
 # Quote Sed special characters (except backslash and newline) with
 # a double backslash.
@@ -340,22 +327,21 @@ msys_sed_sh_escape=sed -e 's/[];$$*.^[]/\\\\&/g'
 
 # The w32 build needs a slightly different editing, and it uses
 # nt/epaths.nt as the template.
+#
 # Use the value of ${locallisppath} supplied by `configure',
 # to support the --enable-locallisppath argument.
 #
-# When building with MinGW inside the MSYS tree, 'pwd' produces directories
-# relative to the root of the MSYS tree, e.g. '/home/user/foo' instead of
-# '/d/MSYS/home/user/foo'.  If such a value of srcdir is written to
-# src/epaths.h, that causes temacs to fail, because, being a MinGW
-# program that knows nothing of MSYS root substitution, it cannot find
-# the data directory.  "pwd -W" produces Windows-style 'd:/foo/bar'
-# absolute directory names, so we use it here to countermand that lossage.
+# In this case, the paths written to 'src/epaths.h' must be in native
+# MS-Windows format (e.g. 'c:/foo/bar'), because temacs is a MinGW
+# program that doesn't support MSYS-style paths (e.g. '/c/foo/bar' or
+# '/foo/bar').
 epaths-force-w32: FRC
-	@(w32srcdir=`cd "${srcdir}"; pwd -W | sed -e 's,^\([A-Za-z]\):,/\1,' | ${msys_to_w32}` ; 	\
-	  prefixpattern=`echo '${prefix}' | ${msys_to_w32} | ${msys_sed_sh_escape}` ; \
-	  locallisppath=`echo '${locallisppath}' | ${msys_lisppath_to_w32} | ${msys_prefix_subst}` ; \
+	@(w32srcdir=`${srcdir}/build-aux/msys-to-w32 "${srcdir}"`; \
+	  w32prefix=`${srcdir}/build-aux/msys-to-w32 "${prefix}" N`; \
+	  w32prefixpattern=`echo "${w32prefix}" | ${msys_sed_sh_escape}` ; \
+	  w32locallisppath=`${srcdir}/build-aux/msys-to-w32 "${locallisppath}" N ":" "\\;" | ${msys_w32prefix_subst}` ; \
 	  sed < ${srcdir}/nt/epaths.nt > epaths.h.$$$$		\
-	  -e 's;\(#.*PATH_SITELOADSEARCH\).*$$;\1 "'"$${locallisppath}"'";' \
+	  -e 's;\(#.*PATH_SITELOADSEARCH\).*$$;\1 "'"$${w32locallisppath}"'";' \
 	  -e '/^.*#/s/@VER@/${version}/g' 			\
 	  -e '/^.*#/s/@CFG@/${configuration}/g' 		\
 	  -e "/^.*#/s|@SRC@|$${w32srcdir}|g") &&		\
diff --git a/build-aux/msys-to-w32 b/build-aux/msys-to-w32
new file mode 100644
index 0000000..f8ae2d4
--- /dev/null
+++ b/build-aux/msys-to-w32
@@ -0,0 +1,188 @@
+#!/bin/sh
+# Take a list of MSYS-compatible paths and convert them to native
+# MS-Windows format.
+# Status is zero if successful, nonzero otherwise.
+
+VERSION='2013-11-09 10:48'; # UTC
+# The definition above must lie within the first 8 lines in order
+# for the Emacs time-stamp write hook (at end) to update it.
+# If you change this file with Emacs, please let the write hook
+# do its job.  Otherwise, update this string manually.
+
+# Copyright (C) 2002-2013 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+usage="usage: $0 PATHLIST [MUSTEXIST] [SEPARATOR [SEPARATOR2]]"
+
+help="$usage
+  or:  $0 OPTION
+
+Convert MSYS-compatible paths to MS-Windows native format.
+
+PATHLIST should be a list of paths separated by SEPARATOR.  This list
+will be written to the standard output after performing the following
+transformations:
+1. Empty paths will be discarded.
+2. Backslashes will be replaced by forward slashed.
+3. Two consecutive slashes will be replaced by a single one.
+4. Each path will be translated to Windows-native format, using
+   forward slashes (e.g. 'c:/foo/bar').  This translated path will
+   never end with a slash.
+5. Ocurrences of SEPARATOR2 withing the paths will be escaped with
+   backslashes.
+6. Different paths will be separated with SEPARATOR2.
+
+If MUSTEXIST is 'Y' (or not supplied), then each path in PATHLIST must
+exists.
+
+If SEPARATOR is not supplied, PATHLIST will be regarded as a single
+path.
+
+If SEPARATOR2 is not supplied, it will take the same value as
+SEPARATOR.
+
+Options:
+  --help     display this help and exit
+  --version  output version information and exit
+
+Report bugs to <bug-gnu-emacs@gnu.org>."
+
+version=`expr "$VERSION" : '\([^ ]*\)'`
+version="msys-to-w32 $version
+Copyright (C) 2011 Free Software Foundation, Inc.
+License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
+This is free software: you are free to change and redistribute it.
+There is NO WARRANTY, to the extent permitted by law."
+
+for arg
+do
+  case $arg in
+    --help | --hel | --he | --h)
+      exec echo "$help" ;;
+    --version | --versio | --versi | --vers | --ver | --ve | --v)
+      exec echo "$version" ;;
+    --)
+      shift
+      break ;;
+    -*)
+      echo "$0: invalid option: $arg" >&2
+      exit 1 ;;
+    *)
+      break ;;
+  esac
+done
+
+{ test $# -ge 1 && test $# -le 4; } ||
+{ echo "$0: $usage" >&2; exit 1; }
+
+# Arguments
+pathlist=$1
+mustexist=${2:-Y}
+separator=$3
+separator2=${4:-${separator}}
+
+# Split pathlist into its path components
+if test -n "$separator"
+then
+    IFS=${separator} patharray=( $pathlist )
+else
+    patharray=( "$pathlist" )
+fi
+
+# Output pathlist
+pathlist2=""
+
+for p in "${patharray[@]}"
+do
+    # Skip empty paths
+    test "$p" = "" && continue
+
+    # Replace '\' with '/', '//' with '/' and remove the final slash
+    # (if any).
+    p=${p//"\\"/"/"}
+    p=${p//\/\//"/"}
+    test ${#p} -gt 1 && p=${p%"/"}
+
+    if test -d "$p"
+    then
+	# The path exists, so just translate it
+	p2=`cd "$p" && pwd -W`
+    else
+	# The path does not exists.  So, try to guess the
+	# Windows-native translation
+
+	test "${mustexist}" = "Y" &&
+	{ echo "$0: invalid path: $p" >&2; exit 1; }
+	
+	# Look for the deepest existing directory in this path,
+	# testing with partial paths by removing one component from
+	# the right side in each iteration
+	IFS=/ pcomponents=( $p )
+	pa=$p
+
+	for (( i=${#pcomponents[@]}-1 ; i>=0 ; i-- ))
+	do
+
+	    if test "${pcomponents[i]}" = ""
+	    then
+		# The path component is empty.  This can only mean
+		# that the path starts with "/" and all components
+		# have been stripped out already.  So in this case
+		# "pa" must be set to the MSYS root directory
+		pa="/"
+	    else
+		pa=${pa%"/${pcomponents[i]}"}
+	    fi
+
+	    if test -d "${pa}"
+	    then
+
+		# Translate the existing part
+		p2=`cd "${pa}" && pwd -W`
+
+		# Remainder
+		pb="${p#${pa}}"
+
+		# Concatenate both parts (existing and remainder)
+		test "${p2}" = "/" || p2="${p2}/"
+		pb=${pb#/}
+		p2="${p2}${pb}"
+
+		break
+	    fi
+
+	done
+
+	# If no existing directory was found, error out
+	test -e "${pa}" ||
+	{ echo "$0: invalid path: ${p}" >&2; exit 1; }
+    fi
+
+    # Add this translated path to the translated pathlist
+    test "${pathlist2}" = "" || pathlist2="${pathlist2}${separator2}"
+    pathlist2="${pathlist2}${p2//${separator2}/\\${separator2}}"
+
+done
+
+# Write the translated pathlist to the standard output
+printf "${pathlist2}"
+
+## Local Variables:
+## eval: (add-hook 'write-file-functions 'time-stamp)
+## time-stamp-start: "VERSION='"
+## time-stamp-format: "%:y-%02m-%02d %02H:%02M"
+## time-stamp-time-zone: "UTC"
+## time-stamp-end: "'; # UTC"
+## End:

Re: Building Emacs from a new MinGW environment

[Dani Moncayo]

[-- Attachment #1: Type: text/plain, Size: 1126 bytes --]

>>> A more ambitious goal, IMO, would be to understand what is the problem
>>> with doing every conversion of MSYS paths to native w32 format with
>>> "pwd -W", so that the pattern-matching technique could be entirely
>>> replaced with that simpler and more elegant one.
>>
>> That's a possibility, but note that "pwd -W" requires that you cd into
>> the directory first, which might be a problem in some rare cases.
>>
>> Also msys_lisppath_to_w32 works on PATH-style directory lists, not on
>> single directories, and some of those directories might not exist, or
>> include %emacs_dir%, which will cause pwd to fail.
>>
>> If these difficulties can be overcome, I have nothing against using
>> "pwd -W".
>
> I'm attaching a patch that works for me, and I think fulfills the
> above requirements.
>
> It makes _all_ translations of paths to native MS-Windows format using
> the "pwd -W" feature of the MSYS shell, which I think is the only
> reliable way of doing such translations.

I'm attaching a second version of the patch, with documentation
improvements and minor fixes to the 'msys-to-w32' script.

-- 
Dani Moncayo

[-- Attachment #2: msys-to-w32.v2.diff --]
[-- Type: text/plain, Size: 9069 bytes --]

diff --git a/Makefile.in b/Makefile.in
index 984dcea..ed3aa8b 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -317,22 +317,9 @@ epaths-force: FRC
 	  -e 's;\(#.*PATH_DOC\).*$$;\1 "${etcdocdir}";') &&		\
 	${srcdir}/build-aux/move-if-change epaths.h.$$$$ src/epaths.h
 
-# Convert MSYS-style /x/foo or Windows-style x:\foo file names
-# into x:/foo that Windows can grok.
-msys_to_w32=sed -e 's,\\\\,/,g' -e 's,^/\([A-Za-z]\)/,\1:/,'
-
-# Transform directory search path and its components.  Original can
-# be MSYS or Windows style.  Set path separator to ";", directory
-# separator to "/" and transform MSYS-style "/c/" to "c:/".
-# Remove empty path components and escape semicolons.
-msys_lisppath_to_w32=sed -e 's,\\\\,/,g' 			\
-	-e 's,\(^\|[:;]\)\([A-Za-z]\):/,\1/\2/,g'		\
-	-e 's/:/;/g' -e 's,\(^\|;\)/\([A-Za-z]\)/,\1\2:/,g'	\
-	-e 's/;\+/;/g' -e 's/^;//' -e 's/;$$//' -e 's/;/\\\\;/g'
-
-# Replace "${prefix}" with '%emacs_dir%' (which expands to install
+# Replace "${w32prefix}" with '%emacs_dir%' (which expands to install
 # directory at runtime).
-msys_prefix_subst=sed -e 's!\(^\|;\)'"$${prefixpattern}"'\([;/]\|$$\)!\1%emacs_dir%\2!g'
+msys_w32prefix_subst=sed -e 's!\(^\|;\)'"$${w32prefixpattern}"'\([;/]\|$$\)!\1%emacs_dir%\2!g'
 
 # Quote Sed special characters (except backslash and newline) with
 # a double backslash.
@@ -340,22 +327,21 @@ msys_sed_sh_escape=sed -e 's/[];$$*.^[]/\\\\&/g'
 
 # The w32 build needs a slightly different editing, and it uses
 # nt/epaths.nt as the template.
+#
 # Use the value of ${locallisppath} supplied by `configure',
 # to support the --enable-locallisppath argument.
 #
-# When building with MinGW inside the MSYS tree, 'pwd' produces directories
-# relative to the root of the MSYS tree, e.g. '/home/user/foo' instead of
-# '/d/MSYS/home/user/foo'.  If such a value of srcdir is written to
-# src/epaths.h, that causes temacs to fail, because, being a MinGW
-# program that knows nothing of MSYS root substitution, it cannot find
-# the data directory.  "pwd -W" produces Windows-style 'd:/foo/bar'
-# absolute directory names, so we use it here to countermand that lossage.
+# In this case, the paths written to 'src/epaths.h' must be in native
+# MS-Windows format (e.g. 'c:/foo/bar'), because temacs is a MinGW
+# program that doesn't support MSYS-style paths (e.g. '/c/foo/bar' or
+# '/foo/bar').
 epaths-force-w32: FRC
-	@(w32srcdir=`cd "${srcdir}"; pwd -W | sed -e 's,^\([A-Za-z]\):,/\1,' | ${msys_to_w32}` ; 	\
-	  prefixpattern=`echo '${prefix}' | ${msys_to_w32} | ${msys_sed_sh_escape}` ; \
-	  locallisppath=`echo '${locallisppath}' | ${msys_lisppath_to_w32} | ${msys_prefix_subst}` ; \
+	@(w32srcdir=`${srcdir}/build-aux/msys-to-w32 "${srcdir}"`; \
+	  w32prefix=`${srcdir}/build-aux/msys-to-w32 "${prefix}" N`; \
+	  w32prefixpattern=`echo "${w32prefix}" | ${msys_sed_sh_escape}` ; \
+	  w32locallisppath=`${srcdir}/build-aux/msys-to-w32 "${locallisppath}" N ":" "\\;" | ${msys_w32prefix_subst}` ; \
 	  sed < ${srcdir}/nt/epaths.nt > epaths.h.$$$$		\
-	  -e 's;\(#.*PATH_SITELOADSEARCH\).*$$;\1 "'"$${locallisppath}"'";' \
+	  -e 's;\(#.*PATH_SITELOADSEARCH\).*$$;\1 "'"$${w32locallisppath}"'";' \
 	  -e '/^.*#/s/@VER@/${version}/g' 			\
 	  -e '/^.*#/s/@CFG@/${configuration}/g' 		\
 	  -e "/^.*#/s|@SRC@|$${w32srcdir}|g") &&		\
diff --git a/build-aux/msys-to-w32 b/build-aux/msys-to-w32
new file mode 100644
index 0000000..d3130bf
--- /dev/null
+++ b/build-aux/msys-to-w32
@@ -0,0 +1,188 @@
+#!/bin/sh
+# Take a list of MSYS-compatible paths and convert them to native
+# MS-Windows format.
+# Status is zero if successful, nonzero otherwise.
+
+VERSION='2013-11-10 14:48'; # UTC
+# The definition above must lie within the first 8 lines in order
+# for the Emacs time-stamp write hook (at end) to update it.
+# If you change this file with Emacs, please let the write hook
+# do its job.  Otherwise, update this string manually.
+
+# Copyright (C) 2002-2013 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Take only the basename from the full pathname
+me=${0//*\//}
+
+usage="usage: ${me} PATHLIST [MUSTEXIST] [SEPARATOR [SEPARATOR2]]"
+
+help="$usage
+  or:  ${me} OPTION
+
+Convert MSYS-compatible paths to MS-Windows native format.
+
+PATHLIST should be a list of paths separated by SEPARATOR.  This list
+will be written to the standard output after performing the following
+transformations:
+1. Discard empty paths.
+2. Replace backslashes with forward slashes.
+3. Replace two consecutive slashes with single ones.
+4. Translate each path to Windows-native format.  The translated path
+   will not end with a slash, except for root directories (e.g. 'c:/').
+5. Escape with backslashes every ocurrence of SEPARATOR2 within the paths.
+6. Concatenate the translated paths with SEPARATOR2.
+
+If MUSTEXIST is 'Y' or not supplied, then each path in PATHLIST must
+exists.  Otherwise, only some part of this path is required to exist
+(that part will be translated and the remainder will be concatenated).
+
+If SEPARATOR is not supplied, PATHLIST will be regarded as a single
+path.
+
+If SEPARATOR2 is not supplied, it will take the same value as
+SEPARATOR.
+
+Options:
+  --help     display this help and exit
+  --version  output version information and exit
+
+Report bugs to <bug-gnu-emacs@gnu.org>."
+
+version=`expr "$VERSION" : '\([^ ]*\)'`
+version="msys-to-w32 $version
+Copyright (C) 2011 Free Software Foundation, Inc.
+License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
+This is free software: you are free to change and redistribute it.
+There is NO WARRANTY, to the extent permitted by law."
+
+for arg
+do
+  case $arg in
+    --help | --hel | --he | --h)
+      exec echo "$help" ;;
+    --version | --versio | --versi | --vers | --ver | --ve | --v)
+      exec echo "$version" ;;
+    --)
+      shift
+      break ;;
+    -*)
+      echo "${me}: invalid option: $arg" >&2
+      exit 1 ;;
+    *)
+      break ;;
+  esac
+done
+
+{ test $# -ge 1 && test $# -le 4; } ||
+{ echo "${me}: $usage" >&2; exit 1; }
+
+# Arguments
+pathlist="$1"
+mustexist="${2:-Y}"
+separator="$3"
+separator2="${4:-${separator}}"
+
+# Split pathlist into its path components
+if test -n "$separator"
+then
+    IFS=${separator} patharray=( $pathlist )
+else
+    patharray=( "$pathlist" )
+fi
+
+# Output pathlist
+pathlist2=""
+
+for p in "${patharray[@]}"
+do
+    # Skip empty paths
+    test "$p" = "" && continue
+
+    # Replace '\' with '/' and '//' with '/'
+    p="${p//\\//}"
+    p="${p//\/\///}"
+
+    if test -d "$p"
+    then
+	# The path exists, so just translate it
+	w32p=`cd "$p" && pwd -W`
+    else
+	# The path does not exists.  So, try to guess the
+	# Windows-native translation, by looking for the deepest
+	# existing directory in this path, and then translating the
+	# existing part and concatenating the remainder.
+
+	test "${mustexist}" = "Y" &&
+	{ echo "${me}: invalid path: $p" >&2; exit 1; }
+
+	p1=$p
+	IFS=/ pcomponents=( $p )
+
+	for (( i=${#pcomponents[@]}-1 ; i>=0 ; i-- ))
+	do
+
+	    if test "${pcomponents[i]}" = ""
+	    then
+		# The path component is empty.  This can only mean
+		# that the path starts with "/" and all components
+		# have been stripped out already.  So in this case we
+		# want to test with the MSYS root directory
+		p1="/"
+	    else
+		p1="${p1%/}"
+		p1="${p1%${pcomponents[i]}}"
+	    fi
+
+	    if test -d "${p1}"
+	    then
+
+		# Existing path found
+
+		# Translate the existing part and concatenate the
+		# remainder (ensuring that only one slash is used in
+		# the join, and no trailing slash is left)
+		w32p1=`cd "${p1}" && pwd -W`
+		remainder="${p#${p1}}"
+		remainder="${remainder#/}"
+		remainder="${remainder%/}"
+		w32p="${w32p1%/}/${remainder}"
+
+		break
+	    fi
+
+	done
+
+	# If no existing directory was found, error out
+	test -e "${p1}" ||
+	{ echo "${me}: invalid path: ${p}" >&2; exit 1; }
+    fi
+
+    # Concatenate the translated path to the translated pathlist
+    test "${pathlist2}" = "" || pathlist2="${pathlist2}${separator2}"
+    pathlist2="${pathlist2}${w32p//${separator2}/\\${separator2}}"
+
+done
+
+# Write the translated pathlist to the standard output
+printf "${pathlist2}"
+
+## Local Variables:
+## eval: (add-hook 'write-file-functions 'time-stamp)
+## time-stamp-start: "VERSION='"
+## time-stamp-format: "%:y-%02m-%02d %02H:%02M"
+## time-stamp-time-zone: "UTC"
+## time-stamp-end: "'; # UTC"
+## End:

Re: Building Emacs from a new MinGW environment

[Glenn Morris]

I can't comment on any of the actual details, since I can't test it, but
here are some pedantic nitpicks! :)

> +VERSION='2013-11-10 14:48'; # UTC

Things like that are generally a nuisance with VCS (merge conflicts).
I don't think it serves any useful purpose in this case.

I guess you have this because you copied the template of move-if-change.
That has more of an excuse for including such a thing; since, as part of
gnulib, it will be included in many projects that have varying version
numbers of their own.

But for yours, the version will just be that of the Emacs that it came
with.

> +# Copyright (C) 2002-2013 Free Software Foundation, Inc.

Probably should just be 2013.

> +Copyright (C) 2011 Free Software Foundation, Inc.

Probably should just be 2013.

> +## Local Variables:
> +## eval: (add-hook 'write-file-functions 'time-stamp)
> +## time-stamp-start: "VERSION='"
> +## time-stamp-format: "%:y-%02m-%02d %02H:%02M"
> +## time-stamp-time-zone: "UTC"
> +## time-stamp-end: "'; # UTC"
> +## End:

Not needed IMO.