Build failure for Emacs master

Build failure for Emacs master

[Angelo Graziosi]

Just for the record, master fail to build on MSYS2/MinGW64:
[...]
Loading faces...
Loading button...
Loading loaddefs.el (source)...
End of file during parsing: 
c:/msys64/tmp/mingw-w64-emacs-git/src/emacs/lisp/loaddefs.el
Makefile:540: set di istruzioni per l'obiettivo "emacs.exe" non riuscito
make[1]: *** [emacs.exe] Errore 127
make[1]: uscita dalla directory 
"/tmp/mingw-w64-emacs-git/src/build-x86_64-w64-mingw32/src"
Makefile:398: set di istruzioni per l'obiettivo "src" non riuscito
make: *** [src] Errore 2

I did a successful build after last merge (commit Merge from 
origin/emacs-25 81ef756e6aea369ec78f19b3609f01ceddc5851f), maybe some 
recent change should be still tuned..

   Angelo

Re: Build failure for Emacs master

[Paul Eggert]

On 02/23/2016 02:11 PM, Angelo Graziosi wrote:
> Loading loaddefs.el (source)...
> End of file during parsing: 
> c:/msys64/tmp/mingw-w64-emacs-git/src/emacs/lisp/loaddefs.el
> Makefile:540: set di istruzioni per l'obiettivo "emacs.exe" non riuscito
> make[1]: *** [emacs.exe] Errore 127 

I could be the guilty party here, since I installed many of the changes 
after the last commit that works for you. However, I don't see how those 
changes could cause an MS-WIndows build to have an instruction error. 
Perhaps a 'make bootstrap' is in order? If that doesn't work, what's the 
difference between lisp/loaddefs.el in your last good commit versus the 
latest commit?

Re: Build failure for Emacs master

[Angelo Graziosi]

Il 24/02/2016 00:14, Paul Eggert ha scritto:
> On 02/23/2016 02:11 PM, Angelo Graziosi wrote:
>> Loading loaddefs.el (source)...
>> End of file during parsing:
>> c:/msys64/tmp/mingw-w64-emacs-git/src/emacs/lisp/loaddefs.el
>> Makefile:540: set di istruzioni per l'obiettivo "emacs.exe" non riuscito
>> make[1]: *** [emacs.exe] Errore 127
>
> I could be the guilty party here, since I installed many of the changes
> after the last commit that works for you. However, I don't see how those
> changes could cause an MS-WIndows build to have an instruction error.
> Perhaps a 'make bootstrap' is in order? If that doesn't work, what's the


I do always build in a clean tree, so I think bootstrap is always 
called. I tried also removing the git tree and recreating it.

Now I am trying a new build and it seems work better..

I will flag new issues if I will find them..

For now.. goodnight and sorry for the noise..

Thanks,
  Angelo.

Re: Build failure for Emacs master

[Eli Zaretskii]

> From: Angelo Graziosi <angelo.graziosi@alice.it>
> Date: Tue, 23 Feb 2016 23:11:42 +0100
> 
> Just for the record, master fail to build on MSYS2/MinGW64:

It doesn't fail for me.

> Loading faces...
> Loading button...
> Loading loaddefs.el (source)...
> End of file during parsing: 
> c:/msys64/tmp/mingw-w64-emacs-git/src/emacs/lisp/loaddefs.el

Please look at your loaddefs.el, and tell if you see something strange
there, like null bytes.

Re: Build failure for Emacs master

[Angelo Graziosi]

Il 24/02/2016 04:42, Eli Zaretskii ha scritto:
>> From: Angelo Graziosi <angelo.graziosi@alice.it>
>> Date: Tue, 23 Feb 2016 23:11:42 +0100
>>
>> Just for the record, master fail to build on MSYS2/MinGW64:
>
> It doesn't fail for me.
>
>> Loading faces...
>> Loading button...
>> Loading loaddefs.el (source)...
>> End of file during parsing:
>> c:/msys64/tmp/mingw-w64-emacs-git/src/emacs/lisp/loaddefs.el
>
> Please look at your loaddefs.el, and tell if you see something strange
> there, like null bytes.
>

Hmm.. I removed the Emacs tree and recreated from scratch, then retried 
the build.. Now it fails on another loaddefs, org-loaddefs.el. I have 
seen the same failure a week ago or so and just retrying worked...

I wonder if this could be related to the "make -j3" (two core) or some 
other thing in MSYS2..

Let's see what happen with another try..


   Angelo

Re: Build failure for Emacs master

[Angelo Graziosi]

Il 24/02/2016 10:36, Angelo Graziosi ha scritto:
>
> Let's see what happen with another try..
>

Indeed, just retrying builds OB..

>
>    Angelo

Re: Build failure for Emacs master

[Eli Zaretskii]

> From: Angelo Graziosi <angelo.graziosi@alice.it>
> Date: Wed, 24 Feb 2016 11:20:56 +0100
> Cc: emacs-devel@gnu.org
> 
> Il 24/02/2016 10:36, Angelo Graziosi ha scritto:
> >
> > Let's see what happen with another try..
> >
> 
> Indeed, just retrying builds OB..

Alas, this tells us nothing about the reason for the original problem.

Re: Build failure for Emacs master

[Angelo Graziosi]

Il 24/02/2016 18:37, Eli Zaretskii ha scritto:
>> From: Angelo Graziosi <angelo.graziosi@alice.it>
>> Date: Wed, 24 Feb 2016 11:20:56 +0100
>> Cc: emacs-devel@gnu.org
>>
>> Il 24/02/2016 10:36, Angelo Graziosi ha scritto:
>>>
>>> Let's see what happen with another try..
>>>
>>
>> Indeed, just retrying builds OB..
>
> Alas, this tells us nothing about the reason for the original problem.
>

Another try, another failure:
[...]
Loading loaddefs.el (source)...
End of file during parsing: 
c:/msys64/tmp/mingw-w64-emacs-git/src/emacs/lisp/loaddefs.el
Makefile:540: set di istruzioni per l'obiettivo "emacs.exe" non riuscito
make[1]: *** [emacs.exe] Errore 127
make[1]: uscita dalla directory 
"/tmp/mingw-w64-emacs-git/src/build-x86_64-w64-mingw32/src"
Makefile:398: set di istruzioni per l'obiettivo "src" non riuscito
make: *** [src] Errore 2

with master-ce6a03cb00fb95b3e32440c1388d5f40375aadb2 (Fix problem in 
tramp.texi).

Really c:/msys64/tmp/mingw-w64-emacs-git/src/emacs/lisp/loaddefs.el 
seems broken

cat /tmp/mingw-w64-emacs-git/src/emacs/lisp/loaddefs.el
[...]
(autoload 'woman "woman" "\
Browse UN*X man page for TOPIC (Without using external Man program).
The major browsing mode used is essentially the standard Man mode.
Choose the filename for the man page using completion, based on the
topic selected from the directories specified in `woman-manpath' and
`woman-path'.  The directory expa

   A LOT OF "APPARENTLY" EMPTY SPACE


but when I visit it with Emacs, 'A LOT OF "APPARENTLY" EMPTY SPACE' is a 
lot of "^@" characters.

BTW, insisting to visit it, Emacs hangs (I have to kill it with task 
manager...)

If you want I can sent it to someone of you, off list (xz compressed is 
about 250K).


   Angelo

Re: Build failure for Emacs master

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Angelo Graziosi <angelo.graziosi@alice.it>
> Date: Wed, 24 Feb 2016 23:20:23 +0100
> 
> cat /tmp/mingw-w64-emacs-git/src/emacs/lisp/loaddefs.el
> [...]
> (autoload 'woman "woman" "\
> Browse UN*X man page for TOPIC (Without using external Man program).
> The major browsing mode used is essentially the standard Man mode.
> Choose the filename for the man page using completion, based on the
> topic selected from the directories specified in `woman-manpath' and
> `woman-path'.  The directory expa
> 
>    A LOT OF "APPARENTLY" EMPTY SPACE
> 
> 
> but when I visit it with Emacs, 'A LOT OF "APPARENTLY" EMPTY SPACE' is a 
> lot of "^@" characters.

Yes, that's what happens in these cases.  So the problem is still
present, sigh.  Unfortunately, it stopped happening on my system, so I
cannot debug it.

To solve this simply copy ldefs-boot.el over loaddefs.el, and rebuild.

Re: Build failure for Emacs master

[Angelo Graziosi]

Just for the record..

Now the build of current master (ac9a931...) fails with:

[...]
Loading button...
Loading loaddefs.el (source)...
Wrong number of arguments: autoload, 1325
Makefile:540: set di istruzioni per l'obiettivo "emacs.exe" non riuscito
make[1]: *** [emacs.exe] Errore 127
make[1]: uscita dalla directory 
"/tmp/mingw-w64-emacs-git/src/build-x86_64-w64-mingw32/src"
Makefile:398: set di istruzioni per l'obiettivo "src" non riuscito
make: *** [src] Errore 2


All the loaddefs.el files are:

$ find . -name "loaddefs*"
./src/emacs/lisp/cedet/ede/loaddefs.el
./src/emacs/lisp/cedet/ede/loaddefs.el~
./src/emacs/lisp/cedet/semantic/loaddefs.el
./src/emacs/lisp/cedet/semantic/loaddefs.el~
./src/emacs/lisp/cedet/srecode/loaddefs.el
./src/emacs/lisp/cedet/srecode/loaddefs.el~
./src/emacs/lisp/loaddefs.el
./src/emacs/lisp/loaddefs.el~

and

$ cat ./src/emacs/lisp/loaddefs.el
[...]
(autoload 'xref-collect-matches "xref" "\
Collect matches for REGEXP inside FILES in DIR.
FILES is a string with glob patterns separated by spaces.
IGNORES is a list of glob patterns.

\(fn REGEXP FILES DIR IGNORES)" nil nil)

;;;***


while for the others

$ cat ./src/emacs/lisp/.../loaddefs.el
[...]
;;;***


(provide 'loaddefs)
;; Local Variables:
;; version-control: never
;; no-byte-compile: t
;; no-update-autoloads: t
;; coding: utf-8
;; End:
;;; loaddefs.el ends here


Maybe just retrying builds..

In any cases this kind of failures are rather recent. I have built 
master on MSYS2 for months without any failure and since, say, first 
decade of February they occur..


   Angelo


Il 25/02/2016 17:43, Eli Zaretskii ha scritto:
>> Cc: emacs-devel@gnu.org
>> From: Angelo Graziosi <angelo.graziosi@alice.it>
>> Date: Wed, 24 Feb 2016 23:20:23 +0100
>>
>> cat /tmp/mingw-w64-emacs-git/src/emacs/lisp/loaddefs.el
>> [...]
>> (autoload 'woman "woman" "\
>> Browse UN*X man page for TOPIC (Without using external Man program).
>> The major browsing mode used is essentially the standard Man mode.
>> Choose the filename for the man page using completion, based on the
>> topic selected from the directories specified in `woman-manpath' and
>> `woman-path'.  The directory expa
>>
>>     A LOT OF "APPARENTLY" EMPTY SPACE
>>
>>
>> but when I visit it with Emacs, 'A LOT OF "APPARENTLY" EMPTY SPACE' is a
>> lot of "^@" characters.
>
> Yes, that's what happens in these cases.  So the problem is still
> present, sigh.  Unfortunately, it stopped happening on my system, so I
> cannot debug it.
>
> To solve this simply copy ldefs-boot.el over loaddefs.el, and rebuild.
>

Re: Build failure for Emacs master

[Eli Zaretskii]

> From: Angelo Graziosi <angelo.graziosi@alice.it>
> Date: Fri, 4 Mar 2016 22:50:31 +0100
> 
> $ cat ./src/emacs/lisp/loaddefs.el
> [...]
> (autoload 'xref-collect-matches "xref" "\
> Collect matches for REGEXP inside FILES in DIR.
> FILES is a string with glob patterns separated by spaces.
> IGNORES is a list of glob patterns.
> 
> \(fn REGEXP FILES DIR IGNORES)" nil nil)
> 
> ;;;***
> 
> 
> while for the others
> 
> $ cat ./src/emacs/lisp/.../loaddefs.el
> [...]
> ;;;***
> 
> 
> (provide 'loaddefs)
> ;; Local Variables:
> ;; version-control: never
> ;; no-byte-compile: t
> ;; no-update-autoloads: t
> ;; coding: utf-8
> ;; End:
> ;;; loaddefs.el ends here
> 
> 
> Maybe just retrying builds..

Yes, this looks like the same problem.

The challenge is to catch the instance when such a faulty loaddefs.el
is produced, and see what happens there.  Ideas for how to do that are
welcome.

> In any cases this kind of failures are rather recent. I have built 
> master on MSYS2 for months without any failure and since, say, first 
> decade of February they occur..

I've seen this first in last November.  Not sure if it's the same
problem, but the symptoms are very similar.

Are all of your builds full bootstraps in a fresh directory using a
freshly cloned repository?

Re: Build failure for Emacs master

[Angelo Graziosi]

Il 05/03/2016 08:25, Eli Zaretskii ha scritto:
>
> Are all of your builds full bootstraps in a fresh directory using a
> freshly cloned repository?
>

Usually I try to build the package as described here:

   https://sourceforge.net/p/msys2/wiki/Contributing%20to%20MSYS2

In short,

   MINGW_INSTALLS=mingw64 makepkg-mingw -sLf

(I changed the PKGBUILD [*] script to use 'make -j3').

The folder in which I run the above command contains an "emacs" 
directory containing the git tree of Emacs master. If I understand, this 
tree is copied at each build in a 'working' build tree inside a 'src' 
folder and I delete the 'src' tree after each build..

Just for completeness, I will try leaving only the PKGBUILD script and 
deleting all other things..

  Angelo

---
[*] 
https://github.com/Alexpux/MINGW-packages/blob/master/mingw-w64-emacs-git/PKGBUILD

Re: Build failure for Emacs master

[Angelo Graziosi]

Il 05/03/2016 08:25, Eli Zaretskii ha scritto:
>> From: Angelo Graziosi <angelo.graziosi@alice.it>
>> Date: Fri, 4 Mar 2016 22:50:31 +0100
>>
>> $ cat ./src/emacs/lisp/loaddefs.el
>> [...]
>> (autoload 'xref-collect-matches "xref" "\
>> Collect matches for REGEXP inside FILES in DIR.
>> FILES is a string with glob patterns separated by spaces.
>> IGNORES is a list of glob patterns.
>>
>> \(fn REGEXP FILES DIR IGNORES)" nil nil)
>>
>> ;;;***
>>
>>
>> while for the others
>>
>> $ cat ./src/emacs/lisp/.../loaddefs.el
>> [...]
>> ;;;***
>>
>>
>> (provide 'loaddefs)
>> ;; Local Variables:
>> ;; version-control: never
>> ;; no-byte-compile: t
>> ;; no-update-autoloads: t
>> ;; coding: utf-8
>> ;; End:
>> ;;; loaddefs.el ends here
>>
>>
>> Maybe just retrying builds..
>
> Yes, this looks like the same problem.
>
> The challenge is to catch the instance when such a faulty loaddefs.el
> is produced, and see what happens there.  Ideas for how to do that are
> welcome.
>
>> In any cases this kind of failures are rather recent. I have built
>> master on MSYS2 for months without any failure and since, say, first
>> decade of February they occur..
>
> I've seen this first in last November.  Not sure if it's the same
> problem, but the symptoms are very similar.
>
> Are all of your builds full bootstraps in a fresh directory using a
> freshly cloned repository?
>

Now I have tried current master ( 
21b509d4449bd33045e019dbcc90f5283434c07e) in a freshly cloned repository 
but it fails differently (I think it is unrelated to our loaddefs issue):

[...]
   CCLD     temacs.exe
process.o:process.c:(.text+0x93cb): undefined reference to `gai_strerrorA'
process.o:process.c:(.text+0x93cb): relocation truncated to fit: 
R_X86_64_PC32 gainst undefined symbol `gai_strerrorA'
collect2.exe: error: ld returned 1 exit status
Makefile:601: set di istruzioni per l'obiettivo "temacs.exe" non riuscito
make[1]: *** [temacs.exe] Errore 1
make[1]: uscita dalla directory 
"/tmp/mingw-w64-emacs-git/src/build-x86_64-w64-ingw32/src"
Makefile:398: set di istruzioni per l'obiettivo "src" non riuscito
make: *** [src] Errore 2

I wonder if it is related to this change:

Lars Magne Ingebrigtsen Allow making TLS negotiation blocking
commit: 
http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=21b509d4449bd33045e019dbcc90f5283434c07e

which changes process.h..


   Angelo

Re: Build failure for Emacs master

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org, larsi@gnus.org
> From: Angelo Graziosi <angelo.graziosi@alice.it>
> Date: Sat, 5 Mar 2016 21:59:49 +0100
> 
> Now I have tried current master ( 
> 21b509d4449bd33045e019dbcc90f5283434c07e) in a freshly cloned repository 
> but it fails differently (I think it is unrelated to our loaddefs issue):
> 
> [...]
>    CCLD     temacs.exe
> process.o:process.c:(.text+0x93cb): undefined reference to `gai_strerrorA'
> process.o:process.c:(.text+0x93cb): relocation truncated to fit: 
> R_X86_64_PC32 gainst undefined symbol `gai_strerrorA'
> collect2.exe: error: ld returned 1 exit status
> Makefile:601: set di istruzioni per l'obiettivo "temacs.exe" non riuscito
> make[1]: *** [temacs.exe] Errore 1
> make[1]: uscita dalla directory 
> "/tmp/mingw-w64-emacs-git/src/build-x86_64-w64-ingw32/src"
> Makefile:398: set di istruzioni per l'obiettivo "src" non riuscito
> make: *** [src] Errore 2
> 
> I wonder if it is related to this change:
> 
> Lars Magne Ingebrigtsen Allow making TLS negotiation blocking
> commit: 
> http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=21b509d4449bd33045e019dbcc90f5283434c07e
> 
> which changes process.h..

No, it's related to my recent changes.  Sounds like MinGW64 headers
have yet another incompatibility with mingw.org's headers, sigh...

Re: Build failure for Emacs master

[Eli Zaretskii]

> Date: Sun, 06 Mar 2016 05:35:44 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: larsi@gnus.org, emacs-devel@gnu.org
> 
> >    CCLD     temacs.exe
> > process.o:process.c:(.text+0x93cb): undefined reference to `gai_strerrorA'
> > process.o:process.c:(.text+0x93cb): relocation truncated to fit: 
> > R_X86_64_PC32 gainst undefined symbol `gai_strerrorA'
> > collect2.exe: error: ld returned 1 exit status
> > Makefile:601: set di istruzioni per l'obiettivo "temacs.exe" non riuscito
> > make[1]: *** [temacs.exe] Errore 1
> > make[1]: uscita dalla directory 
> > "/tmp/mingw-w64-emacs-git/src/build-x86_64-w64-ingw32/src"
> > Makefile:398: set di istruzioni per l'obiettivo "src" non riuscito
> > make: *** [src] Errore 2
> > 
> > I wonder if it is related to this change:
> > 
> > Lars Magne Ingebrigtsen Allow making TLS negotiation blocking
> > commit: 
> > http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=21b509d4449bd33045e019dbcc90f5283434c07e
> > 
> > which changes process.h..
> 
> No, it's related to my recent changes.  Sounds like MinGW64 headers
> have yet another incompatibility with mingw.org's headers, sigh...

I attempted to fix this in commit 69e03dd, please take another look.

Re: Build failure for Emacs master

[Angelo Graziosi]

Il 06/03/2016 17:55, Eli Zaretskii ha scritto:
>
> I attempted to fix this in commit 69e03dd, please take another look.
>

Yes, the fix works..

I have also completed the build in a freshly cloned repository without 
the loaddefs issue... let's see if it will return in the next builds.

Thanks,
  Angelo.

Re: Build failure for Emacs master

[Andy Moreton]

On Sun 06 Mar 2016, Eli Zaretskii wrote:

>> Cc: emacs-devel@gnu.org, larsi@gnus.org
>> From: Angelo Graziosi <angelo.graziosi@alice.it>
>> Date: Sat, 5 Mar 2016 21:59:49 +0100
>> 
>> Now I have tried current master ( 
>> 21b509d4449bd33045e019dbcc90f5283434c07e) in a freshly cloned repository 
>> but it fails differently (I think it is unrelated to our loaddefs issue):
>> 
>> [...]
>>    CCLD     temacs.exe
>> process.o:process.c:(.text+0x93cb): undefined reference to `gai_strerrorA'
>> process.o:process.c:(.text+0x93cb): relocation truncated to fit: 
>> R_X86_64_PC32 gainst undefined symbol `gai_strerrorA'
>> collect2.exe: error: ld returned 1 exit status
>> Makefile:601: set di istruzioni per l'obiettivo "temacs.exe" non riuscito
>> make[1]: *** [temacs.exe] Errore 1
>> make[1]: uscita dalla directory 
>> "/tmp/mingw-w64-emacs-git/src/build-x86_64-w64-ingw32/src"
>> Makefile:398: set di istruzioni per l'obiettivo "src" non riuscito
>> make: *** [src] Errore 2
>> 
>> I wonder if it is related to this change:
>> 
>> Lars Magne Ingebrigtsen Allow making TLS negotiation blocking
>> commit: 
>> http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=21b509d4449bd33045e019dbcc90f5283434c07e
>> 
>> which changes process.h..
>
> No, it's related to my recent changes.  Sounds like MinGW64 headers
> have yet another incompatibility with mingw.org's headers, sigh...

Your fix for this in 69e03dd seems to be overly harsh on Mingw64.

Win95/98/Me have been dead for over 15 years - it's time to kill support
for them, and require that all Windows platforms support Winsock2.

    AndyM

Re: Build failure for Emacs master

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Sun, 06 Mar 2016 17:37:37 +0000
> 
> Your fix for this in 69e03dd seems to be overly harsh on Mingw64.

I didn't find a better way, sorry; ideas are welcome.  And the lossage
is not really so bad: after all, we've lived without gai_strerror
until now, and no one complained.  It's a function that displays
messages for errors that aren't supposed to happen, right?

> Win95/98/Me have been dead for over 15 years - it's time to kill support
> for them, and require that all Windows platforms support Winsock2.

Windows 9X support winsock2 as well, that's not the issue here.  The
issue is that these 3 functions were added on Windows XP; they were
missing before that.

But if you are saying that the 64-build should be linked with
"-lws2_32", I'm okay with accepting patches to that effect only for
that build.

Re: Build failure for Emacs master

[Angelo Graziosi]

Just for the record...

After about a month, the issue has reappeared.

Now it fails with this message:

[...]
Loading button...
Loading loaddefs.el (source)...
Wrong number of arguments: autoload, 1325
Makefile:540: set di istruzioni per l'obiettivo "emacs.exe" non riuscito
make[1]: *** [emacs.exe] Errore 127
make[1]: uscita dalla directory "/tmp/work/emacs-master/src"
Makefile:398: set di istruzioni per l'obiettivo "src" non riuscito
make: *** [src] Errore 2

but the lisp/loaddefs.el seems to have the same defects..

Ciao,
  Angelo.

Il 05/03/2016 08:25, Eli Zaretskii ha scritto:
>> From: Angelo Graziosi <angelo.graziosi@alice.it>
>> Date: Fri, 4 Mar 2016 22:50:31 +0100
>>
>> $ cat ./src/emacs/lisp/loaddefs.el
>> [...]
>> (autoload 'xref-collect-matches "xref" "\
>> Collect matches for REGEXP inside FILES in DIR.
>> FILES is a string with glob patterns separated by spaces.
>> IGNORES is a list of glob patterns.
>>
>> \(fn REGEXP FILES DIR IGNORES)" nil nil)
>>
>> ;;;***
>>
>>
>> while for the others
>>
>> $ cat ./src/emacs/lisp/.../loaddefs.el
>> [...]
>> ;;;***
>>
>>
>> (provide 'loaddefs)
>> ;; Local Variables:
>> ;; version-control: never
>> ;; no-byte-compile: t
>> ;; no-update-autoloads: t
>> ;; coding: utf-8
>> ;; End:
>> ;;; loaddefs.el ends here
>>
>>
>> Maybe just retrying builds..
>
> Yes, this looks like the same problem.
>
> The challenge is to catch the instance when such a faulty loaddefs.el
> is produced, and see what happens there.  Ideas for how to do that are
> welcome.
>
>> In any cases this kind of failures are rather recent. I have built
>> master on MSYS2 for months without any failure and since, say, first
>> decade of February they occur..
>
> I've seen this first in last November.  Not sure if it's the same
> problem, but the symptoms are very similar.
>
> Are all of your builds full bootstraps in a fresh directory using a
> freshly cloned repository?
>

Re: Build failure for Emacs master

[Phillip Lord]

Reappeared randomly, or reappeared consistently on every build?

Phil

Angelo Graziosi <angelo.graziosi@alice.it> writes:

> Just for the record...
>
> After about a month, the issue has reappeared.
>
> Now it fails with this message:
>
> [...]
> Loading button...
> Loading loaddefs.el (source)...
> Wrong number of arguments: autoload, 1325
> Makefile:540: set di istruzioni per l'obiettivo "emacs.exe" non riuscito
> make[1]: *** [emacs.exe] Errore 127
> make[1]: uscita dalla directory "/tmp/work/emacs-master/src"
> Makefile:398: set di istruzioni per l'obiettivo "src" non riuscito
> make: *** [src] Errore 2
>
> but the lisp/loaddefs.el seems to have the same defects..
>
> Ciao,
>  Angelo.
>
> Il 05/03/2016 08:25, Eli Zaretskii ha scritto:
>>> From: Angelo Graziosi <angelo.graziosi@alice.it>
>>> Date: Fri, 4 Mar 2016 22:50:31 +0100
>>>
>>> $ cat ./src/emacs/lisp/loaddefs.el
>>> [...]
>>> (autoload 'xref-collect-matches "xref" "\
>>> Collect matches for REGEXP inside FILES in DIR.
>>> FILES is a string with glob patterns separated by spaces.
>>> IGNORES is a list of glob patterns.
>>>
>>> \(fn REGEXP FILES DIR IGNORES)" nil nil)
>>>
>>> ;;;***
>>>
>>>
>>> while for the others
>>>
>>> $ cat ./src/emacs/lisp/.../loaddefs.el
>>> [...]
>>> ;;;***
>>>
>>>
>>> (provide 'loaddefs)
>>> ;; Local Variables:
>>> ;; version-control: never
>>> ;; no-byte-compile: t
>>> ;; no-update-autoloads: t
>>> ;; coding: utf-8
>>> ;; End:
>>> ;;; loaddefs.el ends here
>>>
>>>
>>> Maybe just retrying builds..
>>
>> Yes, this looks like the same problem.
>>
>> The challenge is to catch the instance when such a faulty loaddefs.el
>> is produced, and see what happens there.  Ideas for how to do that are
>> welcome.
>>
>>> In any cases this kind of failures are rather recent. I have built
>>> master on MSYS2 for months without any failure and since, say, first
>>> decade of February they occur..
>>
>> I've seen this first in last November.  Not sure if it's the same
>> problem, but the symptoms are very similar.
>>
>> Are all of your builds full bootstraps in a fresh directory using a
>> freshly cloned repository?
>>

Re: Build failure for Emacs master

[Angelo Graziosi]

Il 12/04/2016 13:36, Phillip Lord ha scritto:
>
> Reappeared randomly, or reappeared consistently on every build?

I would say "periodically"..

I had this issue on March 5 but then I was able to build OB on March 12, 
23, 24, 29 and April 03. Now it reappeared... sometimes it fails with 
lisp/loaddefs.el and sometimes with other loaddefs. For example this 
evening:

[...]
In toplevel form:
../../emacs/lisp/gnus/gnus-icalendar.el:359:1:Error: End of file during 
parsing: 
c:/msys64/tmp/mingw-w64-emacs-git/src/emacs/lisp/org/org-loaddefs.el
Makefile:282: set di istruzioni per l'obiettivo 
"gnus/gnus-icalendar.elc" non riuscito
make[2]: *** [gnus/gnus-icalendar.elc] Errore 1
[...]

..and org/org-loaddefs.el seems to suffers the same issues of 
lisp/loaddefs.el I already flagged..

BTW, I notice that current build logs have a lot of "garbage" like this:

[...]
C:/msys64/tmp/mingw-w64-emacs-git/src/emacs/lib-src/ntlib.c:110:15: 
warning: format '%d' expects argument of type 'int', but argument 2 has 
type 'DWORD {aka long unsigned int}' [-Wformat=]

        printf ("Checking parent status failed: %d\n", GetLastError ());
[...]


and Emacs also shows a lot of ^M characters..

I think this garbage is related to the configure messages:

[...]
checking whether C compiler handles -W... yes
checking whether C compiler handles -Wabi... yes
checking whether C compiler handles -Waddress... yes
checking whether C compiler handles -Waggressive-loop-optimizations... yes
checking whether C compiler handles -Wall... yes
checking whether C compiler handles -Wattributes... yes
checking whether C compiler handles -Wbool-compare... yes
checking whether C compiler handles -Wbuiltin-macro-redefined... yes
[...]

Why enabling this by default? All this should be OFF by default and only 
the maintainers which need it should enable it

   Angelo


>
> Phil
>
> Angelo Graziosi <angelo.graziosi@alice.it> writes:
>
>> Just for the record...
>>
>> After about a month, the issue has reappeared.
>>
>> Now it fails with this message:
>>
>> [...]
>> Loading button...
>> Loading loaddefs.el (source)...
>> Wrong number of arguments: autoload, 1325
>> Makefile:540: set di istruzioni per l'obiettivo "emacs.exe" non riuscito
>> make[1]: *** [emacs.exe] Errore 127
>> make[1]: uscita dalla directory "/tmp/work/emacs-master/src"
>> Makefile:398: set di istruzioni per l'obiettivo "src" non riuscito
>> make: *** [src] Errore 2
>>
>> but the lisp/loaddefs.el seems to have the same defects..
>>
>> Ciao,
>>   Angelo.
>>
>> Il 05/03/2016 08:25, Eli Zaretskii ha scritto:
>>>> From: Angelo Graziosi <angelo.graziosi@alice.it>
>>>> Date: Fri, 4 Mar 2016 22:50:31 +0100
>>>>
>>>> $ cat ./src/emacs/lisp/loaddefs.el
>>>> [...]
>>>> (autoload 'xref-collect-matches "xref" "\
>>>> Collect matches for REGEXP inside FILES in DIR.
>>>> FILES is a string with glob patterns separated by spaces.
>>>> IGNORES is a list of glob patterns.
>>>>
>>>> \(fn REGEXP FILES DIR IGNORES)" nil nil)
>>>>
>>>> ;;;***
>>>>
>>>>
>>>> while for the others
>>>>
>>>> $ cat ./src/emacs/lisp/.../loaddefs.el
>>>> [...]
>>>> ;;;***
>>>>
>>>>
>>>> (provide 'loaddefs)
>>>> ;; Local Variables:
>>>> ;; version-control: never
>>>> ;; no-byte-compile: t
>>>> ;; no-update-autoloads: t
>>>> ;; coding: utf-8
>>>> ;; End:
>>>> ;;; loaddefs.el ends here
>>>>
>>>>
>>>> Maybe just retrying builds..
>>>
>>> Yes, this looks like the same problem.
>>>
>>> The challenge is to catch the instance when such a faulty loaddefs.el
>>> is produced, and see what happens there.  Ideas for how to do that are
>>> welcome.
>>>
>>>> In any cases this kind of failures are rather recent. I have built
>>>> master on MSYS2 for months without any failure and since, say, first
>>>> decade of February they occur..
>>>
>>> I've seen this first in last November.  Not sure if it's the same
>>> problem, but the symptoms are very similar.
>>>
>>> Are all of your builds full bootstraps in a fresh directory using a
>>> freshly cloned repository?
>>>

Re: MS-Windows warnings (was build failure) for Emacs master

[Paul Eggert]

On 04/12/2016 01:54 PM, Angelo Graziosi wrote:

> current build logs have a lot of "garbage" like this:
>
> [...]
> C:/msys64/tmp/mingw-w64-emacs-git/src/emacs/lib-src/ntlib.c:110:15: 
> warning: format '%d' expects argument of type 'int', but argument 2 
> has type 'DWORD {aka long unsigned int}' [-Wformat=]
>
>        printf ("Checking parent status failed: %d\n", GetLastError ()); 

For portable code, that would be a valid warning. If GetLastError 
returns unsigned long, the format should use %lu, not %d

As I understand it, though, MS-Windows defines GetLastError to return 
unsigned int on 64-bit machines, and unsigned long on 32-bit machines 
(!). This idiosyncrasy could be handled in the MS-Windows port by 
something like this:

|#ifdef __MINGW64__ # define pDWORD "" #else # define pDWORD "l" #endif 
and then the above code could be: printf ("Checking parent status 
failed: %"pDWORD"u\n", GetLastError ()); |

Perhaps %u happens to work on both 32- and 64-bit MS-Windows, and if so 
then plain %u should suffice in practice.

> checking whether C compiler handles -Wbuiltin-macro-redefined... yes
> [...]
>
> Why enabling this by default? All this should be OFF by default and 
> only the maintainers which need it should enable it 

Warnings are enabled by default in master if you have a .git 
subdirectory (and thus are more likely to be a maintainer-type). See the 
thread containing this email:

http://lists.gnu.org/archive/html/emacs-devel/2016-04/msg00174.html

and the followup change here:

http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=5baecbc0ebc11178edd73431b644a5de0a31be25

It would be easy enough to disable -Wformat warnings when compiling 
anything under MS-Windows), if MS-Windows developers would prefer that. 
I'd rather leave these warnings enabled on non-MS-Windows platforms, 
though, as they're useful for catching portability glitches.

Re: MS-Windows warnings (was build failure) for Emacs master

[Angelo Graziosi]

I continue to be of the opinion that those warning should be off by 
default. Only who take care of Windows port etc. should enable the 
option to have them, if one needs them, obviously..

In any case, I have already taken the appropriate countermeasures to 
shutdown them..

  Angelo

Il 13/04/2016 00:52, Paul Eggert ha scritto:
> On 04/12/2016 01:54 PM, Angelo Graziosi wrote:
>
>> current build logs have a lot of "garbage" like this:
>>
>> [...]
>> C:/msys64/tmp/mingw-w64-emacs-git/src/emacs/lib-src/ntlib.c:110:15:
>> warning: format '%d' expects argument of type 'int', but argument 2
>> has type 'DWORD {aka long unsigned int}' [-Wformat=]
>>
>>        printf ("Checking parent status failed: %d\n", GetLastError ());
>
> For portable code, that would be a valid warning. If GetLastError
> returns unsigned long, the format should use %lu, not %d
>
> As I understand it, though, MS-Windows defines GetLastError to return
> unsigned int on 64-bit machines, and unsigned long on 32-bit machines
> (!). This idiosyncrasy could be handled in the MS-Windows port by
> something like this:
>
> |#ifdef __MINGW64__ # define pDWORD "" #else # define pDWORD "l" #endif
> and then the above code could be: printf ("Checking parent status
> failed: %"pDWORD"u\n", GetLastError ()); |
>
> Perhaps %u happens to work on both 32- and 64-bit MS-Windows, and if so
> then plain %u should suffice in practice.
>
>> checking whether C compiler handles -Wbuiltin-macro-redefined... yes
>> [...]
>>
>> Why enabling this by default? All this should be OFF by default and
>> only the maintainers which need it should enable it
>
> Warnings are enabled by default in master if you have a .git
> subdirectory (and thus are more likely to be a maintainer-type). See the
> thread containing this email:
>
> http://lists.gnu.org/archive/html/emacs-devel/2016-04/msg00174.html
>
> and the followup change here:
>
> http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=5baecbc0ebc11178edd73431b644a5de0a31be25
>
>
> It would be easy enough to disable -Wformat warnings when compiling
> anything under MS-Windows), if MS-Windows developers would prefer that.
> I'd rather leave these warnings enabled on non-MS-Windows platforms,
> though, as they're useful for catching portability glitches.

Re: MS-Windows warnings (was build failure) for Emacs master

[Yuri Khan]

On Wed, Apr 13, 2016 at 4:52 AM, Paul Eggert <eggert@cs.ucla.edu> wrote:
> On 04/12/2016 01:54 PM, Angelo Graziosi wrote:
>
>> C:/msys64/tmp/mingw-w64-emacs-git/src/emacs/lib-src/ntlib.c:110:15:
>> warning: format '%d' expects argument of type 'int', but argument 2 has type
>> 'DWORD {aka long unsigned int}' [-Wformat=]
>>
>>        printf ("Checking parent status failed: %d\n", GetLastError ());
>
> For portable code, that would be a valid warning. If GetLastError returns
> unsigned long, the format should use %lu, not %d

The wider issue here is that GetLastError return values are not even
meant for printf’ing. They are for feeding through FormatMessage.

> As I understand it, though, MS-Windows defines GetLastError to return
> unsigned int on 64-bit machines, and unsigned long on 32-bit machines (!).

That is, in <stdint.h> terms, uint32_t everywhere.

> This idiosyncrasy could be handled in the MS-Windows port by something like
> this:
>
>     #ifdef __MINGW64__
>     # define pDWORD ""
>     #else
>     # define pDWORD "l"
>     #endif

Alternatively, use the non-standard %I32u format specification.

Re: MS-Windows warnings (was build failure) for Emacs master

[Eli Zaretskii]

> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Tue, 12 Apr 2016 15:52:43 -0700
> Cc: emacs-devel@gnu.org
> 
> As I understand it, though, MS-Windows defines GetLastError to return 
> unsigned int on 64-bit machines, and unsigned long on 32-bit machines (!).

No, the return value is always DWORD, which is 'unsigned long', an
unsigned 32-bit type, on both 64-bit and 32-bit Windows.

Re: MS-Windows warnings (was build failure) for Emacs master

[Paul Eggert]

On 04/13/2016 08:26 AM, Eli Zaretskii wrote:
> the return value is always DWORD, which is 'unsigned long', an
> unsigned 32-bit type, on both 64-bit and 32-bit Windows.

Ah, sorry, I guess I was confused by Cygwin.

In that case the %ul format spec should suffice to print the number, 
unless you'd prefer to feed the number into FormatMessage as Yuri 
suggested.

Re: MS-Windows warnings (was build failure) for Emacs master

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org, angelo.graziosi@alice.it
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Wed, 13 Apr 2016 11:06:28 -0700
> 
> > In that case the %ul format spec should suffice to print the number, 

Right.

> unless you'd prefer to feed the number into FormatMessage as Yuri 
> suggested.

No, I see no need.  These errors should never happen, and I think the
function in question is not used nowadays anyway.

Re: MS-Windows warnings (was build failure) for Emacs master

[Davis Herring]

> As I understand it, though, MS-Windows defines GetLastError to return
> unsigned int on 64-bit machines, and unsigned long on 32-bit machines
> (!). This idiosyncrasy could be handled in the MS-Windows port by
> something like this:
>
> |#ifdef __MINGW64__ # define pDWORD "" #else # define pDWORD "l" #endif
> and then the above code could be: printf ("Checking parent status
> failed: %"pDWORD"u\n", GetLastError ()); |

I know the real problem has been addressed otherwise; however, surely 
for this sort of variability (between just these two types) one should 
simply write

printf ("Checking parent status failed: %lu\n",
         (unsigned long) GetLastError ());

without any preprocessor excitement.

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or 
too sparse, it is because mass-energy conversion has occurred during 
shipping.

Re: MS-Windows warnings (was build failure) for Emacs master

[Eli Zaretskii]

> From: Davis Herring <herring@lanl.gov>
> Date: Tue, 19 Apr 2016 09:46:43 -0600
> Cc: emacs-devel@gnu.org, Angelo Graziosi <angelo.graziosi@alice.it>
> 
> > As I understand it, though, MS-Windows defines GetLastError to return
> > unsigned int on 64-bit machines, and unsigned long on 32-bit machines
> > (!). This idiosyncrasy could be handled in the MS-Windows port by
> > something like this:
> >
> > |#ifdef __MINGW64__ # define pDWORD "" #else # define pDWORD "l" #endif
> > and then the above code could be: printf ("Checking parent status
> > failed: %"pDWORD"u\n", GetLastError ()); |
> 
> I know the real problem has been addressed otherwise; however, surely 
> for this sort of variability (between just these two types) one should 
> simply write
> 
> printf ("Checking parent status failed: %lu\n",
>          (unsigned long) GetLastError ());
> 
> without any preprocessor excitement.

If GetLastError returned a 64-bit value on 64-bit Windows, as Paul
thought, then your cast wouldn't be correct, of course, because
'unsigned long' is a 32-bit type on all versions of Windows.

Re: MS-Windows warnings (was build failure) for Emacs master

[Davis Herring]

> If GetLastError returned a 64-bit value on 64-bit Windows, as Paul
> thought, then your cast wouldn't be correct, of course, because
> 'unsigned long' is a 32-bit type on all versions of Windows.

I was responding only to the suggestion in its immediate context, not 
any actual statement (or other confusion) about GetLastError.  Paul's 
text mentions "unsigned int" and "unsigned long".  If those are the only 
two choices, the cast to unsigned long will always succeed.

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or 
too sparse, it is because mass-energy conversion has occurred during 
shipping.

Re: Build failure for Emacs master

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Angelo Graziosi <angelo.graziosi@alice.it>
> Date: Tue, 12 Apr 2016 02:36:23 +0200
> 
> Just for the record...
> 
> After about a month, the issue has reappeared.
> 
> Now it fails with this message:
> 
> [...]
> Loading button...
> Loading loaddefs.el (source)...
> Wrong number of arguments: autoload, 1325
> Makefile:540: set di istruzioni per l'obiettivo "emacs.exe" non riuscito
> make[1]: *** [emacs.exe] Errore 127
> make[1]: uscita dalla directory "/tmp/work/emacs-master/src"
> Makefile:398: set di istruzioni per l'obiettivo "src" non riuscito
> make: *** [src] Errore 2
> 
> but the lisp/loaddefs.el seems to have the same defects..

And if you delete loaddefs.el, does the following build succeed?

Re: Build failure for Emacs master

[Angelo Graziosi]

Il 12/04/2016 17:28, Eli Zaretskii ha scritto:
>> Cc: emacs-devel@gnu.org
>> From: Angelo Graziosi <angelo.graziosi@alice.it>
>> Date: Tue, 12 Apr 2016 02:36:23 +0200
>>
>> Just for the record...
>>
>> After about a month, the issue has reappeared.
>>
>> Now it fails with this message:
>>
>> [...]
>> Loading button...
>> Loading loaddefs.el (source)...
>> Wrong number of arguments: autoload, 1325
>> Makefile:540: set di istruzioni per l'obiettivo "emacs.exe" non riuscito
>> make[1]: *** [emacs.exe] Errore 127
>> make[1]: uscita dalla directory "/tmp/work/emacs-master/src"
>> Makefile:398: set di istruzioni per l'obiettivo "src" non riuscito
>> make: *** [src] Errore 2
>>
>> but the lisp/loaddefs.el seems to have the same defects..
>
> And if you delete loaddefs.el, does the following build succeed?

Hmm.. maybe. The fact is that currently I build using the MSYS2 
procedure 
(https://sourceforge.net/p/msys2/wiki/Contributing%20to%20MSYS2), i.e. 
with their build script PKGBUILD etc.

I will try to setup the build differently to verify your conjecture..


   Angelo

Re: Build failure for Emacs master

[Andy Moreton]

On Tue 12 Apr 2016, Eli Zaretskii wrote:

>> Cc: emacs-devel@gnu.org
>> From: Angelo Graziosi <angelo.graziosi@alice.it>
>> Date: Tue, 12 Apr 2016 02:36:23 +0200
>> 
>> Just for the record...
>> 
>> After about a month, the issue has reappeared.
>> 
>> Now it fails with this message:
>> 
>> [...]
>> Loading button...
>> Loading loaddefs.el (source)...
>> Wrong number of arguments: autoload, 1325
>> Makefile:540: set di istruzioni per l'obiettivo "emacs.exe" non riuscito
>> make[1]: *** [emacs.exe] Errore 127
>> make[1]: uscita dalla directory "/tmp/work/emacs-master/src"
>> Makefile:398: set di istruzioni per l'obiettivo "src" non riuscito
>> make: *** [src] Errore 2
>> 
>> but the lisp/loaddefs.el seems to have the same defects..
>
> And if you delete loaddefs.el, does the following build succeed?

As another data point, I have a similar failure from the emacs-25 branch
building with MSYS2 64bit on Win7.

On that machine, building with "make -j4" fails and produces a corrupted
loaddefs.el. Deleting loaddefs.el and rebuilding with "make -j4" also
fails. Deleting loaddefs.el and rebuilding with "make -j1" succeeds.

    AndyM

Re: Build failure for Emacs master

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Tue, 12 Apr 2016 22:49:32 +0100
> 
> As another data point, I have a similar failure from the emacs-25 branch
> building with MSYS2 64bit on Win7.
> 
> On that machine, building with "make -j4" fails and produces a corrupted
> loaddefs.el. Deleting loaddefs.el and rebuilding with "make -j4" also
> fails. Deleting loaddefs.el and rebuilding with "make -j1" succeeds.

Please debug the corruption.  I don't see the problem on any of my
builds, so I cannot debug this, and having all these reports without
anyone working on fixing the problem is very disturbing.

Thanks.

Re: Build failure for Emacs master

[Andy Moreton]

On Wed 13 Apr 2016, Eli Zaretskii wrote:

>> From: Andy Moreton <andrewjmoreton@gmail.com>
>> Date: Tue, 12 Apr 2016 22:49:32 +0100
>> 
>> As another data point, I have a similar failure from the emacs-25 branch
>> building with MSYS2 64bit on Win7.
>> 
>> On that machine, building with "make -j4" fails and produces a corrupted
>> loaddefs.el. Deleting loaddefs.el and rebuilding with "make -j4" also
>> fails. Deleting loaddefs.el and rebuilding with "make -j1" succeeds.
>
> Please debug the corruption.  I don't see the problem on any of my
> builds, so I cannot debug this, and having all these reports without
> anyone working on fixing the problem is very disturbing.

The corrupted loaddefs.el file seems to be the right size, but the last
768 bytes are zero instead of the expected content.

To debug this I have tried extracting the command from the autoload target
in lisp/Makefile and running that standalone. This results in repeatably
building a corrutp loaddefs.el (if it is deleted before each run).

Running that command using "bootstrap-emacs -Q" (i.e. interactively)
shows that the *autoload-file* buffer contains the 768 zero characters
at end, so it appears that the corruption occurs before the buffer is
written to disk.

Any hints on how to debug `batch-update-autoloads' to diagnose this
further would be welcome.

    AndyM

Re: Build failure for Emacs master

[Phillip Lord]

On Thu, April 14, 2016 12:11 am, Andy Moreton wrote:
> On Wed 13 Apr 2016, Eli Zaretskii wrote:
>
>
>>> From: Andy Moreton <andrewjmoreton@gmail.com>
>>> Date: Tue, 12 Apr 2016 22:49:32 +0100
>>>
>>>
>>> As another data point, I have a similar failure from the emacs-25
>>> branch building with MSYS2 64bit on Win7.
>>>
>>> On that machine, building with "make -j4" fails and produces a
>>> corrupted loaddefs.el. Deleting loaddefs.el and rebuilding with "make
>>> -j4" also
>>> fails. Deleting loaddefs.el and rebuilding with "make -j1" succeeds.
>>
>> Please debug the corruption.  I don't see the problem on any of my
>> builds, so I cannot debug this, and having all these reports without
>> anyone working on fixing the problem is very disturbing.
>
> The corrupted loaddefs.el file seems to be the right size, but the last
> 768 bytes are zero instead of the expected content.
>
>
> To debug this I have tried extracting the command from the autoload
> target in lisp/Makefile and running that standalone. This results in
> repeatably building a corrutp loaddefs.el (if it is deleted before each
> run).
>
> Running that command using "bootstrap-emacs -Q" (i.e. interactively)
> shows that the *autoload-file* buffer contains the 768 zero characters at
> end, so it appears that the corruption occurs before the buffer is written
> to disk.
>
> Any hints on how to debug `batch-update-autoloads' to diagnose this
> further would be welcome.


I haven't got this far, but I also have both Andy and Angelo's build
failures working (er, failing) reproducibly now. It also appears that my
autoload patch on master is causing a build failure (only on windows).

If no one solves it first, I will bisect this as soon as possible, but in
practice this is likely to be next week.

Phil

Re: Build failure for Emacs master

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 960 bytes --]

On 04/14/2016 12:32 AM, Phillip Lord wrote:
> I also have both Andy and Angelo's build
> failures working (er, failing) reproducibly now. It also appears that my
> autoload patch on master is causing a build failure (only on windows).
>
> If no one solves it first, I will bisect this as soon as possible, but in
> practice this is likely to be next week.

If you're right then there must be at least two bugs, right? Your 
autoload patch is only on master, but people are also reporting problems 
in the emacs-25 branch.

I am suspicious of copy-file. Why does it copy files possibly in text 
mode on MS-Windows, while truncating based on byte counts? That sounds 
like a recipe for trouble, as the miscount could cause file "truncation" 
to instead append a bunch of null bytes, which sounds like the symptoms 
you're observing.

Does the attached patch work around one of the bugs for you? This patch 
shouldn't affect behavior on GNUish or POSIXish hosts.

[-- Attachment #2: fileio.diff --]
[-- Type: text/x-patch, Size: 1140 bytes --]

diff --git a/src/fileio.c b/src/fileio.c
index dfab3de..69aa3f6 100644
--- a/src/fileio.c
+++ b/src/fileio.c
@@ -1929,7 +1929,7 @@ permissions.  */)
     }
 #else /* not WINDOWSNT */
   immediate_quit = 1;
-  ifd = emacs_open (SSDATA (encoded_file), O_RDONLY, 0);
+  ifd = emacs_open (SSDATA (encoded_file), O_RDONLY | O_BINARY, 0);
   immediate_quit = 0;
 
   if (ifd < 0)
@@ -1963,15 +1963,15 @@ permissions.  */)
   new_mask = S_IREAD | S_IWRITE;
 #endif
 
-  ofd = emacs_open (SSDATA (encoded_newname), O_WRONLY | O_CREAT | O_EXCL,
-		    new_mask);
+  ofd = emacs_open (SSDATA (encoded_newname),
+		    O_WRONLY | O_CREAT | O_EXCL | O_BINARY, new_mask);
   if (ofd < 0 && errno == EEXIST)
     {
       if (NILP (ok_if_already_exists) || INTEGERP (ok_if_already_exists))
 	barf_or_query_if_file_exists (newname, true, "copy to it",
 				      INTEGERP (ok_if_already_exists), false);
       already_exists = true;
-      ofd = emacs_open (SSDATA (encoded_newname), O_WRONLY, 0);
+      ofd = emacs_open (SSDATA (encoded_newname), O_WRONLY | O_BINARY, 0);
     }
   if (ofd < 0)
     report_file_error ("Opening output file", newname);

Re: Build failure for Emacs master

[Eli Zaretskii]

> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Thu, 14 Apr 2016 09:06:03 -0700
> Cc: emacs-devel@gnu.org
> 
> I am suspicious of copy-file. Why does it copy files possibly in text 
> mode on MS-Windows

No, we set the mode to be binary by default in emacs_open, if text
isn't explicitly requested:

  int
  emacs_open (const char *file, int oflags, int mode)
  {
    int fd;
    if (! (oflags & O_TEXT))
      oflags |= O_BINARY; <<<<<<<<<<<<<<<<<<<<<<<<<
    oflags |= O_CLOEXEC;
    while ((fd = open (file, oflags, mode)) < 0 && errno == EINTR)
      QUIT;

(Emacs on Windows always used binary mode by default, it's just that
we changed the implementation of that lately: instead of setting the
global variable _fmode, we now conceal that in emacs_open.)

O_BINARY and emacs_open (was: Build failure for Emacs master)

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 343 bytes --]

On 04/14/2016 09:19 AM, Eli Zaretskii wrote:
> No, we set the mode to be binary by default in emacs_open, if text
> isn't explicitly requested:

Ah, in that case we can simplify emacs_open callers, as they no longer 
need to worry about O_BINARY and this will help to lessen confusion such 
as mine. I installed the attached patch on master.


[-- Attachment #2: 0001-Simplify-use-of-O_BINARY.patch --]
[-- Type: application/x-patch, Size: 4928 bytes --]

Re: Build failure for Emacs master

[Phillip Lord]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 04/14/2016 12:32 AM, Phillip Lord wrote:
>> I also have both Andy and Angelo's build
>> failures working (er, failing) reproducibly now. It also appears that my
>> autoload patch on master is causing a build failure (only on windows).
>>
>> If no one solves it first, I will bisect this as soon as possible, but in
>> practice this is likely to be next week.
>
> If you're right then there must be at least two bugs, right? Your autoload
> patch is only on master, but people are also reporting problems in the
> emacs-25 branch.


If am right.

I'm trying to bisect - I'm using the following bisect script


git clean -xf --exclude "phil*"

./autogen.sh
PKG_CONFIG_PATH=/mingw64/lib/pkgconfig ./configure
make -j 20


The --exclude "phil*" is just to stop git clean from deleting my bisect
script. The make -j 20 is because Andrew said he was getting issues with
-j > 0.

I seem to get this crash a lot:


Checking d:/home/msys-emacs/git/master-copy/lisp/leim/quail/ARRAY30.el ...
Checking d:/home/msys-emacs/git/master-copy/lisp/leim/quail/arabic.el ...
Checking d:/home/msys-emacs/git/master-copy/lisp/leim/quail/4Corner.el ...
make[2]: Leaving directory '/d/home/msys-emacs/git/master-copy/leim'
Making generated-autoload-file local to  *autoload-file* while let-bound!
Loading macroexp.elc...
leim/ja-dic/ja-dic.el:0:0: error: scan-error: (Unbalanced parentheses 413482 2336649)
Makefile:181: recipe for target 'loaddefs.el' failed
make[2]: *** [loaddefs.el] Error 127
make[2]: Leaving directory '/d/home/msys-emacs/git/master-copy/lisp'
Makefile:701: recipe for target '../lisp/loaddefs.el' failed
make[1]: *** [../lisp/loaddefs.el] Error 2
make[1]: Leaving directory '/d/home/msys-emacs/git/master-copy/src'
Makefile:398: recipe for target 'src' failed
make: *** [src] Error 2



ja-dic.el is seriously ill. It has lots of ^M's all over the place and
the japanese characters are like so: "\344\202\222".

I am also seeing:

make[2]: Entering directory '/d/home/msys-emacs/git/master-copy/leim'
  GEN      ../lisp/leim/ja-dic/ja-dic.el
Symbol's function definition is void: char-displayable-p

Popping up periodically.

I haven't been able to try your patch yet, I am afraid.

Phil

Re: Build failure for Emacs master

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Date: Fri, 15 Apr 2016 12:08:20 +0100
> Cc: Andy Moreton <andrewjmoreton@gmail.com>, emacs-devel@gnu.org
> 
> > If you're right then there must be at least two bugs, right? Your autoload
> > patch is only on master, but people are also reporting problems in the
> > emacs-25 branch.
> 
> If am right.
> 
> I'm trying to bisect - I'm using the following bisect script
> 
> 
> git clean -xf --exclude "phil*"
> 
> ./autogen.sh
> PKG_CONFIG_PATH=/mingw64/lib/pkgconfig ./configure
> make -j 20
> 
> 
> The --exclude "phil*" is just to stop git clean from deleting my bisect
> script. The make -j 20 is because Andrew said he was getting issues with
> -j > 0.
> 
> I seem to get this crash a lot:

Please try again, after updating from emacs-25, or apply commit
ab849b7 from emacs-25 if you want to try that on master.  I think I
might just have squashed this elusive bug.

Thanks.

Re: Build failure for Emacs master

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: phillip.lord@russet.org.uk (Phillip Lord)
>> Date: Fri, 15 Apr 2016 12:08:20 +0100
>> Cc: Andy Moreton <andrewjmoreton@gmail.com>, emacs-devel@gnu.org
>> 
>> > If you're right then there must be at least two bugs, right? Your autoload
>> > patch is only on master, but people are also reporting problems in the
>> > emacs-25 branch.
>> 
>> If am right.
>> 
>> I'm trying to bisect - I'm using the following bisect script
>> 
>> 
>> git clean -xf --exclude "phil*"
>> 
>> ./autogen.sh
>> PKG_CONFIG_PATH=/mingw64/lib/pkgconfig ./configure
>> make -j 20
>> 
>> 
>> The --exclude "phil*" is just to stop git clean from deleting my bisect
>> script. The make -j 20 is because Andrew said he was getting issues with
>> -j > 0.
>> 
>> I seem to get this crash a lot:
>
> Please try again, after updating from emacs-25, or apply commit
> ab849b7 from emacs-25 if you want to try that on master.  I think I
> might just have squashed this elusive bug.


Sorry for the delay in testing, but I tried this out today multiple
times -- master is still crashing and emacs-25 is building freely, so it
does appear that the fix works. Even when using make highly parallel.

Phil

Re: Build failure for Emacs master

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Cc: eggert@cs.ucla.edu,  andrewjmoreton@gmail.com,  emacs-devel@gnu.org
> Date: Mon, 18 Apr 2016 20:08:14 +0100
> 
> > Please try again, after updating from emacs-25, or apply commit
> > ab849b7 from emacs-25 if you want to try that on master.  I think I
> > might just have squashed this elusive bug.
> 
> Sorry for the delay in testing, but I tried this out today multiple
> times -- master is still crashing and emacs-25 is building freely, so it
> does appear that the fix works. Even when using make highly parallel.

Thanks for testing.

Is the way master crashes documented in detail in some bug report or
some thread you can point to?

Re: Build failure for Emacs master

[Phillip Lord]

On Mon, April 18, 2016 8:33 pm, Eli Zaretskii wrote:
>> From: phillip.lord@russet.org.uk (Phillip Lord)
>> Cc: eggert@cs.ucla.edu,  andrewjmoreton@gmail.com,  emacs-devel@gnu.org
>> Date: Mon, 18 Apr 2016 20:08:14 +0100
>>
>>
>>> Please try again, after updating from emacs-25, or apply commit
>>> ab849b7 from emacs-25 if you want to try that on master.  I think I
>>> might just have squashed this elusive bug.
>>
>> Sorry for the delay in testing, but I tried this out today multiple
>> times -- master is still crashing and emacs-25 is building freely, so it
>>  does appear that the fix works. Even when using make highly parallel.
>
> Thanks for testing.
>
>
> Is the way master crashes documented in detail in some bug report or
> some thread you can point to?


The one you fixed on Emacs-25---corruption at the end of loaddefs.

Phil

Re: Build failure for Emacs master

[Paul Eggert]

On 04/18/2016 01:05 PM, Phillip Lord wrote:
> The one you fixed on Emacs-25---corruption at the end of loaddefs.

I just now merged emacs-25 into master, so the problem should be fixed 
in master too now.

Re: Build failure for Emacs master

[Phillip Lord]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 04/18/2016 01:05 PM, Phillip Lord wrote:
>> The one you fixed on Emacs-25---corruption at the end of loaddefs.
>
> I just now merged emacs-25 into master, so the problem should be fixed in
> master too now.


Unfortunately, I am still getting failing builds on master, so I guess
that there is a separate issue. It occurs repeatably, especially on
parallel builds.

The end of the build looks like this:


 ELC      emacs-lisp/autoload.elc
make[2]: Leaving directory '/d/home/msys-emacs/git/master-copy/lisp'
make -C ../lisp autoloads EMACS="../src/bootstrap-emacs.exe"
make[2]: Entering directory '/d/home/msys-emacs/git/master-copy/lisp'
  GEN      calendar/cal-loaddefs.el
  GEN      calendar/diary-loaddefs.el
  GEN      calendar/hol-loaddefs.el
  GEN      mh-e/mh-loaddefs.el
  GEN      net/tramp-loaddefs.el
Directories for loaddefs: . ./calc ./calendar ./cedet ./cedet/ede ./cedet/semantic ./cedet/semantic/analyze ./cedet/semantic/bovine ./cedet/semantic/decorate ./cedet/semantic/symref ./cedet/semantic/wisent ./cedet/srecode ./emacs-lisp ./emulation ./erc ./eshell ./gnus ./international ./language ./leim ./leim/ja-dic ./leim/quail ./mail ./mh-e ./net ./nxml ./org ./play ./progmodes ./textmodes ./url ./vc
  GEN      loaddefs.el
Making generated-autoload-file local to  *autoload-file* while let-bound!
Loading macroexp.elc...
leim/ja-dic/ja-dic.el:0:0: error: scan-error: (Unbalanced parentheses 413482 2336649)
Makefile:181: recipe for target 'loaddefs.el' failed
make[2]: *** [loaddefs.el] Error 127
make[2]: Leaving directory '/d/home/msys-emacs/git/master-copy/lisp'
Makefile:727: recipe for target '../lisp/loaddefs.el' failed
make[1]: *** [../lisp/loaddefs.el] Error 2
make[1]: Leaving directory '/d/home/msys-emacs/git/master-copy/src'
Makefile:398: recipe for target 'src' failed
make: *** [src] Error 2


I am finding some inconsistency between different machines; on one
machine everything works fine, on another this happens.

Phil

Re: Build failure for Emacs master

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Cc: Eli Zaretskii <eliz@gnu.org>,  andrewjmoreton@gmail.com,  emacs-devel@gnu.org
> Date: Wed, 20 Apr 2016 09:18:47 +0100
> 
> Paul Eggert <eggert@cs.ucla.edu> writes:
> 
> > On 04/18/2016 01:05 PM, Phillip Lord wrote:
> >> The one you fixed on Emacs-25---corruption at the end of loaddefs.
> >
> > I just now merged emacs-25 into master, so the problem should be fixed in
> > master too now.
> 
> 
> Unfortunately, I am still getting failing builds on master, so I guess
> that there is a separate issue. It occurs repeatably, especially on
> parallel builds.
> 
> The end of the build looks like this:
> 
> 
>  ELC      emacs-lisp/autoload.elc
> make[2]: Leaving directory '/d/home/msys-emacs/git/master-copy/lisp'
> make -C ../lisp autoloads EMACS="../src/bootstrap-emacs.exe"
> make[2]: Entering directory '/d/home/msys-emacs/git/master-copy/lisp'
>   GEN      calendar/cal-loaddefs.el
>   GEN      calendar/diary-loaddefs.el
>   GEN      calendar/hol-loaddefs.el
>   GEN      mh-e/mh-loaddefs.el
>   GEN      net/tramp-loaddefs.el
> Directories for loaddefs: . ./calc ./calendar ./cedet ./cedet/ede ./cedet/semantic ./cedet/semantic/analyze ./cedet/semantic/bovine ./cedet/semantic/decorate ./cedet/semantic/symref ./cedet/semantic/wisent ./cedet/srecode ./emacs-lisp ./emulation ./erc ./eshell ./gnus ./international ./language ./leim ./leim/ja-dic ./leim/quail ./mail ./mh-e ./net ./nxml ./org ./play ./progmodes ./textmodes ./url ./vc
>   GEN      loaddefs.el
> Making generated-autoload-file local to  *autoload-file* while let-bound!
> Loading macroexp.elc...
> leim/ja-dic/ja-dic.el:0:0: error: scan-error: (Unbalanced parentheses 413482 2336649)
> Makefile:181: recipe for target 'loaddefs.el' failed

Is this a full bootstrap, or just a rebuild after "git pull"?  If the
latter, copy ldefs-boot.el over loaddefs.el, and try again, or make a
full bootstrap.  (The problem might be triggered by a bad loaddefs.el
left from an old build, before the bug was solved.)

Re: Build failure for Emacs master

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: phillip.lord@russet.org.uk (Phillip Lord)
>> Cc: Eli Zaretskii <eliz@gnu.org>,  andrewjmoreton@gmail.com,  emacs-devel@gnu.org
>> Date: Wed, 20 Apr 2016 09:18:47 +0100
>> 
>> Paul Eggert <eggert@cs.ucla.edu> writes:
>> 
>> > On 04/18/2016 01:05 PM, Phillip Lord wrote:
>> >> The one you fixed on Emacs-25---corruption at the end of loaddefs.
>> >
>> > I just now merged emacs-25 into master, so the problem should be fixed in
>> > master too now.
>> 
>> 
>> Unfortunately, I am still getting failing builds on master, so I guess
>> that there is a separate issue. It occurs repeatably, especially on
>> parallel builds.
>> 
>> The end of the build looks like this:
>> 
>> 
>>  ELC      emacs-lisp/autoload.elc
>> make[2]: Leaving directory '/d/home/msys-emacs/git/master-copy/lisp'
>> make -C ../lisp autoloads EMACS="../src/bootstrap-emacs.exe"
>> make[2]: Entering directory '/d/home/msys-emacs/git/master-copy/lisp'
>>   GEN      calendar/cal-loaddefs.el
>>   GEN      calendar/diary-loaddefs.el
>>   GEN      calendar/hol-loaddefs.el
>>   GEN      mh-e/mh-loaddefs.el
>>   GEN      net/tramp-loaddefs.el
>> Directories for loaddefs: . ./calc ./calendar ./cedet ./cedet/ede ./cedet/semantic ./cedet/semantic/analyze ./cedet/semantic/bovine ./cedet/semantic/decorate ./cedet/semantic/symref ./cedet/semantic/wisent ./cedet/srecode ./emacs-lisp ./emulation ./erc ./eshell ./gnus ./international ./language ./leim ./leim/ja-dic ./leim/quail ./mail ./mh-e ./net ./nxml ./org ./play ./progmodes ./textmodes ./url ./vc
>>   GEN      loaddefs.el
>> Making generated-autoload-file local to  *autoload-file* while let-bound!
>> Loading macroexp.elc...
>> leim/ja-dic/ja-dic.el:0:0: error: scan-error: (Unbalanced parentheses 413482 2336649)
>> Makefile:181: recipe for target 'loaddefs.el' failed
>
> Is this a full bootstrap, or just a rebuild after "git pull"?  If the
> latter, copy ldefs-boot.el over loaddefs.el, and try again, or make a
> full bootstrap.  (The problem might be triggered by a bad loaddefs.el
> left from an old build, before the bug was solved.)

It's full bootstrap, am afraid.

git clean -xf
./autogen.sh
./configure
make

That the problem is occuring during generation of loaddefs.el is a red
herring I think. loaddefs.el is fine (although empty except for the
header). I'm guessing that ja-dic.el is loaded for the first time during
the loaddefs.el generation.

ja-dic.el looks like it has some serious encoding problems.

Phil

Re: Build failure for Emacs master

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Cc: eggert@cs.ucla.edu,  andrewjmoreton@gmail.com,  emacs-devel@gnu.org
> Date: Wed, 20 Apr 2016 17:55:36 +0100
> 
> That the problem is occuring during generation of loaddefs.el is a red
> herring I think. loaddefs.el is fine (although empty except for the
> header). I'm guessing that ja-dic.el is loaded for the first time during
> the loaddefs.el generation.
> 
> ja-dic.el looks like it has some serious encoding problems.

What encoding problems do you see there?  I see none: it loads as a
UTF-8 file with DOS EOLs.  What do you see?

Is your Git set up to not convert EOLs?  nt/INSTALL has this to say
about that:

  During Git installation, be sure to select the "Checkout as-is,
  commit as-is" option from the "Configure line ending conversions"
  dialog.  Otherwise, Git will convert text files to DOS-style CRLF
  end-of-line (EOL) format, which will cause subtle problems when
  building Emacs, because MSYS tools (see below) used to build Emacs
  use binary file I/O that preserves the CR characters that get in the
  way of some text-processing tools, like 'makeinfo' and the commands
  invoked by the autogen.sh script.

  If you already have Git installed and configured with some other EOL
  conversion option, you will need to reconfigure it, removing the
  following variables from all of your .gitconfig files:

    core.eol
    core.safecrlf
    core.autocrlf

  If you cloned the Emacs directory before changing these config
  variables, you will have to delete the repository and re-clone it
  after the change.

Re: Build failure for Emacs master

[Andy Moreton]

On Wed 20 Apr 2016, Phillip Lord wrote:

> Eli Zaretskii <eliz@gnu.org> writes:
>>
>> Is this a full bootstrap, or just a rebuild after "git pull"?  If the
>> latter, copy ldefs-boot.el over loaddefs.el, and try again, or make a
>> full bootstrap.  (The problem might be triggered by a bad loaddefs.el
>> left from an old build, before the bug was solved.)
>
> It's full bootstrap, am afraid.
>
> git clean -xf
> ./autogen.sh
> ./configure
> make
>
> That the problem is occuring during generation of loaddefs.el is a red
> herring I think. loaddefs.el is fine (although empty except for the
> header). I'm guessing that ja-dic.el is loaded for the first time during
> the loaddefs.el generation.
>
> ja-dic.el looks like it has some serious encoding problems.

lisp/leim/ja-dic/ is generated during bootstrap, so will not be cleaned
by your command above (and may still be corrupt from a previous ffailed
build). Try "git clean -xdf" instead.

    AndyM

Re: Build failure for Emacs master

[Phillip Lord]

Andy Moreton <andrewjmoreton@gmail.com> writes:

> On Wed 20 Apr 2016, Phillip Lord wrote:
>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>>
>>> Is this a full bootstrap, or just a rebuild after "git pull"?  If the
>>> latter, copy ldefs-boot.el over loaddefs.el, and try again, or make a
>>> full bootstrap.  (The problem might be triggered by a bad loaddefs.el
>>> left from an old build, before the bug was solved.)
>>
>> It's full bootstrap, am afraid.
>>
>> git clean -xf
>> ./autogen.sh
>> ./configure
>> make
>>
>> That the problem is occuring during generation of loaddefs.el is a red
>> herring I think. loaddefs.el is fine (although empty except for the
>> header). I'm guessing that ja-dic.el is loaded for the first time during
>> the loaddefs.el generation.
>>
>> ja-dic.el looks like it has some serious encoding problems.
>
> lisp/leim/ja-dic/ is generated during bootstrap, so will not be cleaned
> by your command above (and may still be corrupt from a previous ffailed
> build). Try "git clean -xdf" instead.

Ah, yes, indeed.

Thank you very much, that was driving me crazy.

Phil

Re: Build failure for Emacs master

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Thu, 14 Apr 2016 00:11:42 +0100
> 
> The corrupted loaddefs.el file seems to be the right size, but the last
> 768 bytes are zero instead of the expected content.

What is the total size of the loaddefs.el's text, in bytes?

Also, can you look in GDB at the buffer text, before it is written to
disk, and tell how far, in bytes, is the first null from the beginning
of buffer text?  (Let me know if you need advice for how to do that
with GDB commands.)

> Running that command using "bootstrap-emacs -Q" (i.e. interactively)
> shows that the *autoload-file* buffer contains the 768 zero characters
> at end, so it appears that the corruption occurs before the buffer is
> written to disk.

Yes, of course.  The corruption is caused by inserting and deleting
text into/from the buffer, as part as producing the autoload forms.

> Any hints on how to debug `batch-update-autoloads' to diagnose this
> further would be welcome.

The bug is unlikely to be on the Lisp level, it's something very deep
in managing buffer memory.  Perhaps even on the level of mmap_alloc
and mmap_realloc, as they are implemented in w32heap.c (since the only
people who complain about this are building Emacs on Windows).

If you can somehow establish where exactly in processing of the
autoloads the corruption happens (i.e. what file is Emacs processing
and what is the contents of the *autoload-file* buffer at that point),
that would be a step forward, I think.

Thanks.

Re: Build failure for Emacs master

[Andy Moreton]

On Thu 14 Apr 2016, Eli Zaretskii wrote:

>> From: Andy Moreton <andrewjmoreton@gmail.com>
>> Date: Thu, 14 Apr 2016 00:11:42 +0100
>> 
>> The corrupted loaddefs.el file seems to be the right size, but the last
>> 768 bytes are zero instead of the expected content.
>
> What is the total size of the loaddefs.el's text, in bytes?

For emacs-25 commit 24b87a1d4aadbdeafbc0db17e3a760cc9a7e21ef, the
corrupted loaddefs.el was 1192529 bytes, with the last 768 bytes zeroed.

Changing the files scanned for autoloads produces a loaddefs file of
different size (e.g. changed doc strings or different autoloads), and can
change the size of the zeroed portion at the end of the file.
 e.g. with local changes, 
    loaddefs.el 1192680 bytes, last 918 bytes zeroed
    loaddefs.el 1193247 bytes, last 918 bytes zeroed

> Also, can you look in GDB at the buffer text, before it is written to
> disk, and tell how far, in bytes, is the first null from the beginning
> of buffer text?  (Let me know if you need advice for how to do that
> with GDB commands.)

Please give precise instructions as I have no idea how to do that.

>> Running that command using "bootstrap-emacs -Q" (i.e. interactively)
>> shows that the *autoload-file* buffer contains the 768 zero characters
>> at end, so it appears that the corruption occurs before the buffer is
>> written to disk.
>
> Yes, of course.  The corruption is caused by inserting and deleting
> text into/from the buffer, as part as producing the autoload forms.

That wasn't at all obvious to me without some testing: it could easily
have been caused by problems serialising the buffer to disk.

>> Any hints on how to debug `batch-update-autoloads' to diagnose this
>> further would be welcome.
>
> The bug is unlikely to be on the Lisp level, it's something very deep
> in managing buffer memory.  Perhaps even on the level of mmap_alloc
> and mmap_realloc, as they are implemented in w32heap.c (since the only
> people who complain about this are building Emacs on Windows).
>
> If you can somehow establish where exactly in processing of the
> autoloads the corruption happens (i.e. what file is Emacs processing
> and what is the contents of the *autoload-file* buffer at that point),
> that would be a step forward, I think.

Any ideas to help do that are welcome. I've tried running the steps used
to build loaddefs.el from "bootstrap-emacs.exe -Q" and stepping though
it with edebug - that produces a working loaddefs.el :-(

If I start from a completely clean tree (after "git clean -Xdf") and
then copy the working loaddefs.el (produced above) to ldefs-boot.el,
then a full bootstrap succeeds, and the resulting loaddefs.el is good.

    AndyM

Re: Build failure for Emacs master

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Thu, 14 Apr 2016 18:06:02 +0100
> 
> > Also, can you look in GDB at the buffer text, before it is written to
> > disk, and tell how far, in bytes, is the first null from the beginning
> > of buffer text?  (Let me know if you need advice for how to do that
> > with GDB commands.)
> 
> Please give precise instructions as I have no idea how to do that.

Put  breakpoint in Fwrite_region, and when it breaks, type these
commands:

 (gdb) p/x &current_buffer->text->beg[0]
 (gdb) find /b9 current_buffer->text->beg,current_buffer->text->beg+GPT_BYTE-1,0

The first one should display the address of the beginning of buffer
text.  The second one should display up to 9 places where there are
nulls in the buffer, I expect them to be consecutive addresses.  The
first one of those is the address I'm looking for.

> > The corruption is caused by inserting and deleting text into/from
> > the buffer, as part as producing the autoload forms.
> 
> That wasn't at all obvious to me without some testing: it could easily
> have been caused by problems serialising the buffer to disk.

There's no serializing, buffer text is (almost) a linear array of
bytes.  (Or do you mean encoding?)

> > If you can somehow establish where exactly in processing of the
> > autoloads the corruption happens (i.e. what file is Emacs processing
> > and what is the contents of the *autoload-file* buffer at that point),
> > that would be a step forward, I think.
> 
> Any ideas to help do that are welcome.

I'd start by establishing what Lisp file contributes the portion that
should be where you have nulls.

> If I start from a completely clean tree (after "git clean -Xdf") and
> then copy the working loaddefs.el (produced above) to ldefs-boot.el,
> then a full bootstrap succeeds, and the resulting loaddefs.el is good.

Are you saying that loaddefs.el is corrupted because the previous
loaddefs.el was already corrupted?  If so, the corruption is not
really reproducible, is it?  What happens if you manually correct
loaddefs.el (e.g., by bringing a copy from another build), and then
try building without "git clean"?

Thanks.

Re: Build failure for Emacs master

[Andy Moreton]

On Thu 14 Apr 2016, Eli Zaretskii wrote:

>> From: Andy Moreton <andrewjmoreton@gmail.com>
>> Date: Thu, 14 Apr 2016 18:06:02 +0100
>> 
>> > Also, can you look in GDB at the buffer text, before it is written to
>> > disk, and tell how far, in bytes, is the first null from the beginning
>> > of buffer text?  (Let me know if you need advice for how to do that
>> > with GDB commands.)
>> 
>> Please give precise instructions as I have no idea how to do that.
>
> Put  breakpoint in Fwrite_region, and when it breaks, type these
> commands:
>
>  (gdb) p/x &current_buffer->text->beg[0]
>  (gdb) find /b9 current_buffer->text->beg,current_buffer->text->beg+GPT_BYTE-1,0
>
> The first one should display the address of the beginning of buffer
> text.  The second one should display up to 9 places where there are
> nulls in the buffer, I expect them to be consecutive addresses.  The
> first one of those is the address I'm looking for.

Starting from a tree without a loaddefs.el file, the breakpoint is hit
twice. The first time is when the skeleton loaddefs.el is written, that
contains just the comemnt header and the trailer with the local
variable settings. The second time is when it writes the completed
autoloads buffer:

Thread 1 hit Breakpoint 1, Fwrite_region (start=..., end=..., filename=..., append=..., visit=..., lockname=..., mustbenew=...) at ../../src/fileio.c:4678
4678	  return write_region (start, end, filename, append, visit, lockname, mustbenew,
(gdb) p/x &current_buffer->text->beg[0]
$4 = 0x65a0000
(gdb) find /b9 current_buffer->text->beg,current_buffer->text->beg+GPT_BYTE-1,0
0x66c36b4
1 pattern found.
(gdb) 

>> > The corruption is caused by inserting and deleting text into/from
>> > the buffer, as part as producing the autoload forms.
>> 
>> That wasn't at all obvious to me without some testing: it could easily
>> have been caused by problems serialising the buffer to disk.
>
> There's no serializing, buffer text is (almost) a linear array of
> bytes.  (Or do you mean encoding?)

I meant any of the processing to get from content in memory to bits on
spinning rust. That can include marshalling, encoding, flushing buffers
etc. (not all of which apply here).

>> If I start from a completely clean tree (after "git clean -Xdf") and
>> then copy the working loaddefs.el (produced above) to ldefs-boot.el,
>> then a full bootstrap succeeds, and the resulting loaddefs.el is good.
>
> Are you saying that loaddefs.el is corrupted because the previous
> loaddefs.el was already corrupted?  If so, the corruption is not
> really reproducible, is it?  What happens if you manually correct
> loaddefs.el (e.g., by bringing a copy from another build), and then
> try building without "git clean"?

The whole purpose of starting from a clean tree is to ensure that there
is no interference from a previous failed build. batch-update-autoloads
updates loaddefs.el in place if that file already exists, so if a
previous build has left a corrupted version then subsequent builds will
fail.

My point was that replacing ldefs-boot.el with an up to date copy of
loaddefs.el stopped the corruption. The bootstrap-emacs.exe built using
the udpated ldefs-boot.exe produced a working loaddefs.el.

    AndyM

Re: Build failure for Emacs master

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Thu, 14 Apr 2016 19:40:47 +0100
> 
> Thread 1 hit Breakpoint 1, Fwrite_region (start=..., end=..., filename=..., append=..., visit=..., lockname=..., mustbenew=...) at ../../src/fileio.c:4678
> 4678	  return write_region (start, end, filename, append, visit, lockname, mustbenew,
> (gdb) p/x &current_buffer->text->beg[0]
> $4 = 0x65a0000
> (gdb) find /b9 current_buffer->text->beg,current_buffer->text->beg+GPT_BYTE-1,0
> 0x66c36b4
> 1 pattern found.

That's not the nulls you reported.  What is the value of GPT_BYTE at
this point, and what is the value of Z_BYTE?

> > There's no serializing, buffer text is (almost) a linear array of
> > bytes.  (Or do you mean encoding?)
> 
> I meant any of the processing to get from content in memory to bits on
> spinning rust. That can include marshalling, encoding, flushing buffers
> etc. (not all of which apply here).

Theoretically, yes.  But not in practice, not in this case.

> > Are you saying that loaddefs.el is corrupted because the previous
> > loaddefs.el was already corrupted?  If so, the corruption is not
> > really reproducible, is it?  What happens if you manually correct
> > loaddefs.el (e.g., by bringing a copy from another build), and then
> > try building without "git clean"?
> 
> The whole purpose of starting from a clean tree is to ensure that there
> is no interference from a previous failed build. batch-update-autoloads
> updates loaddefs.el in place if that file already exists, so if a
> previous build has left a corrupted version then subsequent builds will
> fail.
> 
> My point was that replacing ldefs-boot.el with an up to date copy of
> loaddefs.el stopped the corruption. The bootstrap-emacs.exe built using
> the udpated ldefs-boot.exe produced a working loaddefs.el.

Sorry, I'm still confused: are the nulls coming from a previous
loaddefs.el, or are they coming from Emacs?  If the latter, is it true
that to reproduce the problem, you need to start from a clean tree,
but use an outdated version of ldefs-boot.el?

Thanks.

Re: Build failure for Emacs master

[Andy Moreton]

On Thu 14 Apr 2016, Eli Zaretskii wrote:

>> From: Andy Moreton <andrewjmoreton@gmail.com>
>> Date: Thu, 14 Apr 2016 19:40:47 +0100
>> 
>> Thread 1 hit Breakpoint 1, Fwrite_region (start=..., end=..., filename=...,
>> append=..., visit=..., lockname=..., mustbenew=...) at
>> ../../src/fileio.c:4678
>> 4678	  return write_region (start, end, filename, append, visit, lockname, mustbenew,
>> (gdb) p/x &current_buffer->text->beg[0]
>> $4 = 0x65a0000
>> (gdb) find /b9 current_buffer->text->beg,current_buffer->text->beg+GPT_BYTE-1,0
>> 0x66c36b4
>> 1 pattern found.
>
> That's not the nulls you reported.  What is the value of GPT_BYTE at
> this point, and what is the value of Z_BYTE?

It's not the same binary - efforts to detect what causes the corruption
necessitate rebuilding. 

>> > Are you saying that loaddefs.el is corrupted because the previous
>> > loaddefs.el was already corrupted?  If so, the corruption is not
>> > really reproducible, is it?  What happens if you manually correct
>> > loaddefs.el (e.g., by bringing a copy from another build), and then
>> > try building without "git clean"?
>> 
>> The whole purpose of starting from a clean tree is to ensure that there
>> is no interference from a previous failed build. batch-update-autoloads
>> updates loaddefs.el in place if that file already exists, so if a
>> previous build has left a corrupted version then subsequent builds will
>> fail.
>> 
>> My point was that replacing ldefs-boot.el with an up to date copy of
>> loaddefs.el stopped the corruption. The bootstrap-emacs.exe built using
>> the udpated ldefs-boot.exe produced a working loaddefs.el.
>
> Sorry, I'm still confused: are the nulls coming from a previous
> loaddefs.el, or are they coming from Emacs?  If the latter, is it true
> that to reproduce the problem, you need to start from a clean tree,
> but use an outdated version of ldefs-boot.el?

As I understand it, building from a clean tree:
 a) temacs.exe is built (a bare emacs)
 b) bootstrap-emacs.exe is built from temacs.exe, using ldefs-boot.el
 c) bootstrap-emacs.exe is used to create or update loaddefs.el
 d) emacs.exe is built using loaddefs.el

If (c) fails leaving a corrupted loaddefs.el file, then any following
build will also fail. This means that trying to dettermine what causes
the corruption requires that loaddefs.el is deleted before each attempt.

The zero bytes seen in this problem come from step (c). If similar
commands are run using "bootstrap-emacs -Q" then when the corruption
occurs, the zero bytes are visible in the emacs buffer.

As one of my experiments produced a good loaddefs.el, I did this:
 - start from a completely clean emacs-25 tree
 - overwrite ldefs-boot.el with the previously built loaddefs.el
 - do a full bootstrap, which succeeded.

I do not know if replacing ldefs-boot.el with a freshly built loaddefs.el
fixed the problem, or simply moved the timing around enough that this
Heisenbug no longer occurred.

    AndyM

Re: Build failure for Emacs master

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Thu, 14 Apr 2016 21:30:19 +0100
> 
> >> (gdb) p/x &current_buffer->text->beg[0]
> >> $4 = 0x65a0000
> >> (gdb) find /b9 current_buffer->text->beg,current_buffer->text->beg+GPT_BYTE-1,0
> >> 0x66c36b4
> >> 1 pattern found.
> >
> > That's not the nulls you reported.  What is the value of GPT_BYTE at
> > this point, and what is the value of Z_BYTE?
> 
> It's not the same binary - efforts to detect what causes the corruption
> necessitate rebuilding. 

You said you can reproduce this at will, so I thought doing that is
not a problem, and then the values would be the same as in this run.
Apologies if I misunderstood.

> > Sorry, I'm still confused: are the nulls coming from a previous
> > loaddefs.el, or are they coming from Emacs?  If the latter, is it true
> > that to reproduce the problem, you need to start from a clean tree,
> > but use an outdated version of ldefs-boot.el?
> 
> As I understand it, building from a clean tree:
>  a) temacs.exe is built (a bare emacs)
>  b) bootstrap-emacs.exe is built from temacs.exe, using ldefs-boot.el

The last part is only true if there's no loaddefs.el file in the
tree.  See the commentary and the relevant code in loadup.el.  If
loaddefs.el does exist, bootstrap-emacs build will use it.

> As one of my experiments produced a good loaddefs.el, I did this:
>  - start from a completely clean emacs-25 tree
>  - overwrite ldefs-boot.el with the previously built loaddefs.el
>  - do a full bootstrap, which succeeded.

What are the differences between that "successful" loaddefs.el and
ldefs-boot.el which produces a corrupted loaddefs.el?  The only
changes I see are in the time stamp cookies.

> I do not know if replacing ldefs-boot.el with a freshly built loaddefs.el
> fixed the problem, or simply moved the timing around enough that this
> Heisenbug no longer occurred.

Sorry, I don't understand: what timing could change as result of
modifying one of the inputs?  Are you saying that the differences (in
size or content) are so significant s to affect the time to read the
file into memory?

Re: Build failure for Emacs master

[Andy Moreton]

On Fri 15 Apr 2016, Eli Zaretskii wrote:

>> From: Andy Moreton <andrewjmoreton@gmail.com>
>> Date: Thu, 14 Apr 2016 21:30:19 +0100
>> 
>> >> (gdb) p/x &current_buffer->text->beg[0]
>> >> $4 = 0x65a0000
>> >> (gdb) find /b9 current_buffer->text->beg,current_buffer->text->beg+GPT_BYTE-1,0
>> >> 0x66c36b4
>> >> 1 pattern found.
>> >
>> > That's not the nulls you reported.  What is the value of GPT_BYTE at
>> > this point, and what is the value of Z_BYTE?
>> 
>> It's not the same binary - efforts to detect what causes the corruption
>> necessitate rebuilding. 
>
> You said you can reproduce this at will, so I thought doing that is
> not a problem, and then the values would be the same as in this run.
> Apologies if I misunderstood.

I can reproduce this repeatably from running make, or after building
bootstrap-emacs.exe using just the command line used to update
loadddefs.el.

Attempts to debug with bootstrap-emacs -Q and evaluating the equivalent
commands in the scratch buffer (or loaded from a file) do not show the
problem. Running under gdb gives variable results.

>> > Sorry, I'm still confused: are the nulls coming from a previous
>> > loaddefs.el, or are they coming from Emacs?  If the latter, is it true
>> > that to reproduce the problem, you need to start from a clean tree,
>> > but use an outdated version of ldefs-boot.el?
>> 
>> As I understand it, building from a clean tree:
>>  a) temacs.exe is built (a bare emacs)
>>  b) bootstrap-emacs.exe is built from temacs.exe, using ldefs-boot.el
>
> The last part is only true if there's no loaddefs.el file in the
> tree.  See the commentary and the relevant code in loadup.el.  If
> loaddefs.el does exist, bootstrap-emacs build will use it.

Noted, but here I am building from a clean tree so that loaddefs.el does
not exist.

>> As one of my experiments produced a good loaddefs.el, I did this:
>>  - start from a completely clean emacs-25 tree
>>  - overwrite ldefs-boot.el with the previously built loaddefs.el
>>  - do a full bootstrap, which succeeded.
>
> What are the differences between that "successful" loaddefs.el and
> ldefs-boot.el which produces a corrupted loaddefs.el?  The only
> changes I see are in the time stamp cookies.

The changes I see are:
 - timestamp changes
 - changed autoloads from your changes to grep.el
 - changes to the files listed in the comment at the end.

--[diff ldefs-boot.el loaddefs.el]------------------------------------
@@ -12876,16 +12829,22 @@
 The default grep program for `grep-command' and `grep-find-command'.
 This variable's value takes effect when `grep-compute-defaults' is called.")
 
-(defvar find-program (purecopy "find") "\
+(custom-autoload 'grep-program "grep" t)
+
+(defvar grep-find-program (purecopy "find") "\
 The default find program.
 This is used by commands like `grep-find-command', `find-dired'
 and others.")
 
-(defvar xargs-program (purecopy "xargs") "\
+(custom-autoload 'grep-find-program "grep" t)
+
+(defvar grep-xargs-program (purecopy "xargs") "\
 The default xargs program for `grep-find-command'.
 See `grep-find-use-xargs'.
 This variable's value takes effect when `grep-compute-defaults' is called.")
 
+(custom-autoload 'grep-xargs-program "grep" t)
+
 (defvar grep-find-use-xargs nil "\
 How to invoke find and grep.
 If `exec', use `find -exec {} ;'.
--[diff ldefs-boot.el loaddefs.el]------------------------------------

Should the define-obsolete-variable-alias forms for find-program and
xargs-program be marked as autoloads ?

>> I do not know if replacing ldefs-boot.el with a freshly built loaddefs.el
>> fixed the problem, or simply moved the timing around enough that this
>> Heisenbug no longer occurred.
>
> Sorry, I don't understand: what timing could change as result of
> modifying one of the inputs?  Are you saying that the differences (in
> size or content) are so significant s to affect the time to read the
> file into memory?

No, but changing the content of ldefs-boot.el can change the timing and
behaviour of the bootstrap-emacs.exe that is built from it. Whether the
observed bug is timing related or completely deterministic is not yet
known.

    AndyM

Re: Build failure for Emacs master

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Fri, 15 Apr 2016 09:18:58 +0100
> 
> No, but changing the content of ldefs-boot.el can change the timing and
> behaviour of the bootstrap-emacs.exe that is built from it. Whether the
> observed bug is timing related or completely deterministic is not yet
> known.

I've just fixed on the emacs-25 branch a serious bug in the w32
implementation of mmap_realloc.  The conditions under which the bug
happened are very rare, but when it did happen, we would in effect
lose a portion of buffer text, and instead get binary nulls.  Sounds
familiar?

So please try reproducing your problem after updating the repository
from upstream, I hope that the problems will be gone.  (If you are
tracking master, wait until ab849b7 is merged to it, or apply it
manually.)

Re: Build failure for Emacs master

[Andy Moreton]

On Fri 15 Apr 2016, Eli Zaretskii wrote:

>> From: Andy Moreton <andrewjmoreton@gmail.com>
>> Date: Fri, 15 Apr 2016 09:18:58 +0100
>> 
>> No, but changing the content of ldefs-boot.el can change the timing and
>> behaviour of the bootstrap-emacs.exe that is built from it. Whether the
>> observed bug is timing related or completely deterministic is not yet
>> known.
>
> I've just fixed on the emacs-25 branch a serious bug in the w32
> implementation of mmap_realloc.  The conditions under which the bug
> happened are very rare, but when it did happen, we would in effect
> lose a portion of buffer text, and instead get binary nulls.  Sounds
> familiar?
>
> So please try reproducing your problem after updating the repository
> from upstream, I hope that the problems will be gone.  (If you are
> tracking master, wait until ab849b7 is merged to it, or apply it
> manually.)

I've tried boostrapping a completely clean emacs-25 tree at commit
ab849b7, which was successful. Yuor patch looks good, so let's hope this
problem is gone.

    AndyM

Re: Build failure for Emacs master

[Angelo Graziosi]

Il 15/04/2016 16:38, Eli Zaretskii ha scritto:

> I've just fixed on the emacs-25 branch a serious bug in the w32
> implementation of mmap_realloc.  The conditions under which the bug

> So please try reproducing your problem after updating the repository
> from upstream, I hope that the problems will be gone.  (If you are
> tracking master, wait until ab849b7 is merged to it, or apply it
> manually.)
>

Since I use this patch, the build seems to be completed without issues..

Thanks,
  Angelo.

Re: Build failure for Emacs master

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Angelo Graziosi <angelo.graziosi@alice.it>
> Date: Mon, 18 Apr 2016 10:31:50 +0200
> 
> Il 15/04/2016 16:38, Eli Zaretskii ha scritto:
> 
> > I've just fixed on the emacs-25 branch a serious bug in the w32
> > implementation of mmap_realloc.  The conditions under which the bug
> 
> > So please try reproducing your problem after updating the repository
> > from upstream, I hope that the problems will be gone.  (If you are
> > tracking master, wait until ab849b7 is merged to it, or apply it
> > manually.)
> >
> 
> Since I use this patch, the build seems to be completed without issues..

Thanks for letting us know.

Re: Build failure for Emacs master

[Andy Moreton]

On Mon 18 Apr 2016, Eli Zaretskii wrote:

>> Cc: emacs-devel@gnu.org
>> From: Angelo Graziosi <angelo.graziosi@alice.it>
>> Date: Mon, 18 Apr 2016 10:31:50 +0200
>> 
>> Il 15/04/2016 16:38, Eli Zaretskii ha scritto:
>> 
>> > I've just fixed on the emacs-25 branch a serious bug in the w32
>> > implementation of mmap_realloc.  The conditions under which the bug
>> 
>> > So please try reproducing your problem after updating the repository
>> > from upstream, I hope that the problems will be gone.  (If you are
>> > tracking master, wait until ab849b7 is merged to it, or apply it
>> > manually.)
>> >
>> 
>> Since I use this patch, the build seems to be completed without issues..
>
> Thanks for letting us know.

FYI I have also bootstrapped emacs-25, and master (with the patch applied)
on the machine showing problems. Both built without the corruption, so I
think you patch has fixed this problem.

Thanks Eli!

    AndyM

Re: Build failure for Emacs master

[Angelo Graziosi]

Now master builds OB.

I just noticed the following message about SED while configuring (it was 
there also in my previous builds)

./autogen.sh

PKG_CONFIG_PATH=/mingw64/lib/pkgconfig ./configure --without-imagemagick
[...]
checking the archiver (ar) interface... ar
checking for ar... (cached) ar
checking for ranlib... ranlib
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... 64
checking whether gcc -I ./nt/inc accepts -g3 -O2... yes
sed: can't read conftest.c: No such file or directory
checking whether the compiler is clang... no
checking whether C compiler handles -Werror -Wunknown-warning-option... no
checking whether make supports nested variables... (cached) yes
checking whether ln -s works for files in the same directory... no, 
using /bin/ln -s
checking for install-info... /usr/bin/install-info
checking for gzip... /usr/bin/gzip
[...]


Ciao,
   Angelo.

Re: Build failure for Emacs master

[Eli Zaretskii]

> From: Angelo Graziosi <angelo.graziosi@alice.it>
> Date: Tue, 19 Apr 2016 11:58:57 +0200
> 
> I just noticed the following message about SED while configuring (it was 
> there also in my previous builds)
> 
> ./autogen.sh
> 
> PKG_CONFIG_PATH=/mingw64/lib/pkgconfig ./configure --without-imagemagick
> [...]
> checking the archiver (ar) interface... ar
> checking for ar... (cached) ar
> checking for ranlib... ranlib
> checking for special C compiler options needed for large files... no
> checking for _FILE_OFFSET_BITS value needed for large files... 64
> checking whether gcc -I ./nt/inc accepts -g3 -O2... yes
> sed: can't read conftest.c: No such file or directory

Doesn't happen here.

Re: Build failure for Emacs master

[Angelo Graziosi]

Il 19/04/2016 16:39, Eli Zaretskii ha scritto:
>> From: Angelo Graziosi <angelo.graziosi@alice.it>
>> Date: Tue, 19 Apr 2016 11:58:57 +0200
>>
>> I just noticed the following message about SED while configuring (it was
>> there also in my previous builds)
>>
>> ./autogen.sh
>>
>> PKG_CONFIG_PATH=/mingw64/lib/pkgconfig ./configure --without-imagemagick
>> [...]
>> checking the archiver (ar) interface... ar
>> checking for ar... (cached) ar
>> checking for ranlib... ranlib
>> checking for special C compiler options needed for large files... no
>> checking for _FILE_OFFSET_BITS value needed for large files... 64
>> checking whether gcc -I ./nt/inc accepts -g3 -O2... yes
>> sed: can't read conftest.c: No such file or directory
>
> Doesn't happen here.
>

I image you are using a git check out as source, instead I am using the 
tar-ball 
http://git.savannah.gnu.org/cgit/emacs.git/snapshot/emacs-master.tar.gz. 
The cause of that SED failure seems the omission of 
"--enable-gcc-warnings=no" option. Indeed when I use

   PKG_CONFIG_PATH=/mingw64/lib/pkgconfig ./configure 
--without-imagemagick --enable-gcc-warnings=no

the output is
[...]
checking for ranlib... ranlib
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... 64
checking whether gcc -I ./nt/inc accepts -g3 -O2... yes
checking whether the compiler is clang... no
checking whether C compiler handles -Werror -Wunknown-warning-option... no
checking whether make supports nested variables... (cached) yes
[...]

No SED failure.

Now all this seems strange because out of git check out, the option 
"--enable-gcc-warnings=no" should be enabled by default, if I understand 
some recent commit about it.. (It is YES, by default, in git check out, 
i think).

Maybe Paul has some idea about this..

  Ciao,
   Angelo.

Re: Build failure for Emacs master

[Angelo Graziosi]

Il 19/04/2016 16:39, Eli Zaretskii ha scritto:
>> From: Angelo Graziosi <angelo.graziosi@alice.it>
>> Date: Tue, 19 Apr 2016 11:58:57 +0200
>>
>> I just noticed the following message about SED while configuring (it was
>> there also in my previous builds)
>>
>> ./autogen.sh
>>
>> PKG_CONFIG_PATH=/mingw64/lib/pkgconfig ./configure --without-imagemagick
>> [...]
>> checking the archiver (ar) interface... ar
>> checking for ar... (cached) ar
>> checking for ranlib... ranlib
>> checking for special C compiler options needed for large files... no
>> checking for _FILE_OFFSET_BITS value needed for large files... 64
>> checking whether gcc -I ./nt/inc accepts -g3 -O2... yes
>> sed: can't read conftest.c: No such file or directory
>
> Doesn't happen here.
>

I image you are using a git check out as source, instead I am using the 
tar-ball 
http://git.savannah.gnu.org/cgit/emacs.git/snapshot/emacs-master.tar.gz. 
The cause of that SED failure seems the omission of 
"--enable-gcc-warnings=no" option. Indeed when I use

   PKG_CONFIG_PATH=/mingw64/lib/pkgconfig ./configure 
--without-imagemagick --enable-gcc-warnings=no

the output is
[...]
checking for ranlib... ranlib
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... 64
checking whether gcc -I ./nt/inc accepts -g3 -O2... yes
checking whether the compiler is clang... no
checking whether C compiler handles -Werror -Wunknown-warning-option... no
checking whether make supports nested variables... (cached) yes
[...]

No SED failure.

Now all this seems strange because out of git check out, the option 
"--enable-gcc-warnings=no" should be enabled by default, if I understand 
some recent commit about it.. (It is YES, by default, in git check out, 
i think).

Maybe Paul has some idea about this..

  Ciao,
   Angelo.

Re: Build failure for Emacs master

[Angelo Graziosi]

Il 19/04/2016 16:39, Eli Zaretskii ha scritto:
>> From: Angelo Graziosi <angelo.graziosi@alice.it>
>> Date: Tue, 19 Apr 2016 11:58:57 +0200
>>
>> I just noticed the following message about SED while configuring (it was
>> there also in my previous builds)
>>
>> ./autogen.sh
>>
>> PKG_CONFIG_PATH=/mingw64/lib/pkgconfig ./configure --without-imagemagick
>> [...]
>> checking the archiver (ar) interface... ar
>> checking for ar... (cached) ar
>> checking for ranlib... ranlib
>> checking for special C compiler options needed for large files... no
>> checking for _FILE_OFFSET_BITS value needed for large files... 64
>> checking whether gcc -I ./nt/inc accepts -g3 -O2... yes
>> sed: can't read conftest.c: No such file or directory
>
> Doesn't happen here.
>

I image you are using a git check out as source, instead I am using the 
tar-ball 
http://git.savannah.gnu.org/cgit/emacs.git/snapshot/emacs-master.tar.gz. 
The cause of that SED failure seems the omission of 
"--enable-gcc-warnings=no" option. Indeed when I use

   PKG_CONFIG_PATH=/mingw64/lib/pkgconfig ./configure 
--without-imagemagick --enable-gcc-warnings=no

the output is
[...]
checking for ranlib... ranlib
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... 64
checking whether gcc -I ./nt/inc accepts -g3 -O2... yes
checking whether the compiler is clang... no
checking whether C compiler handles -Werror -Wunknown-warning-option... no
checking whether make supports nested variables... (cached) yes
[...]

No SED failure.

Now all this seems strange because out of git check out, the option 
"--enable-gcc-warnings=no" should be enabled by default, if I understand 
some recent commit about it.. (It is YES, by default, in git check out, 
i think).

Maybe Paul has some idea about this..

  Ciao,
   Angelo.

Re: Build failure for Emacs master

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 176 bytes --]

On 04/19/2016 02:20 PM, Angelo Graziosi wrote:
> Maybe Paul has some idea about this.. 

Yes, and I installed the attached patch into 'master'. I hope this fixes 
the problem.

[-- Attachment #2: 0001-Avoid-AC_PREPROC_IFELSE-glitch-in-configure.ac.txt --]
[-- Type: text/plain, Size: 1187 bytes --]

From 32da59bb0c32ec726beb7871a6871fe980575f96 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Tue, 19 Apr 2016 17:22:22 -0700
Subject: [PATCH] Avoid AC_PREPROC_IFELSE glitch in configure.ac
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Problem reported by Angelo Graziosi in:
http://lists.gnu.org/archive/html/emacs-devel/2016-04/msg00545.html
* configure.ac (gl_gcc_warnings): Work around an Autoconf glitch:
AC_PREPROC_IFELSE doesn’t generate a simple shell command.
---
 configure.ac | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/configure.ac b/configure.ac
index dbb5ad2..1cd9017 100644
--- a/configure.ac
+++ b/configure.ac
@@ -879,8 +879,9 @@ AC_DEFUN
    # however, if there is also a .tarball-version file it is probably
    # just a release imported into Git for patch management.
    gl_gcc_warnings=no
-   test -e "$srcdir"/.git && test ! -f "$srcdir"/.tarball-version &&
+   if test -e "$srcdir"/.git && test ! -f "$srcdir"/.tarball-version; then
      gl_GCC_VERSION_IFELSE([5], [3], [gl_gcc_warnings=warn-only])]
+   fi
 )
 
 # clang is unduly picky about some things.
-- 
2.5.5

Re: Build failure for Emacs master

[Angelo Graziosi]

Il 20/04/2016 02:26, Paul Eggert ha scritto:
> On 04/19/2016 02:20 PM, Angelo Graziosi wrote:
>> Maybe Paul has some idea about this..
>
> Yes, and I installed the attached patch into 'master'. I hope this fixes
> the problem.

Yes, it works!

Many thanks,
  Angelo.


PS. I am afraid you have received more than a copy of my recent posts 
but there were an SMTP server issue. I hope it is fixed now otherwise 
you can receive more copies of this mail too..

Re: Build failure for Emacs master

[Angelo Graziosi]

Il 12/04/2016 17:28, Eli Zaretskii ha scritto:
>> From: Angelo Graziosi
>> Date: Tue, 12 Apr 2016 02:36:23 +0200
>>
>> Just for the record...
>>
>> After about a month, the issue has reappeared.
>>
>> Now it fails with this message:
>>
>> [...]
>> Loading button...
>> Loading loaddefs.el (source)...
>> Wrong number of arguments: autoload, 1325
>> Makefile:540: set di istruzioni per l'obiettivo "emacs.exe" non riuscito
>> make[1]: *** [emacs.exe] Errore 127
>> make[1]: uscita dalla directory "/tmp/work/emacs-master/src"
>> Makefile:398: set di istruzioni per l'obiettivo "src" non riuscito
>> make: *** [src] Errore 2
>>
>> but the lisp/loaddefs.el seems to have the same defects..
>
> And if you delete loaddefs.el, does the following build succeed?
>

Now I did the following (in a MSYS2/MINGW64 shell).


$ wget 
http://git.savannah.gnu.org/cgit/emacs.git/snapshot/emacs-master.tar.gz

$ tar -xf emacs-master.tar.gz

$ cd emacs-master/

$ ./autogen.sh 2>&1 | tee  ../emacs-build.log

$ ./configure --prefix=/opt/emacs --build=x86_64-w64-mingw32 
--enable-gcc-warnings=no --without-imagemagick 'CFLAGS=-pipe -O3 
-fomit-frame-pointer -funroll-loops' 2>&1 | tee -a ../emacs-build.log

$ make 2>&1 | tee -a ../emacs-build.log
[...]
   ELC      gnus/gnus-icalendar.elc
   ^M^M
In toplevel form:^M^M
gnus/gnus-icalendar.el:359:1:Error: End of file during parsing: 
c:/msys64/tmp/emacs-master/lisp/org/org-loaddefs.el^M
Makefile:282: set di istruzioni per l'obiettivo 
"gnus/gnus-icalendar.elc" non riuscito
make[2]: *** [gnus/gnus-icalendar.elc] Errore 1
make[2]: uscita dalla directory "/tmp/emacs-master/lisp"
make[2]: ingresso nella directory "/tmp/emacs-master/lisp"
   ELC      mh-e/mh-thread.elc
[...]
ELC      org/org-agenda.elc
^M^M
In toplevel form:^M^M
org/org-agenda.el:48:1:Error: End of file during parsing: 
c:/msys64/tmp/emacs-master/lisp/org/org-loaddefs.el^M
Makefile:282: set di istruzioni per l'obiettivo "org/org-agenda.elc" non 
riuscito
make[2]: *** [org/org-agenda.elc] Errore 1
make[2]: uscita dalla directory "/tmp/emacs-master/lisp"
make[2]: ingresso nella directory "/tmp/emacs-master/lisp"
[...]
   GEN      ../../info/wisent.info
   GEN      ../../info/woman.info
   GEN      ../../info/efaq-w32.info
make[2]: uscita dalla directory "/tmp/emacs-master/doc/misc"
   GEN      info/dir
make[1]: uscita dalla directory "/tmp/emacs-master"

In short, "make", apparently, did not stop but continued to the end..

So:

$ cd ..

$ grep -i error emacs-build.log
[...]
gnus/gnus-icalendar.el:359:1:Error: End of file during parsing: 
c:/msys64/tmp/emacs-master/lisp/org/org-loaddefs.el
make[2]: *** [gnus/gnus-icalendar.elc] Errore 1
In org-babel-lilypond-mark-error-line:
org/org-agenda.el:48:1:Error: End of file during parsing: 
c:/msys64/tmp/emacs-master/lisp/org/org-loaddefs.el
make[2]: *** [org/org-agenda.elc] Errore 1


At this point, I did:

$ rm emacs-master/lisp/org/org-loaddefs.el
$ cd emacs-master

$ make 2>&1 | tee -a ../emacs-build.log

Many many .elc file were created and

$ grep -i error ../emacs-build.log

did not show other errors. So, it seems that removing the offending file 
(org-loaddefs.el, in this case) fixes the build..


  Angelo

Re: Build failure for Emacs master

[Angelo Graziosi]

Surely this has nothing to do with the issue described here but why now


./autogen.sh
Checking whether you have the necessary tools...
(Read INSTALL.REPO for more details on building Emacs)
Checking for autoconf (need at least version 2.65)...
ok
Checking for automake (need at least version 1.11)...
ok
Your system has the required tools.
Running 'autoreconf -fi -I m4' ...
configure.ac:796: installing 'build-aux/ar-lib'
configure.ac:139: installing 'build-aux/config.guess'
configure.ac:139: installing 'build-aux/config.sub'
configure.ac:136: installing 'build-aux/install-sh'
configure.ac:136: installing 'build-aux/missing'
lib/Makefile.am: installing 'build-aux/depcomp'
You can now run './autogen.sh git'.


Really we need to run './autogen.sh git'?

Maybe I am wrong but INSTALL.REPO, which should contain the "more 
details on building Emacs", does not document this nor the "git" option 
for autogen.sh nor the other "all" option..

Maybe I am only adopting the wrong procedure to build Emacs and.. hem.. 
hem.. I see this build failure.. hem.. hem...

  Angelo

>> From: Angelo Graziosi
>> Date: Tue, 12 Apr 2016 02:36:23 +0200
>>
>> Just for the record...
>>
>> After about a month, the issue has reappeared.
>>
>> Now it fails with this message:
>>
>> [...]
>> Loading button...
>> Loading loaddefs.el (source)...
>> Wrong number of arguments: autoload, 1325
>> Makefile:540: set di istruzioni per l'obiettivo "emacs.exe" non riuscito
>> make[1]: *** [emacs.exe] Errore 127
>> make[1]: uscita dalla directory "/tmp/work/emacs-master/src"
>> Makefile:398: set di istruzioni per l'obiettivo "src" non riuscito
>> make: *** [src] Errore 2

Re: Build failure for Emacs master

[Paul Eggert]

On 04/13/2016 01:12 PM, Angelo Graziosi wrote:
> why now
>
> ./autogen.sh
> ...
> You can now run './autogen.sh git'.
>
> Really we need to run './autogen.sh git'?

Yes you need to, and yes it is silly.  This extra step was put in as a 
result of the email thread starting here:

http://lists.gnu.org/archive/html/emacs-devel/2016-02/msg00012.html

I preferred a simpler system where ./autogen.sh just works, without any 
extra steps, but I was outvoted.

> Maybe I am wrong but INSTALL.REPO, which should contain the "more 
> details on building Emacs", does not document this nor the "git" 
> option for autogen.sh nor the other "all" option.

Although INSTALL.REPO contains "more details", it shouldn't contain 
*all* details (as there are too many). It's not clear that this 
particular detail is worth recording in INSTALL.REPO.

> Maybe I am only adopting the wrong procedure to build Emacs

Yes, as you used a procedure that is not recommended in INSTALL.REPO. 
You would have avoided the './install.sh git' glitch if you had run 
plain 'make', or had run './autogen.sh all', which are the two methods 
that INSTALL.REPO suggests.

Re: Build failure for Emacs master

[Angelo Graziosi]

Il 13/04/2016 23:32, Paul Eggert ha scritto:
> On 04/13/2016 01:12 PM, Angelo Graziosi wrote:
>> why now
>>
>> ./autogen.sh
>> ...
>> You can now run './autogen.sh git'.
>>
>> Really we need to run './autogen.sh git'?
>
> Yes you need to, and yes it is silly.  This extra step was put in as a
> result of the email thread starting here:
>
> http://lists.gnu.org/archive/html/emacs-devel/2016-02/msg00012.html
>
> I preferred a simpler system where ./autogen.sh just works, without any
> extra steps, but I was outvoted.
>
>> Maybe I am wrong but INSTALL.REPO, which should contain the "more
>> details on building Emacs", does not document this nor the "git"
>> option for autogen.sh nor the other "all" option.
>
> Although INSTALL.REPO contains "more details", it shouldn't contain
> *all* details (as there are too many). It's not clear that this
> particular detail is worth recording in INSTALL.REPO.
>
>> Maybe I am only adopting the wrong procedure to build Emacs
>
> Yes, as you used a procedure that is not recommended in INSTALL.REPO.
> You would have avoided the './install.sh git' glitch if you had run
> plain 'make', or had run './autogen.sh all', which are the two methods
> that INSTALL.REPO suggests.


Hmmm... './autogen.sh all' does not work using 
http://git.savannah.gnu.org/cgit/emacs.git/snapshot/emacs-master.tar.gz 
as source:


./autogen.sh all
Checking whether you have the necessary tools...
(Read INSTALL.REPO for more details on building Emacs)
Checking for autoconf (need at least version 2.65)...
ok
Checking for automake (need at least version 1.11)...
ok
Your system has the required tools.
Running 'autoreconf -fi -I m4' ...
configure.ac:796: installing 'build-aux/ar-lib'
configure.ac:139: installing 'build-aux/config.guess'
configure.ac:139: installing 'build-aux/config.sub'
configure.ac:136: installing 'build-aux/install-sh'
configure.ac:136: installing 'build-aux/missing'
lib/Makefile.am: installing 'build-aux/depcomp'
Configuring local git repository...
cp: impossibile eseguire stat di '.git/config': No such file or directory

or one is "constrained" to use a clone of git repo? Only an ill mind 
could think that..

  Angelo

Re: Build failure for Emacs master

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 618 bytes --]

On 04/13/2016 03:00 PM, Angelo Graziosi wrote:
>
> './autogen.sh all' does not work using 
> http://git.savannah.gnu.org/cgit/emacs.git/snapshot/emacs-master.tar.gz as 
> source: 

That's a different approach from what's in INSTALL.REPO. INSTALL.REPO 
assumes you're building from a Git clone; among other things, that's why 
INSTALL.REPO requires that you have Git installed. Since you're using a 
strategy that does not involve Git, a different procedure is called for.

'./autogen.sh all' can be adjusted to work for your use case. I 
installed the attached patch, which attempts to do that. Please give it 
a try.

[-- Attachment #2: 0001-Port-.-autogen.sh-git-to-non-clones.txt --]
[-- Type: text/plain, Size: 3020 bytes --]

From 1aa7a86397a750cf93a6377e413cfb4ea4db2735 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Wed, 13 Apr 2016 18:19:04 -0700
Subject: [PATCH] =?UTF-8?q?Port=20=E2=80=98./autogen.sh=20git=E2=80=99=20t?=
 =?UTF-8?q?o=20non-clones?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Problem reported by Angelo Graziosi in:
http://lists.gnu.org/archive/html/emacs-devel/2016-04/msg00341.html
* autogen.sh (do_git): Default to false when the arg is ‘all’ but
there is no ‘.git’.
(git_common_dir, hooks): New vars.
(git_config, tailored_hooks, sample_hooks): Use them.
---
 autogen.sh | 26 +++++++++++++++++---------
 1 file changed, 17 insertions(+), 9 deletions(-)

diff --git a/autogen.sh b/autogen.sh
index 2e10a77..cd0accd 100755
--- a/autogen.sh
+++ b/autogen.sh
@@ -112,7 +112,8 @@ do_git=
       --help)
 	exec echo "$0: usage: $0 [all|autoconf|git]";;
       all)
-	do_autoconf=true do_git=true;;
+	do_autoconf=true
+	test -e .git && do_git=true;;
       autoconf)
 	do_autoconf=true;;
       git)
@@ -260,7 +261,8 @@ git_config ()
 		echo 'Configuring local git repository...'
 		case $cp_options in
 		  --backup=*)
-		    cp $cp_options --force .git/config .git/config || exit;;
+		    config=$git_common_dir/config
+		    cp $cp_options --force -- "$config" "$config" || exit;;
 		esac
 	    fi
 	    echo "git config $name '$value'"
@@ -272,6 +274,13 @@ git_config ()
 
 ## Configure Git, if requested.
 
+# Get location of Git's common configuration directory.  For older Git
+# versions this is just '.git'.  Newer Git versions support worktrees.
+
+test -e .git && git_common_dir=`git rev-parse --git-common-dir 2>/dev/null` ||
+  git_common_dir=.git
+hooks=$git_common_dir/hooks
+
 # Check hashes when transferring objects among repositories.
 
 git_config transfer.fsckObjects true
@@ -296,12 +305,11 @@ tailored_hooks=
 sample_hooks=
 
 for hook in commit-msg pre-commit; do
-    cmp build-aux/git-hooks/$hook .git/hooks/$hook >/dev/null 2>&1 ||
+    cmp -- build-aux/git-hooks/$hook "$hooks/$hook" >/dev/null 2>&1 ||
 	tailored_hooks="$tailored_hooks $hook"
 done
 for hook in applypatch-msg pre-applypatch; do
-    src=.git/hooks/$hook.sample
-    cmp "$src" .git/hooks/$hook >/dev/null 2>&1 ||
+    cmp -- "$hooks/$hook.sample" "$hooks/$hook" >/dev/null 2>&1 ||
 	sample_hooks="$sample_hooks $hook"
 done
 
@@ -311,15 +319,15 @@ sample_hooks=
 
 	if test -n "$tailored_hooks"; then
 	    for hook in $tailored_hooks; do
-		dst=.git/hooks/$hook
-		cp $cp_options build-aux/git-hooks/$hook "$dst" || exit
-		chmod a-w "$dst" || exit
+		dst=$hooks/$hook
+		cp $cp_options -- build-aux/git-hooks/$hook "$dst" || exit
+		chmod -- a-w "$dst" || exit
 	    done
 	fi
 
 	if test -n "$sample_hooks"; then
 	    for hook in $sample_hooks; do
-		cp $cp_options .git/hooks/$hook.sample .git/hooks/$hook || exit
+		cp $cp_options -- "$hooks/$hook.sample" "$hooks/$hook" || exit
 		chmod a-w .git/hooks/$hook || exit
 	    done
 	fi
-- 
2.5.5

Re: Build failure for Emacs master

[Angelo Graziosi]

Il 14/04/2016 03:31, Paul Eggert ha scritto:
> On 04/13/2016 03:00 PM, Angelo Graziosi wrote:
>>
>> './autogen.sh all' does not work using
>> http://git.savannah.gnu.org/cgit/emacs.git/snapshot/emacs-master.tar.gz as
>> source:
>
> That's a different approach from what's in INSTALL.REPO. INSTALL.REPO

In MSYS2 and other OS there are *-git packages just to have the last 
development source not because that people aims to develop and maintain 
the applications of those packages..

So even when Emacs source comes from git you should not need other 
option for autogen.sh...

> assumes you're building from a Git clone; among other things, that's why
> INSTALL.REPO requires that you have Git installed. Since you're using a
> strategy that does not involve Git, a different procedure is called for.
>
> './autogen.sh all' can be adjusted to work for your use case. I
> installed the attached patch, which attempts to do that. Please give it
> a try.

Maybe when I have some time..

Re: Build failure for Emacs master

[Eli Zaretskii]

> From: Angelo Graziosi <angelo.graziosi@alice.it>
> Date: Thu, 14 Apr 2016 10:03:19 +0200
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
> Il 14/04/2016 03:31, Paul Eggert ha scritto:
> > On 04/13/2016 03:00 PM, Angelo Graziosi wrote:
> >>
> >> './autogen.sh all' does not work using
> >> http://git.savannah.gnu.org/cgit/emacs.git/snapshot/emacs-master.tar.gz as
> >> source:
> >
> > That's a different approach from what's in INSTALL.REPO. INSTALL.REPO
> 
> In MSYS2 and other OS there are *-git packages just to have the last 
> development source not because that people aims to develop and maintain 
> the applications of those packages..

I'd urge all those who are tracking development sources to please use
the official established build procedure.  Using 3rd-party packages
whose procedures deviate from the project instructions is susceptible
to bumping into problems that stem from using those deviant
procedures, which is an added maintenance burden I think we'd like to
avoid.  Also, doing so means less testing for the official build
procedures.

If you can convince the MSYS2 maintainers to adopt the official build
procedure, it would be even better.  Failing that, please consider
switching back to what the project recommends, what is documented in
the relevant INSTALL documents.

TIA

Building Emacs on MSYS2 (was: Build failure for Emacs master)

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> In MSYS2 and other OS there are *-git packages just to have the last 
>> development source not because that people aims to develop and maintain 
>> the applications of those packages..
>
> I'd urge all those who are tracking development sources to please use
> the official established build procedure.  Using 3rd-party packages
> whose procedures deviate from the project instructions is susceptible
> to bumping into problems that stem from using those deviant
> procedures, which is an added maintenance burden I think we'd like to
> avoid.  Also, doing so means less testing for the official build
> procedures.

Currently this is the code for configuring the build of emacs-git on
MSYS2 MinGW-packages:

build() {
  [[ -d "${srcdir}/build-${MINGW_CHOST}" ]] && rm -rf "${srcdir}/build-${MINGW_CHOST}"
  mkdir -p "${srcdir}/build-${MINGW_CHOST}"
  cd "build-${MINGW_CHOST}"

  local with_wide_int='no'

  if test "${CARCH}" == 'x86_64'; then
    with_wide_int='yes'
  fi

  CPPFLAGS="-DNDEBUG -isystem ${MINGW_PREFIX}/include"
  CFLAGS="-pipe -O3 -fomit-frame-pointer -funroll-loops"
  LDFLAGS="-s -Wl,-s"
  "${srcdir}/${_realname}/configure" \
    --prefix="${MINGW_PREFIX}" \
    --build="${MINGW_CHOST}" \
    --with-wide-int="${with_wide_int}" \
    --with-sound="yes" \
    --with-file-notification="yes" \
    --without-gpm \
    --without-gconf \
    --without-gsettings \
    --without-selinux

  make
}

Apart from setting with_wide_int and adding some (unnecessary, IMHO)
optimization options, it looks like a fairly standard configuration.

The package contains two patches too. This one that looks like trying to
avoid a link problem:


--- src/image.c.orig	2014-10-15 14:18:29.088716000 +0200
+++ src/image.c	2014-10-15 15:54:12.407522800 +0200
@@ -7862,7 +7862,7 @@
   };
 
 #if defined HAVE_NTGUI && defined WINDOWSNT
-static bool init_imagemagick_functions (void);
+#define init_imagemagick_functions NULL
 #else
 #define init_imagemagick_functions NULL
 #endif



and this one that seems related to locating the source directory:

--- src/lread.c.orig	2014-11-04 20:29:22.129549000 +0100
+++ src/lread.c	2014-11-04 22:33:07.346414100 +0100
@@ -4351,6 +4351,12 @@
             } /* Vinstallation_directory != Vsource_directory */
 
         } /* if Vinstallation_directory */
+      else
+        {
+          Vsource_directory
+            = Fexpand_file_name (build_string ("../"),
+                                 Fcar (decode_env_path (0, PATH_DATA, 0)));
+        }
     }
   else                          /* !initialized */
     {


> If you can convince the MSYS2 maintainers to adopt the official build
> procedure, it would be even better.

I can change the emacs-git package for not adding those optimization
options and not setting with-wide-int. Do you think that's enough?

> Failing that, please consider
> switching back to what the project recommends, what is documented in
> the relevant INSTALL documents.

Because MSYS2 does not distribute binaries for emacs-git, I guess that
the number of people who currently uses the PKGBUILD shown above is very
small. AFAIK Angelo and others that build Emacs on MSYS2 are not using
the PKGBUILD, but following the recipe on the Emacs' docs (possibly with
their own personal modifications.)

Re: Building Emacs on MSYS2

[Paul Eggert]

On 04/14/2016 08:58 AM, Óscar Fuentes wrote:
>    local with_wide_int='no'
>
>    if test "${CARCH}" == 'x86_64'; then
>      with_wide_int='yes'
>    fi

You shouldn't need this. --with-wide-int is effective only on 32-bit 
platforms; integers are always wide on 64-bit platforms.

Re: Building Emacs on MSYS2

[Óscar Fuentes]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 04/14/2016 08:58 AM, Óscar Fuentes wrote:
>>    local with_wide_int='no'
>>
>>    if test "${CARCH}" == 'x86_64'; then
>>      with_wide_int='yes'
>>    fi
>
> You shouldn't need this. --with-wide-int is effective only on 32-bit
> platforms; integers are always wide on 64-bit platforms.

Right. Maybe the person who introduced that code on the PKGBUILD worried
about the possibility of changing the default on x86 to wide-int. IIRC
there was some discussion about that on the past.

Anyway, if the emacs maintainers decide to change the default, there is
no reason for overriding it on the PKGBUILD.

Re: Building Emacs on MSYS2

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Thu, 14 Apr 2016 18:13:41 +0200
> 
> Paul Eggert <eggert@cs.ucla.edu> writes:
> 
> > On 04/14/2016 08:58 AM, Óscar Fuentes wrote:
> >>    local with_wide_int='no'
> >>
> >>    if test "${CARCH}" == 'x86_64'; then
> >>      with_wide_int='yes'
> >>    fi
> >
> > You shouldn't need this. --with-wide-int is effective only on 32-bit
> > platforms; integers are always wide on 64-bit platforms.
> 
> Right. Maybe the person who introduced that code on the PKGBUILD worried
> about the possibility of changing the default on x86 to wide-int. IIRC
> there was some discussion about that on the past.

The discussions were about the 32-bit build.  If anything, the script
should use --with-wide-int in the 32-bit build, i.e. exactly the
opposite of what it does now.

Re: Building Emacs on MSYS2

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> >>    local with_wide_int='no'
>> >>
>> >>    if test "${CARCH}" == 'x86_64'; then
>> >>      with_wide_int='yes'
>> >>    fi
>> >
>> > You shouldn't need this. --with-wide-int is effective only on 32-bit
>> > platforms; integers are always wide on 64-bit platforms.
>> 
>> Right. Maybe the person who introduced that code on the PKGBUILD worried
>> about the possibility of changing the default on x86 to wide-int. IIRC
>> there was some discussion about that on the past.
>
> The discussions were about the 32-bit build.  If anything, the script
> should use --with-wide-int in the 32-bit build, i.e. exactly the
> opposite of what it does now.

My point is that the PKGBUILD author possibly was "protecting" the
package against upstream changing the x86 default to --with-wide-int.

Re: Building Emacs on MSYS2

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Thu, 14 Apr 2016 18:46:02 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> >>    local with_wide_int='no'
> >> >>
> >> >>    if test "${CARCH}" == 'x86_64'; then
> >> >>      with_wide_int='yes'
> >> >>    fi
> >> >
> >> > You shouldn't need this. --with-wide-int is effective only on 32-bit
> >> > platforms; integers are always wide on 64-bit platforms.
> >> 
> >> Right. Maybe the person who introduced that code on the PKGBUILD worried
> >> about the possibility of changing the default on x86 to wide-int. IIRC
> >> there was some discussion about that on the past.
> >
> > The discussions were about the 32-bit build.  If anything, the script
> > should use --with-wide-int in the 32-bit build, i.e. exactly the
> > opposite of what it does now.
> 
> My point is that the PKGBUILD author possibly was "protecting" the
> package against upstream changing the x86 default to --with-wide-int.

It does exactly the opposite, and to the non-x86 build, so I don't
understand how did you reach that conclusion.

Re: Building Emacs on MSYS2

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> >> >>    local with_wide_int='no'
>> >> >>
>> >> >>    if test "${CARCH}" == 'x86_64'; then
>> >> >>      with_wide_int='yes'
>> >> >>    fi
>> >> >
>> >> > You shouldn't need this. --with-wide-int is effective only on 32-bit
>> >> > platforms; integers are always wide on 64-bit platforms.
>> >> 
>> >> Right. Maybe the person who introduced that code on the PKGBUILD worried
>> >> about the possibility of changing the default on x86 to wide-int. IIRC
>> >> there was some discussion about that on the past.
>> >
>> > The discussions were about the 32-bit build.  If anything, the script
>> > should use --with-wide-int in the 32-bit build, i.e. exactly the
>> > opposite of what it does now.
>> 
>> My point is that the PKGBUILD author possibly was "protecting" the
>> package against upstream changing the x86 default to --with-wide-int.
>
> It does exactly the opposite, and to the non-x86 build, so I don't
> understand how did you reach that conclusion.

You misread the code. It unconditionally sets with_wide_int='no' and
*then* checks if the arch is x86_64, in which case sets
with_wide_int='yes'.

Hence the code is explicitly setting --with-wide-int on both cases to a
value which happens to coincide with the default *for now*.

Re: Building Emacs on MSYS2

[Paul Eggert]

On 04/14/2016 01:09 PM, Óscar Fuentes wrote:
> You misread the code. It unconditionally sets with_wide_int='no' and
> *then*  checks if the arch is x86_64, in which case sets
> with_wide_int='yes'.
>
> Hence the code is explicitly setting --with-wide-int on both cases to a
> value which happens to coincide with the default*for now*.

--with-wide-int is not the default on any platform.

Re: Building Emacs on MSYS2

[Óscar Fuentes]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 04/14/2016 01:09 PM, Óscar Fuentes wrote:
>> You misread the code. It unconditionally sets with_wide_int='no' and
>> *then*  checks if the arch is x86_64, in which case sets
>> with_wide_int='yes'.
>>
>> Hence the code is explicitly setting --with-wide-int on both cases to a
>> value which happens to coincide with the default*for now*.
>
> --with-wide-int is not the default on any platform.

Isn't it the default on 64 bit platforms, as you mentioned some messages
upthread?

Re: Building Emacs on MSYS2

[Paul Eggert]

On 04/15/2016 08:37 AM, Óscar Fuentes wrote:
>> --with-wide-int is not the default on any platform.
> Isn't it the default on 64 bit platforms, as you mentioned some messages
> upthread?
>
>

No. --with-wide-int causes Emacs to use 'long long int' for EMACS_INT, 
even if pointers fit in a narrower type. On my platform (Fedora 23 
x86-64), EMACS_INT is 'long int' because that is wide enough for 
pointers. Although 'long int' and 'long long int' both happen to be 
64-bit integers on my platform, 'long int' is a bit nicer (e.g., printf 
formats can use "%ld" rather than "%lld"), so using --with-wide-int 
would be a minor loss.

Re: Building Emacs on MSYS2

[Óscar Fuentes]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 04/15/2016 08:37 AM, Óscar Fuentes wrote:
>>> --with-wide-int is not the default on any platform.
>> Isn't it the default on 64 bit platforms, as you mentioned some messages
>> upthread?
>
> No. --with-wide-int causes Emacs to use 'long long int' for EMACS_INT,
> even if pointers fit in a narrower type. On my platform (Fedora 23
> x86-64), EMACS_INT is 'long int' because that is wide enough for
> pointers. Although 'long int' and 'long long int' both happen to be
> 64-bit integers on my platform, 'long int' is a bit nicer (e.g.,
> printf formats can use "%ld" rather than "%lld"), so using
> --with-wide-int would be a minor loss.

Which is irrelevant for the user who builds Emacs, isn't it?

I was just saying that Emacs wide integers are the default on 64 bit
platforms.

Re: Building Emacs on MSYS2

[Paul Eggert]

On 04/15/2016 10:58 AM, Óscar Fuentes wrote:
> Which is irrelevant for the user who builds Emacs, isn't it?

Yes, and this supports Eli's point. Generally speaking, it's better to 
keep the MSYS2 build simple so that it tracks mainline Emacs 
development. In particular, the --with-wide-int settings of the MSYS2 
build are irrelevant to the user who builds Emacs, so they should be 
omitted.

Re: Building Emacs on MSYS2

[Óscar Fuentes]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 04/15/2016 10:58 AM, Óscar Fuentes wrote:
>> Which is irrelevant for the user who builds Emacs, isn't it?
>
> Yes, and this supports Eli's point. Generally speaking, it's better to
> keep the MSYS2 build simple so that it tracks mainline Emacs
> development.

Why? The MSYS2 build for emacs-git (or any other package) is configured
for the MSYS2 user's convenience, as per the best judgement of the
PKGBUILD author.

> In particular, the --with-wide-int settings of the MSYS2
> build are irrelevant to the user who builds Emacs, so they should be
> omitted.

First things first, I was not involved on writing that PKGBUILD file. I
was just *guessing* that the author's intention is to enforce
--with-wide-int=no on the 32 bit build regardless of current and future
upstream's default setting.

Neither you nor I agree with that decision, but that's a different
topic. Maybe the author knows something that we ignore. Or maybe he is
simply imposing his personal preferences and prejudices. We don't know.

Re: Building Emacs on MSYS2

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Fri, 15 Apr 2016 21:40:55 +0200
> 
> Paul Eggert <eggert@cs.ucla.edu> writes:
> 
> > On 04/15/2016 10:58 AM, Óscar Fuentes wrote:
> >> Which is irrelevant for the user who builds Emacs, isn't it?
> >
> > Yes, and this supports Eli's point. Generally speaking, it's better to
> > keep the MSYS2 build simple so that it tracks mainline Emacs
> > development.
> 
> Why? The MSYS2 build for emacs-git (or any other package) is configured
> for the MSYS2 user's convenience, as per the best judgement of the
> PKGBUILD author.

If there are problems in the upstream build procedure that fail to
account for some use case, that use case should be brought to the
project's attention, so that the maintainers could decide if and how
they'd like to support that use case.  Deviating from the upstream
build procedure without talking with the project is not a good idea,
IMO.  Especially so since quite a few people who use the deviant
procedure are active participants in emacs-devel discussions and
report bugs, which could well be (and in fact were in the past found
to be) the result of the deviation.

> Neither you nor I agree with that decision, but that's a different
> topic. Maybe the author knows something that we ignore. Or maybe he is
> simply imposing his personal preferences and prejudices. We don't know.

All I say is that I urge people to use the official build procedure.
If you or someone else wants to talk to the author, or invite him/her
to discuss this here, that's a bonus.

Thanks.

Re: Building Emacs on MSYS2

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

> All I say is that I urge people to use the official build procedure.
> If you or someone else wants to talk to the author, or invite him/her
> to discuss this here, that's a bonus.

I'll like to remove that stuff from the PKGBUILD and submit a build
request (which will be accepted, I'm sure) but first I need to test the
build on x86 and x86_64, and that means quite a lot of time with my
(non-)hardware and scarce Windows usage. I hope to get it ready before
Emacs 25 is released.

Re: Building Emacs on MSYS2

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Fri, 15 Apr 2016 22:20:19 +0200
> 
> I'll like to remove that stuff from the PKGBUILD and submit a build
> request (which will be accepted, I'm sure) but first I need to test the
> build on x86 and x86_64, and that means quite a lot of time with my
> (non-)hardware and scarce Windows usage. I hope to get it ready before
> Emacs 25 is released.

Thanks.

Re: Building Emacs on MSYS2

[Stephen Leake]

Óscar Fuentes <ofv@wanadoo.es> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> All I say is that I urge people to use the official build procedure.
>> If you or someone else wants to talk to the author, or invite him/her
>> to discuss this here, that's a bonus.
>
> I'll like to remove that stuff from the PKGBUILD and submit a build
> request (which will be accepted, I'm sure) but first I need to test the
> build on x86 and x86_64, and that means quite a lot of time with my
> (non-)hardware and scarce Windows usage. I hope to get it ready before
> Emacs 25 is released.

If it would help, I also build Emacs with MSYS2. I have not been using
the PKGBUILD script, but it doesn't look hard.

So I can help with testing. I have Windows 8.1 64 bit, with msys2
mingw64 installed; I can install mingw32 if we want to test there.

One thing I'd like to fix; my build of emacs 25 does not display png
files. The binary pretest install does. I'd like to figure out the
difference.

-- 
-- Stephe

Re: Building Emacs on MSYS2

[Óscar Fuentes]

Stephen Leake <stephen_leake@stephe-leake.org> writes:

> Óscar Fuentes <ofv@wanadoo.es> writes:
>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>>> All I say is that I urge people to use the official build procedure.
>>> If you or someone else wants to talk to the author, or invite him/her
>>> to discuss this here, that's a bonus.
>>
>> I'll like to remove that stuff from the PKGBUILD and submit a build
>> request (which will be accepted, I'm sure) but first I need to test the
>> build on x86 and x86_64, and that means quite a lot of time with my
>> (non-)hardware and scarce Windows usage. I hope to get it ready before
>> Emacs 25 is released.
>
> If it would help, I also build Emacs with MSYS2. I have not been using
> the PKGBUILD script, but it doesn't look hard.
>
> So I can help with testing. I have Windows 8.1 64 bit, with msys2
> mingw64 installed; I can install mingw32 if we want to test there.

Thank you.

You need the PKGBUILD and accompanying files. You can check out
MINGW-Packages (https://github.com/Alexpux/MINGW-packages) or copy the
files of the mingw-w64-emacs-git directory from that repo on a local
directory. Remove from PKGBUILD all CPPFLAGS, CFLAGS and LDFLAGS
variables on the `build' function. Also remove all `configure'
paramaters except --prefix and --build. From a MSYS2 shell (not a
MinGWXX shell), cd to the directory that contains the PKBUILD file and
execute makepkg-mingw. Probably you will need to execute `makepkg-mingw
-s' for installing the dependencies. Taking a look at `makepkg-mingw
--help' will be useful. If the build goes well, this will create two
pacman package files, (32 and 64 bits). Install one of both packages
with pacman -U <file name of emacs package> and then you should have
emacs installed along the rest of your Mingw-w64 packages distributed
with MSYS2. Use it for some time.

You can also try to see what happens if you omit the patches. Remove
image.c.diff from `source' and the corresponding SHA hash from
`sha256sums' (the hashes of the files are listed in the same order as in
`sources'). See if the build breaks. See if imagemagick is detected. See
if you can use imagemagick from the resulting package. Report your
results. Ditto for lread.c.diff and locating Emacs .el files from Emacs.

> One thing I'd like to fix; my build of emacs 25 does not display png
> files. The binary pretest install does. I'd like to figure out the
> difference.

libpng is listed as an optional dependency, you need to install it
explicitly if you install the emacs binary package, but the build
process should download and install it when you build on your machine.

For the record, my old emacs-25 MSYS2 build can display png images fine.

Re: Building Emacs on MSYS2

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Sat, 16 Apr 2016 00:11:30 +0200
> 
> You can also try to see what happens if you omit the patches. Remove
> image.c.diff from `source' and the corresponding SHA hash from
> `sha256sums' (the hashes of the files are listed in the same order as in
> `sources'). See if the build breaks. See if imagemagick is detected. See
> if you can use imagemagick from the resulting package. Report your
> results.

The Imagemagick build is not officially supported in the MS-Windows
build, because no one submitted patches to make it behave like all the
other optional image libraries.  By that I mean: be able to run an
Emacs binary built with Imagemagick support on a system that doesn't
have the Imagemagick DLLs installed.  AFAIU, the image.c.diff hack
allows one to build Emacs with Imagemagick, but the resulting binary
will refuse to run on a system where it cannot load the Imagemagick
DLLs at startup.  That is not how we handle optional libraries in the
Windows build.

So the right way of handling this problem is to modify the Emacs
Imagemagick support code in image.c so that it uses LOAD_DLL_FN etc.
(There's one complication with Imagemagick: it needs to load 2 DLLs
rather than one, and figuring out how to deal with that is part of the
job.)

Fixing imagemagick on W32

[Stephen Leake]

Eli Zaretskii <eliz@gnu.org> writes:

> The Imagemagick build is not officially supported in the MS-Windows
> build, because no one submitted patches to make it behave like all the
> other optional image libraries.  By that I mean: be able to run an
> Emacs binary built with Imagemagick support on a system that doesn't
> have the Imagemagick DLLs installed.  AFAIU, the image.c.diff hack
> allows one to build Emacs with Imagemagick, but the resulting binary
> will refuse to run on a system where it cannot load the Imagemagick
> DLLs at startup.  That is not how we handle optional libraries in the
> Windows build.
>
> So the right way of handling this problem is to modify the Emacs
> Imagemagick support code in image.c so that it uses LOAD_DLL_FN etc.
> (There's one complication with Imagemagick: it needs to load 2 DLLs
> rather than one, and figuring out how to deal with that is part of the
> job.)

I'd like to work on this.

First I need a test; what elisp level functions use the imagemagick
functions in image.c? 

-- 
-- Stephe

Re: Fixing imagemagick on W32

[Eli Zaretskii]

> From: Stephen Leake <stephen_leake@stephe-leake.org>
> Cc: Óscar Fuentes <ofv@wanadoo.es>,  emacs-devel@gnu.org
> Date: Sun, 17 Apr 2016 16:46:31 -0500
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > The Imagemagick build is not officially supported in the MS-Windows
> > build, because no one submitted patches to make it behave like all the
> > other optional image libraries.  By that I mean: be able to run an
> > Emacs binary built with Imagemagick support on a system that doesn't
> > have the Imagemagick DLLs installed.  AFAIU, the image.c.diff hack
> > allows one to build Emacs with Imagemagick, but the resulting binary
> > will refuse to run on a system where it cannot load the Imagemagick
> > DLLs at startup.  That is not how we handle optional libraries in the
> > Windows build.
> >
> > So the right way of handling this problem is to modify the Emacs
> > Imagemagick support code in image.c so that it uses LOAD_DLL_FN etc.
> > (There's one complication with Imagemagick: it needs to load 2 DLLs
> > rather than one, and figuring out how to deal with that is part of the
> > job.)
> 
> I'd like to work on this.

Thank you.

> First I need a test; what elisp level functions use the imagemagick
> functions in image.c? 

Something that scales an image, perhaps in shr.el?

Re: Building Emacs on MSYS2

[Eli Zaretskii]

> From: Stephen Leake <stephen_leake@stephe-leake.org>
> Date: Fri, 15 Apr 2016 16:37:33 -0500
> Cc: emacs-devel@gnu.org
> 
> One thing I'd like to fix; my build of emacs 25 does not display png
> files. The binary pretest install does. I'd like to figure out the
> difference.

Are both binaries in the same directory?  Is the libpng DLL on PATH or
with one of the binaries?  Do you have more than one libpng DLL, one
of which might be in conflict with the w64 build, or from a different
version of libpng, or lacks its own support libraries?  What is the
version of libpng whose header files are visible when you build your
own Emacs, and are those headers from the same version of libpng as
the DLL you have installed?

These are some of the reasons that might explain the problem.  In
general, it's most probably some kind of DLL hell.

Re: Building Emacs on MSYS2

[Stephen Leake]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Stephen Leake <stephen_leake@stephe-leake.org>
>> Date: Fri, 15 Apr 2016 16:37:33 -0500
>> Cc: emacs-devel@gnu.org
>>
>> One thing I'd like to fix; my build of emacs 25 does not display png
>> files. The binary pretest install does. I'd like to figure out the
>> difference.
>
> Are both binaries in the same directory?

My build is in c:/Projects/emacs/emacs-25-build-mingw64/src (I don't
install, so I can easily switch versions), with libpng16-16.dll in
d:/msys64/mingw54/bin in PATH.

The error message is "Cannot display image: (Invalid image
specification)". I traced thru some of the code that tries to open
the image; image.c init-image-library returns nil. I haven't tried
running under the debugger to go further.

config.h sets HAVE_PNG true.


The pretest binary is in d:/Apps/emacs-25.0.90/bin, both emacs.exe and
libpng16-16.dll; mingw64 is _not_ in PATH.

cygwin "diff" says the two dlls differ.

> Is the libpng DLL on PATH or
> with one of the binaries?  Do you have more than one libpng DLL, one
> of which might be in conflict with the w64 build, or from a different
> version of libpng, or lacks its own support libraries?

depends.exe on emacs and libpng (both builds) reports no obvious errors
(it seems there is always something missing on windows; I assume those
are optional Windows components).

depends.exe says emacs.exe does not depend on libpng; I assume that's
because emacs only loads it if it is present.

> What is the
> version of libpng whose header files are visible when you build your
> own Emacs, and are those headers from the same version of libpng as
> the DLL you have installed?

The libpng comes from the msys2 package manager, so I assume it's all
consistent. How do I discover the version of the dll to check?

> These are some of the reasons that might explain the problem.  In
> general, it's most probably some kind of DLL hell.

Yes, I figured it was some obscure dll issue, which is why I have not
tried to track it down yet. Since the problem is not present in the
pretest, I figured it's a problem with my setup, not the Emacs sources,
so I did not report a bug.

--
-- Stephe

Re: Building Emacs on MSYS2

[Eli Zaretskii]

> From: Stephen Leake <stephen_leake@stephe-leake.org>
> Cc: ofv@wanadoo.es,  emacs-devel@gnu.org
> Date: Sat, 16 Apr 2016 07:17:25 -0500
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Stephen Leake <stephen_leake@stephe-leake.org>
> >> Date: Fri, 15 Apr 2016 16:37:33 -0500
> >> Cc: emacs-devel@gnu.org
> >>
> >> One thing I'd like to fix; my build of emacs 25 does not display png
> >> files. The binary pretest install does. I'd like to figure out the
> >> difference.
> >
> > Are both binaries in the same directory?
> 
> My build is in c:/Projects/emacs/emacs-25-build-mingw64/src (I don't
> install, so I can easily switch versions), with libpng16-16.dll in
> d:/msys64/mingw54/bin in PATH.
> 
> The error message is "Cannot display image: (Invalid image
> specification)". I traced thru some of the code that tries to open
> the image; image.c init-image-library returns nil. I haven't tried
> running under the debugger to go further.
> 
> config.h sets HAVE_PNG true.
> 
> 
> The pretest binary is in d:/Apps/emacs-25.0.90/bin, both emacs.exe and
> libpng16-16.dll; mingw64 is _not_ in PATH.
> 
> cygwin "diff" says the two dlls differ.

So this is the root cause, right there: the DLLs differ.  Where did
you get them?  Are both 64-bit DLLs?  Are they both MinGW DLLs?

> depends.exe on emacs and libpng (both builds) reports no obvious errors
> (it seems there is always something missing on windows; I assume those
> are optional Windows components).

I didn't mean errors.  Libpng should depend on zlib DLL; do you have
it? is it good? is there only one zlib DLL?

> The libpng comes from the msys2 package manager, so I assume it's all
> consistent.

Does it come with zlib DLL, or did you install zlib DLL separately?
If the latter, they could be inconsistent.

> Yes, I figured it was some obscure dll issue, which is why I have not
> tried to track it down yet. Since the problem is not present in the
> pretest, I figured it's a problem with my setup, not the Emacs sources,
> so I did not report a bug.

I don't understand why you (and others) use different PATH
arrangements to run different binaries, and have multiple versions of
support DLLs.  Isn't it clear that by doing so you create your own DLL
hell?  It's much better to have just one copy of each support DLL, in
a place that is always on PATH.

Re: png in emacs 25 mingw64

[Stephen Leake]

TLDR; I usually have Cygwin in the PATH as well; it has shell tools I
can't get in Mingw (yet). It turns out that is the problem; if I run my
build of Emacs from a Mingw64 shell (no Cygwin in PATH), it can show
PNGs.

More details below.

Question; exactly which path is searched for DLLs? I have two:

PATH from the parent process

exec-path after editing in ~/.emacs

The png image is opened well after ~/.emacs is processed, so does it
look on the edited exec-path for the dll?

Or does the dll search take place during startup, in which case it is
the parent PATH (or exec-path before editing in ~/.emacs).

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Stephen Leake <stephen_leake@stephe-leake.org>
>> Cc: ofv@wanadoo.es,  emacs-devel@gnu.org
>> Date: Sat, 16 Apr 2016 07:17:25 -0500
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> From: Stephen Leake <stephen_leake@stephe-leake.org>
>> >> Date: Fri, 15 Apr 2016 16:37:33 -0500
>> >> Cc: emacs-devel@gnu.org
>> >>
>> >> One thing I'd like to fix; my build of emacs 25 does not display png
>> >> files. The binary pretest install does. I'd like to figure out the
>> >> difference.
>> >
>> > Are both binaries in the same directory?
>> 
>> My build is in c:/Projects/emacs/emacs-25-build-mingw64/src (I don't
>> install, so I can easily switch versions), with libpng16-16.dll in
>> d:/msys64/mingw54/bin in PATH.
>> 
>> The error message is "Cannot display image: (Invalid image
>> specification)". I traced thru some of the code that tries to open
>> the image; image.c init-image-library returns nil. I haven't tried
>> running under the debugger to go further.
>> 
>> config.h sets HAVE_PNG true.
>> 
>> 
>> The pretest binary is in d:/Apps/emacs-25.0.90/bin, both emacs.exe and
>> libpng16-16.dll; mingw64 is _not_ in PATH.
>> 
>> cygwin "diff" says the two dlls differ.
>
> So this is the root cause, right there: the DLLs differ.  

I don't see why that has to be the problem; the difference could be in
some date.

> Where did
> you get them?  

One from the MSYS2 distribution, the other from the pretest binary,
which may have been built with the MSYS2 distribution, but I'm not sure.


>Are both 64-bit DLLs? Are they both MinGW DLLs?

I don't know how to tell.

>> depends.exe on emacs and libpng (both builds) reports no obvious errors
>> (it seems there is always something missing on windows; I assume those
>> are optional Windows components).
>
> I didn't mean errors.  Libpng should depend on zlib DLL; 

Yes; zlib1.dll

>do you have
> it? 

Yes; in the respective distributions.

>is it good? 

I don't know how to tell. I guess I need to find some other Mingw64
program that uses it?

>is there only one zlib DLL?

In PATH for each build, yes. Although "type -a zlib1.dll" says "not
found", so it's hard to be sure.

>> The libpng comes from the msys2 package manager, so I assume it's all
>> consistent.
>
> Does it come with zlib DLL, or did you install zlib DLL separately?
> If the latter, they could be inconsistent.

Still from the package manager. Yes, the packages could be inconsistent
with each other.

>> Yes, I figured it was some obscure dll issue, which is why I have not
>> tried to track it down yet. Since the problem is not present in the
>> pretest, I figured it's a problem with my setup, not the Emacs sources,
>> so I did not report a bug.
>
> I don't understand why you (and others) use different PATH
> arrangements to run different binaries, and have multiple versions of
> support DLLs.  

If I want to use the Mingw distribution, I use that path.

If I want to use the Emacs pretest binary distribution, I use that path.

I do _not_ mix distributions.

I have no idea how the Emacs pretest was built; simply assuming it is
the same as my current Mingw installation is almost guarranteed to be
false.

That's what PATH is for.

>Isn't it clear that by doing so you create your own DLL
> hell?  

No, I'm avoiding it by not mixing distributions.

>It's much better to have just one copy of each support DLL, in
> a place that is always on PATH.

That would be nice. I can do that on Debian, that has only one package
manager, and a rigorous distribution policy.

I could do that on Windows by only using MSYS2. But then I would not be
able to use the Emacs pretest.


Hmm. I do usually have Cygwin in the PATH as well; it has shell tools I
can't get in Mingw (yet). It turns out that is the problem; if I run my
build of Emacs from a Mingw64 shell (no Cygwin in PATH), it can show
PNGs.

So your point about mixing distributions is valid; I was not being
careful enough. Most of the time it's not a problem (this is the first
problem I can remember).

-- 
-- Stephe

Re: png in emacs 25 mingw64

[Stephen Leake]

Stephen Leake <stephen_leake@stephe-leake.org> writes:

> Question; exactly which path is searched for DLLs? I have two:
>
> PATH from the parent process
>
> exec-path after editing in ~/.emacs
>
> The png image is opened well after ~/.emacs is processed, so does it
> look on the edited exec-path for the dll?
>
> Or does the dll search take place during startup, in which case it is
> the parent PATH (or exec-path before editing in ~/.emacs).

Apparently the dll is searched for on the parent process PATH; moving
mingw64/bin earlier in the parent process PATH (without changing the
editing in ~/.emacs) fixes the PNG problem for me.

-- 
-- Stephe

Re: png in emacs 25 mingw64

[Eli Zaretskii]

> From: Stephen Leake <stephen_leake@stephe-leake.org>
> Cc: ofv@wanadoo.es,  emacs-devel@gnu.org
> Date: Sat, 16 Apr 2016 16:02:06 -0500
> 
> Apparently the dll is searched for on the parent process PATH

No, it's Windows PATH augmented by several directories.

Re: png in emacs 25 mingw64

[Eli Zaretskii]

> From: Stephen Leake <stephen_leake@stephe-leake.org>
> Cc: ofv@wanadoo.es,  emacs-devel@gnu.org
> Date: Sat, 16 Apr 2016 15:34:16 -0500
> 
> 
> Question; exactly which path is searched for DLLs? I have two:
> 
> PATH from the parent process
> 
> exec-path after editing in ~/.emacs
> 
> The png image is opened well after ~/.emacs is processed, so does it
> look on the edited exec-path for the dll?
> 
> Or does the dll search take place during startup, in which case it is
> the parent PATH (or exec-path before editing in ~/.emacs).

DLL search is done by the OS.  See here:

  https://msdn.microsoft.com/en-us/library/windows/desktop/ms682586%28v=vs.85%29.aspx

> >Are both 64-bit DLLs? Are they both MinGW DLLs?
> 
> I don't know how to tell.

The 'file' command is one way.

> > I don't understand why you (and others) use different PATH
> > arrangements to run different binaries, and have multiple versions of
> > support DLLs.  
> 
> If I want to use the Mingw distribution, I use that path.
> 
> If I want to use the Emacs pretest binary distribution, I use that path.

They are both MinGW programs, so I don't understand why you would need
to keep them apart.

> I have no idea how the Emacs pretest was built; simply assuming it is
> the same as my current Mingw installation is almost guarranteed to be
> false.

There's only one procedure to build Emacs on Windows, so yes, you can
assume that, and it will be almost certainly true.

> >Isn't it clear that by doing so you create your own DLL
> > hell?  
> 
> No, I'm avoiding it by not mixing distributions.

Suit yourself, but then don't be surprised when you have such
problems.

> Hmm. I do usually have Cygwin in the PATH as well

Now, _that_ you should keep separate from MinGW.

Re: Building Emacs on MSYS2

[Phillip Lord]

Stephen Leake <stephen_leake@stephe-leake.org> writes:

> Óscar Fuentes <ofv@wanadoo.es> writes:
>
> If it would help, I also build Emacs with MSYS2. I have not been using
> the PKGBUILD script, but it doesn't look hard.
>
> So I can help with testing. I have Windows 8.1 64 bit, with msys2
> mingw64 installed; I can install mingw32 if we want to test there.
>
> One thing I'd like to fix; my build of emacs 25 does not display png
> files. The binary pretest install does. I'd like to figure out the
> difference.


All I did was make sure that the libpng libraries were available on the
build machine. The binary pretest needs the DLLs locally, though, or it
fails. Only Xpm is supported by the bundle.

Phil

Re: Building Emacs on MSYS2

[Stefan Monnier]

> I was just saying that Emacs wide integers are the default on 64 bit
> platforms.

No: --with-wide-int was introduced so that Emacs can use integers which
are twice the size of pointers (at the cost of making Lisp_Object twice
the size, hence wasting space for all non-integer values).  I.e. on
a 64bit system, this would mean that the Lisp_Object type is 128bit
wide, which would be a waste.

Luckily for that script you quoted, on 64bit hosts, we basically
ignore --with-wide-int, so even though it goes to extra trouble to
set --with-wide-int on amd64 we disregard this "preposterous request".


        Stefan

Re: Building Emacs on MSYS2

[Óscar Fuentes]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I was just saying that Emacs wide integers are the default on 64 bit
>> platforms.
>
> No: --with-wide-int was introduced so that Emacs can use integers which
> are twice the size of pointers (at the cost of making Lisp_Object twice
> the size, hence wasting space for all non-integer values).

AFAIK that was the reasoning when 64 bit platforms were not prevalent
and 32 bit the norm.

> I.e. on
> a 64bit system, this would mean that the Lisp_Object type is 128bit
> wide, which would be a waste.
>
> Luckily for that script you quoted, on 64bit hosts, we basically
> ignore --with-wide-int, so even though it goes to extra trouble to
> set --with-wide-int on amd64 we disregard this "preposterous request".

Hence effectively --with-wide-int is a method for ensuring that a 64 bit
type is used for storing Emacs integers. But as on 64 bit platforms
Emacs already uses a 64 bit type for that, there is no observable
difference from the POV of the user among passing --with-wide-int=yes to
the configure script and not doing it. Which is in line with what you
quoted from my previous message.

Now, can be agree on that I'm right and stop this pointless discussion
:-)

Re: Building Emacs on MSYS2

[Stefan Monnier]

>>> I was just saying that Emacs wide integers are the default on 64 bit
>>> platforms.
>> No: --with-wide-int was introduced so that Emacs can use integers which
>> are twice the size of pointers (at the cost of making Lisp_Object twice
>> the size, hence wasting space for all non-integer values).
> AFAIK that was the reasoning when 64 bit platforms were not prevalent
> and 32 bit the norm.

--with-wide-int is pretty recent, so 64bit was already prevalent when
it appeared.  IOW The reasoning hasn't changed since.

> Hence effectively --with-wide-int is a method for ensuring that a 64 bit
> type is used for storing Emacs integers.

That's the end result in practice, I guess.  But from the implementation
point of view it's quite different.

> Now, can be agree on that I'm right and stop this pointless discussion
> :-)

Admit it: I'm right,


        Stefan

Re: Building Emacs on MSYS2 (was: Build failure for Emacs master)

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Thu, 14 Apr 2016 17:58:09 +0200
> 
> Currently this is the code for configuring the build of emacs-git on
> MSYS2 MinGW-packages:
> 
> build() {
>   [[ -d "${srcdir}/build-${MINGW_CHOST}" ]] && rm -rf "${srcdir}/build-${MINGW_CHOST}"
>   mkdir -p "${srcdir}/build-${MINGW_CHOST}"
>   cd "build-${MINGW_CHOST}"
> 
>   local with_wide_int='no'
> 
>   if test "${CARCH}" == 'x86_64'; then
>     with_wide_int='yes'
>   fi
> 
>   CPPFLAGS="-DNDEBUG -isystem ${MINGW_PREFIX}/include"
>   CFLAGS="-pipe -O3 -fomit-frame-pointer -funroll-loops"
>   LDFLAGS="-s -Wl,-s"
>   "${srcdir}/${_realname}/configure" \
>     --prefix="${MINGW_PREFIX}" \
>     --build="${MINGW_CHOST}" \
>     --with-wide-int="${with_wide_int}" \
>     --with-sound="yes" \
>     --with-file-notification="yes" \
>     --without-gpm \
>     --without-gconf \
>     --without-gsettings \
>     --without-selinux
> 
>   make
> }
> 
> Apart from setting with_wide_int and adding some (unnecessary, IMHO)
> optimization options, it looks like a fairly standard configuration.

Actually, I see nothing standard at all here.  I see no reason for
_any_ of the options, neither in CPPFLAGS nor in CFLAGS.  They are all
potential source of trouble for the uninitiated.  (If there are real
reasons for them, they must be bugs that should be reported and fixed
upstream.)  And why "-s" in LDFLAGS? that's just hostile to
developers, since no user can possibly provide any details about any
bugs.  (Of course, this is in line with -fomit-frame-pointer, which is
also a killer for any attempts to debug.)  And all the switches to
'configure', with the single exception of --prefix (assuming that it
has the correct /d/foo/bar form and points to the MinGW directory, not
to MSYS2 directory) are simply clutter, they are no-ops at best and
potential trouble at worst.

Bottom line, I really don't like this script.

> The package contains two patches too. This one that looks like trying to
> avoid a link problem:
> 
> 
> --- src/image.c.orig	2014-10-15 14:18:29.088716000 +0200
> +++ src/image.c	2014-10-15 15:54:12.407522800 +0200
> @@ -7862,7 +7862,7 @@
>    };
>  
>  #if defined HAVE_NTGUI && defined WINDOWSNT
> -static bool init_imagemagick_functions (void);
> +#define init_imagemagick_functions NULL
>  #else
>  #define init_imagemagick_functions NULL
>  #endif

What link problem is that?  Why those who use the official procedure
never have or report it?

> and this one that seems related to locating the source directory:
> 
> --- src/lread.c.orig	2014-11-04 20:29:22.129549000 +0100
> +++ src/lread.c	2014-11-04 22:33:07.346414100 +0100
> @@ -4351,6 +4351,12 @@
>              } /* Vinstallation_directory != Vsource_directory */
>  
>          } /* if Vinstallation_directory */
> +      else
> +        {
> +          Vsource_directory
> +            = Fexpand_file_name (build_string ("../"),
> +                                 Fcar (decode_env_path (0, PATH_DATA, 0)));
> +        }
>      }
>    else                          /* !initialized */
>      {

What problem does this try to fix, and why?

> I can change the emacs-git package for not adding those optimization
> options and not setting with-wide-int. Do you think that's enough?

I think this should simply invoke the commands shown in
nt/INSTALL.W64, and that's it.  IOW, it should run the official
procedure, and if there are reasons to change that, those reasons
should be reported and discussed.

> > Failing that, please consider
> > switching back to what the project recommends, what is documented in
> > the relevant INSTALL documents.
> 
> Because MSYS2 does not distribute binaries for emacs-git, I guess that
> the number of people who currently uses the PKGBUILD shown above is very
> small.

I have similar issues with the MSYS2 procedure for building official
releases.

Thanks.

Re: Building Emacs on MSYS2

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> Apart from setting with_wide_int and adding some (unnecessary, IMHO)
>> optimization options, it looks like a fairly standard configuration.
>
> Actually, I see nothing standard at all here.  I see no reason for
> _any_ of the options, neither in CPPFLAGS nor in CFLAGS.  They are all
> potential source of trouble for the uninitiated.  (If there are real
> reasons for them, they must be bugs that should be reported and fixed
> upstream.)  And why "-s" in LDFLAGS? that's just hostile to
> developers, since no user can possibly provide any details about any
> bugs.  (Of course, this is in line with -fomit-frame-pointer, which is
> also a killer for any attempts to debug.)

The MSYS2 packages (as well as the Arch Linux packages which serves as
models) are intended for use on production, not for debugging.

OTOH, PKGBUILDs are supposed to have code for choosing debug builds if
the user passes certain flags to the build tool. This specific PKGBUILD
lacks that code. Adding it is not difficult.

> And all the switches to
> 'configure', with the single exception of --prefix (assuming that it
> has the correct /d/foo/bar form and points to the MinGW directory, not
> to MSYS2 directory)

Of course, the MinGW hierarchy is used.

> are simply clutter, they are no-ops at best and
> potential trouble at worst.

I guess those came from the Arch Linux package that served as model.

OTOH is a good practice to explicitly state "I have this feature so use
it". The MSYS2 build procedure already makes sure that the dependencies
are installed, so if something is not properly detected by the configure
script, it is a good thing to error out. Most of the listed options on
that PKGBUILD are not related to this, though.

> Bottom line, I really don't like this script.
>
>> The package contains two patches too. This one that looks like trying to
>> avoid a link problem:
>> 
>> 
>> --- src/image.c.orig	2014-10-15 14:18:29.088716000 +0200
>> +++ src/image.c	2014-10-15 15:54:12.407522800 +0200
>> @@ -7862,7 +7862,7 @@
>>    };
>>  
>>  #if defined HAVE_NTGUI && defined WINDOWSNT
>> -static bool init_imagemagick_functions (void);
>> +#define init_imagemagick_functions NULL
>>  #else
>>  #define init_imagemagick_functions NULL
>>  #endif
>
> What link problem is that?  Why those who use the official procedure
> never have or report it?

I have no idea. Probably imagemagick, as built and distributed by MSYS2,
does not have or need that feature.

>> and this one that seems related to locating the source directory:
>> 
>> --- src/lread.c.orig	2014-11-04 20:29:22.129549000 +0100
>> +++ src/lread.c	2014-11-04 22:33:07.346414100 +0100
>> @@ -4351,6 +4351,12 @@
>>              } /* Vinstallation_directory != Vsource_directory */
>>  
>>          } /* if Vinstallation_directory */
>> +      else
>> +        {
>> +          Vsource_directory
>> +            = Fexpand_file_name (build_string ("../"),
>> +                                 Fcar (decode_env_path (0, PATH_DATA, 0)));
>> +        }
>>      }
>>    else                          /* !initialized */
>>      {
>
> What problem does this try to fix, and why?

Just guessing: the post-build install directory is not the same as the
directory used for running Emacs (after running `make install', the
resulting files are packaged, stored and distributed by the project;
then users unpackage in their own installs; it's like doing `make
install' and the moving the install directory elsewhere.)

>> I can change the emacs-git package for not adding those optimization
>> options and not setting with-wide-int. Do you think that's enough?
>
> I think this should simply invoke the commands shown in
> nt/INSTALL.W64, and that's it.  IOW, it should run the official
> procedure, and if there are reasons to change that, those reasons
> should be reported and discussed.

Some modifications may be unavoidable due to how MSYS2 works (see above)
but I agree with the sentiment.

>> > Failing that, please consider
>> > switching back to what the project recommends, what is documented in
>> > the relevant INSTALL documents.
>> 
>> Because MSYS2 does not distribute binaries for emacs-git, I guess that
>> the number of people who currently uses the PKGBUILD shown above is very
>> small.
>
> I have similar issues with the MSYS2 procedure for building official
> releases.

The idea is to put emacs-git on good shape and, when emacs 25 is
released, port the modifications to the stable MSYS emacs PKGBUILD.

Re: Building Emacs on MSYS2

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Thu, 14 Apr 2016 18:43:08 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Apart from setting with_wide_int and adding some (unnecessary, IMHO)
> >> optimization options, it looks like a fairly standard configuration.
> >
> > Actually, I see nothing standard at all here.  I see no reason for
> > _any_ of the options, neither in CPPFLAGS nor in CFLAGS.  They are all
> > potential source of trouble for the uninitiated.  (If there are real
> > reasons for them, they must be bugs that should be reported and fixed
> > upstream.)  And why "-s" in LDFLAGS? that's just hostile to
> > developers, since no user can possibly provide any details about any
> > bugs.  (Of course, this is in line with -fomit-frame-pointer, which is
> > also a killer for any attempts to debug.)
> 
> The MSYS2 packages (as well as the Arch Linux packages which serves as
> models) are intended for use on production, not for debugging.

We are talking about building the development snapshots.  It is IMO
silly to build those without debugging info.

> OTOH, PKGBUILDs are supposed to have code for choosing debug builds if
> the user passes certain flags to the build tool. This specific PKGBUILD
> lacks that code. Adding it is not difficult.

Development snapshots are best compiled with debugging info (and with
"--enable-checking" as well, btw).

> > are simply clutter, they are no-ops at best and
> > potential trouble at worst.
> 
> I guess those came from the Arch Linux package that served as model.

I wonder why someone might think that copying configure-time option
from a totally different OS could be a good idea.  Why not ask here
instead?

> OTOH is a good practice to explicitly state "I have this feature so use
> it".

Emacs can do that without these switches, see
system-configuration-features.

> The MSYS2 build procedure already makes sure that the dependencies
> are installed, so if something is not properly detected by the configure
> script, it is a good thing to error out. Most of the listed options on
> that PKGBUILD are not related to this, though.

None of them are related to this, actually.

> >> --- src/lread.c.orig	2014-11-04 20:29:22.129549000 +0100
> >> +++ src/lread.c	2014-11-04 22:33:07.346414100 +0100
> >> @@ -4351,6 +4351,12 @@
> >>              } /* Vinstallation_directory != Vsource_directory */
> >>  
> >>          } /* if Vinstallation_directory */
> >> +      else
> >> +        {
> >> +          Vsource_directory
> >> +            = Fexpand_file_name (build_string ("../"),
> >> +                                 Fcar (decode_env_path (0, PATH_DATA, 0)));
> >> +        }
> >>      }
> >>    else                          /* !initialized */
> >>      {
> >
> > What problem does this try to fix, and why?
> 
> Just guessing: the post-build install directory is not the same as the
> directory used for running Emacs (after running `make install', the
> resulting files are packaged, stored and distributed by the project;
> then users unpackage in their own installs; it's like doing `make
> install' and the moving the install directory elsewhere.)

This kind of problem (if it indeed is it) should be fixed in
site-init.el files, not by patching sources.

> >> > Failing that, please consider
> >> > switching back to what the project recommends, what is documented in
> >> > the relevant INSTALL documents.
> >> 
> >> Because MSYS2 does not distribute binaries for emacs-git, I guess that
> >> the number of people who currently uses the PKGBUILD shown above is very
> >> small.
> >
> > I have similar issues with the MSYS2 procedure for building official
> > releases.
> 
> The idea is to put emacs-git on good shape and, when emacs 25 is
> released, port the modifications to the stable MSYS emacs PKGBUILD.

That's not what I meant, I meant the argument about the small number
of people who use this script.  I don't think it's small, because the
same problems are in building the official releases.

Re: Build failure for Emacs master

[Angelo Graziosi]

Just for completeness...

I have seen you have committed the patch.. Using emacs-master.tar.gz as 
source, the result is

$ ./autogen.sh all
Checking whether you have the necessary tools...
(Read INSTALL.REPO for more details on building Emacs)
Checking for autoconf (need at least version 2.65)...
ok
Checking for automake (need at least version 1.11)...
ok
Your system has the required tools.
Running 'autoreconf -fi -I m4' ...
configure.ac:796: installing 'build-aux/ar-lib'
configure.ac:139: installing 'build-aux/config.guess'
configure.ac:139: installing 'build-aux/config.sub'
configure.ac:136: installing 'build-aux/install-sh'
configure.ac:136: installing 'build-aux/missing'
lib/Makefile.am: installing 'build-aux/depcomp'
You can now run './configure'.

which should be Ok. Right?

  Angelo

Il 14/04/2016 03:31, Paul Eggert ha scritto:

> './autogen.sh all' can be adjusted to work for your use case. I
> installed the attached patch, which attempts to do that. Please give it
> a try.

Re: Build failure for Emacs master

[Paul Eggert]

On 04/14/2016 09:36 AM, Angelo Graziosi wrote:
>
> which should be Ok. Right?
Yes, looks good, thanks.

Re: Build failure for Emacs master

[Stefan Monnier]

>> You can now run './autogen.sh git'.
>> Really we need to run './autogen.sh git'?
> Yes you need to, and yes it is silly.

Does he really need to?  I thought the whole point is that it's
optional, and not desired by everyone.  IOW, he might like to do that,
but he shouldn't need to.


        Stefan

Re: Build failure for Emacs master

[John Wiegley]

>>>>> Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Does he really need to? I thought the whole point is that it's optional, and
> not desired by everyone. IOW, he might like to do that, but he shouldn't
> need to.

I thought that it should always be optional as well.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: Build failure for Emacs master

[Paul Eggert]

On 04/13/2016 06:38 PM, Stefan Monnier wrote:
>>> You can now run './autogen.sh git'.
>>> >>Really we need to run './autogen.sh git'?
>> >Yes you need to, and yes it is silly.
> Does he really need to?  I thought the whole point is that it's
> optional, and not desired by everyone.

You're right, it's optional (though it is a good idea unless one knows 
Git well). And for his particular case, where he didn't have a .git 
subdirectory, it shouldn't be done at all. Anyway, this should be fixed now.

Re: Build failure for Emacs master

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Angelo Graziosi <angelo.graziosi@alice.it>
> Date: Wed, 24 Feb 2016 10:36:13 +0100
> 
> Il 24/02/2016 04:42, Eli Zaretskii ha scritto:
> >> From: Angelo Graziosi <angelo.graziosi@alice.it>
> >> Date: Tue, 23 Feb 2016 23:11:42 +0100
> >>
> >> Just for the record, master fail to build on MSYS2/MinGW64:
> >
> > It doesn't fail for me.
> >
> >> Loading faces...
> >> Loading button...
> >> Loading loaddefs.el (source)...
> >> End of file during parsing:
> >> c:/msys64/tmp/mingw-w64-emacs-git/src/emacs/lisp/loaddefs.el
> >
> > Please look at your loaddefs.el, and tell if you see something strange
> > there, like null bytes.
> >
> 
> Hmm.. I removed the Emacs tree and recreated from scratch, then retried 
> the build.. Now it fails on another loaddefs, org-loaddefs.el. I have 
> seen the same failure a week ago or so and just retrying worked...
> 
> I wonder if this could be related to the "make -j3" (two core) or some 
> other thing in MSYS2..
> 
> Let's see what happen with another try..

Please also look at these files and try to figure out what's wrong
with them.  I think there's a subtle bug that hits from time to time,
and it's important for us to collect as much info about it as
possible.

Thanks.