Building master fails on Windows 10 when testing etc/NEWS file

Building master fails on Windows 10 when testing etc/NEWS file

[martin rudalics]

With master living in c:/emacs/trunk/ building Emacs in
c:/emacs/trunk/non-64/ currently fails here on Windows 10 as

rm -f bootstrap-emacs.pdmp
./temacs --batch  -l loadup --temacs=pbootstrap
emacs: Testing file: Permission denied, c:/emacs/trunk/non-64/share/emacs/27.0.50/etc/NEWS
make[1]: *** [Makefile:817: bootstrap-emacs.pdmp] Fehler 1
make[1]: Verzeichnis „/c/emacs/trunk/non-64/src“ wird verlassen
make: *** [Makefile:424: src] Fehler 2

martin

Re: Building master fails on Windows 10 when testing etc/NEWS file

[Paul Eggert]

On 9/18/19 12:45 AM, martin rudalics wrote:

> ./temacs --batch  -l loadup --temacs=pbootstrap
> emacs: Testing file: Permission denied, 
> c:/emacs/trunk/non-64/share/emacs/27.0.50/etc/NEWS

Thanks for reporting that. I installed what I hope is a patch on master; please 
give it a try.

Re: Building master fails on Windows 10 when testing etc/NEWS file

[Richard Copley]

[-- Attachment #1: Type: text/plain, Size: 1449 bytes --]

On Wed, 18 Sep 2019 at 11:08, Paul Eggert <eggert@cs.ucla.edu> wrote:

> On 9/18/19 12:45 AM, martin rudalics wrote:
>
> > ./temacs --batch  -l loadup --temacs=pbootstrap
> > emacs: Testing file: Permission denied,
> > c:/emacs/trunk/non-64/share/emacs/27.0.50/etc/NEWS
>
> Thanks for reporting that. I installed what I hope is a patch on master;
> please
> give it a try.
>

Thanks, it gets a little further now.

rm -f bootstrap-emacs.pdmp
./temacs --batch  -l loadup --temacs=pbootstrap
Loading loadup.el (source)...
dump mode: pbootstrap
Using load-path (c:/projects/emacs/lisp c:/projects/emacs/lisp/emacs-lisp
c:/projects/emacs/lisp/progmodes c:/projects/emacs/lisp/language
c:/projects/emacs/lisp/international c:/projects/emacs/lisp/textmodes
c:/projects/emacs/lisp/vc)
Loading emacs-lisp/byte-run...
Loading emacs-lisp/backquote...
Loading subr...
Loading version...
Loading widget...
Loading custom...
Loading emacs-lisp/map-ynp...
Loading international/mule...
Loading international/mule-conf...
Loading env...
Loading format...
Loading bindings...
Loading window...
Loading files...
Loading emacs-lisp/macroexp...
Loading cus-face...
Loading faces...
Loading button...
Loading c:/projects/emacs/lisp/loaddefs.el (source)...
Loading c:/projects/emacs/lisp/ldefs-boot.el (source)...
Testing file: Permission denied, c:/var/mail
make[1]: *** [Makefile:817: bootstrap-emacs.pdmp] Error 127
make[1]: Leaving directory '/c/projects/emacs/src'

[-- Attachment #2: Type: text/html, Size: 1914 bytes --]

Re: Building master fails on Windows 10 when testing etc/NEWS file

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 363 bytes --]

(Sorry for duplicating this, which I also replied to the patch e-mail)

Now it fails here:

Loading d:/Devel/emacs/repo/trunk/lisp/loaddefs.el (source)...
Loading d:/Devel/emacs/repo/trunk/lisp/ldefs-boot.el (source)...
Testing file: Permission denied, d:/var/mail
make[1]: *** [Makefile:817: bootstrap-emacs.pdmp] Error 127
make: *** [Makefile:424: src] Error 2

[-- Attachment #2: Type: text/html, Size: 421 bytes --]

Re: Building master fails on Windows 10 when testing etc/NEWS file

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 270 bytes --]

On 9/18/19 3:33 AM, Richard Copley wrote:

> Testing file: Permission denied, c:/var/mail

Oh, my. Is the MS-Windows tradition to pretend that permission is denied to 
every missing directory? :-)

Anyway, I installed the attached patch, which I hope fixes the problem.

[-- Attachment #2: 0001-Be-less-picky-about-EACCES-in-file-test-predicates.patch --]
[-- Type: text/x-patch, Size: 1907 bytes --]

From efa4edfadb3d8ac051ac21ccb4238f3eadb091dd Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Wed, 18 Sep 2019 04:21:19 -0700
Subject: [PATCH] Be less picky about EACCES in file test predicates

Problem reported by Tino Calancha (Bug#37445) and others.
* src/fileio.c (PICKY_EACCES): New constant, false by default.
(file_test_errno): Ignore EACCES if not picky.
(check_file_access): Investigate EACCES problems further
if picky.
---
 src/fileio.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/src/fileio.c b/src/fileio.c
index 0977516f01..58bc6b7ee8 100644
--- a/src/fileio.c
+++ b/src/fileio.c
@@ -253,9 +253,23 @@ file_attribute_errno (Lisp_Object file, int err)
   return file_metadata_errno ("Getting attributes", file, err);
 }
 
+/* In theory, EACCES errors for predicates like file-readable-p should
+   be checked further because they may be problems with an ancestor
+   directory instead of with the file itself, which means that we
+   don't have reliable info about the requested file.  In practice,
+   though, such errors are common enough that signaling them can be
+   annoying even if the errors are real (e.g., Bug#37445).  So return
+   nil for EACCES unless compiling with -DPICKY_EACCES, which is off
+   by default.  */
+#ifndef PICKY_EACCES
+enum { PICKY_EACCES = false };
+#endif
+
 static Lisp_Object
 file_test_errno (Lisp_Object file, int err)
 {
+  if (!PICKY_EACCES && err == EACCES)
+    return Qnil;
   return file_metadata_errno ("Testing file", file, err);
 }
 
@@ -2745,7 +2759,7 @@ check_file_access (Lisp_Object file, Lisp_Object operation, int amode)
     return Qt;
   int err = errno;
   if (err == EROFS || err == ETXTBSY
-      || (err == EACCES && amode != F_OK
+      || (PICKY_EACCES && err == EACCES && amode != F_OK
 	  && file_access_p (encoded_file, F_OK)))
     {
       errno = err;
-- 
2.17.1

Re: Building master fails on Windows 10 when testing etc/NEWS file

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 1084 bytes --]

Build finishes now.

Unexpected things:

Dumping under the name bootstrap-emacs.pdmp
dumping fingerprint:
2d6b0b41b332e60fca44efc80358845a5516e552d909e0c752185284c5c43268
Dump complete
Byte counts: header=80 hot=7843920 discardable=129360 cold=4484888
Reloc counts: hot=420352 discardable=4896
  ELC      ../lisp/replace.elc
Can't find D:\Devel\emacs\repo\trunk\src\bootstrap-emacs.exe to refresh
preloaded Lisp files

the build follows, and then

Loading leim/leim-list.el (source)...
Waiting for git...
Error running git rev-parse: (file-error "Searching for program"
"Permission denied" "git")
Waiting for git...
Error running git rev-parse --abbrev-ref: (file-error "Searching for
program" "Permission denied" "git")




On Wed, Sep 18, 2019 at 1:26 PM Paul Eggert <eggert@cs.ucla.edu> wrote:

> On 9/18/19 3:33 AM, Richard Copley wrote:
>
> > Testing file: Permission denied, c:/var/mail
>
> Oh, my. Is the MS-Windows tradition to pretend that permission is denied
> to
> every missing directory? :-)
>
> Anyway, I installed the attached patch, which I hope fixes the problem.
>

[-- Attachment #2: Type: text/html, Size: 1622 bytes --]

Re: Building master fails on Windows 10 when testing etc/NEWS file

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 724 bytes --]

On 9/18/19 4:36 AM, Juanma Barranquero wrote:

>    ELC      ../lisp/replace.elc
> Can't find D:\Devel\emacs\repo\trunk\src\bootstrap-emacs.exe to refresh
> preloaded Lisp files

Thanks, I installed the attached to try to fix that.

> Loading leim/leim-list.el (source)...
> Waiting for git...
> Error running git rev-parse: (file-error "Searching for program"
> "Permission denied" "git")
> Waiting for git...
> Error running git rev-parse --abbrev-ref: (file-error "Searching for
> program" "Permission denied" "git")

Is there a 'git' program in your PATH? What's the output of the following shell 
script (or equivalent) on your host?

IFS=':'
for dir in $PATH; do
   ls -ld "$dir" "$dir/git"
   echo =============
done

[-- Attachment #2: 0001-Be-less-picky-about-EACCES-in-file-test-predicates.patch --]
[-- Type: text/x-patch, Size: 1907 bytes --]

From efa4edfadb3d8ac051ac21ccb4238f3eadb091dd Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Wed, 18 Sep 2019 04:21:19 -0700
Subject: [PATCH] Be less picky about EACCES in file test predicates

Problem reported by Tino Calancha (Bug#37445) and others.
* src/fileio.c (PICKY_EACCES): New constant, false by default.
(file_test_errno): Ignore EACCES if not picky.
(check_file_access): Investigate EACCES problems further
if picky.
---
 src/fileio.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/src/fileio.c b/src/fileio.c
index 0977516f01..58bc6b7ee8 100644
--- a/src/fileio.c
+++ b/src/fileio.c
@@ -253,9 +253,23 @@ file_attribute_errno (Lisp_Object file, int err)
   return file_metadata_errno ("Getting attributes", file, err);
 }
 
+/* In theory, EACCES errors for predicates like file-readable-p should
+   be checked further because they may be problems with an ancestor
+   directory instead of with the file itself, which means that we
+   don't have reliable info about the requested file.  In practice,
+   though, such errors are common enough that signaling them can be
+   annoying even if the errors are real (e.g., Bug#37445).  So return
+   nil for EACCES unless compiling with -DPICKY_EACCES, which is off
+   by default.  */
+#ifndef PICKY_EACCES
+enum { PICKY_EACCES = false };
+#endif
+
 static Lisp_Object
 file_test_errno (Lisp_Object file, int err)
 {
+  if (!PICKY_EACCES && err == EACCES)
+    return Qnil;
   return file_metadata_errno ("Testing file", file, err);
 }
 
@@ -2745,7 +2759,7 @@ check_file_access (Lisp_Object file, Lisp_Object operation, int amode)
     return Qt;
   int err = errno;
   if (err == EROFS || err == ETXTBSY
-      || (err == EACCES && amode != F_OK
+      || (PICKY_EACCES && err == EACCES && amode != F_OK
 	  && file_access_p (encoded_file, F_OK)))
     {
       errno = err;
-- 
2.17.1

Re: Building master fails on Windows 10 when testing etc/NEWS file

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 2638 bytes --]

On Wed, Sep 18, 2019 at 2:23 PM Paul Eggert <eggert@cs.ucla.edu> wrote:

> Thanks, I installed the attached to try to fix that.

Thanks.

> Is there a 'git' program in your PATH?

Yes.

$ path
/c/bin64/Git/cmd
/mingw64/bin
/usr/local/bin
/usr/bin
/bin
/c/Windows/System32
/c/Windows
/c/Windows/System32/Wbem
/c/Windows/System32/WindowsPowerShell/v1.0/
/usr/bin/site_perl
/usr/bin/vendor_perl
/usr/bin/core_perl

$ ls -l /c/bin64/Git/cmd/
total 384
-rwxr-xr-x 1 Juanma Juanma 148768 Feb 26  2019 git-gui.exe
-rwxr-xr-x 2 Juanma Juanma  40728 Feb 26  2019 git-lfs.exe
-rwxr-xr-x 2 Juanma Juanma  40728 Feb 26  2019 git.exe
-rwxr-xr-x 1 Juanma Juanma 148760 Feb 26  2019 gitk.exe
-rw-r--r-- 1 Juanma Juanma   3022 Feb 26  2019 start-ssh-agent.cmd
-rw-r--r-- 1 Juanma Juanma   2723 Feb 26  2019 start-ssh-pageant.cmd


> What's the output of the following shell
> script (or equivalent) on your host?

$ ./paul.sh
drwxr-xr-x 1 Juanma Juanma     0 May 31 02:59 /c/bin64/Git/cmd
-rwxr-xr-x 2 Juanma Juanma 40728 Feb 26  2019 /c/bin64/Git/cmd/git
=============
ls: cannot access '/mingw64/bin/git': No such file or directory
drwxr-xr-x 1 Juanma Juanma 0 Sep 17 01:54 /mingw64/bin
=============
ls: cannot access '/usr/local/bin': No such file or directory
ls: cannot access '/usr/local/bin/git': No such file or directory
=============
ls: cannot access '/usr/bin/git': No such file or directory
drwxr-xr-x 1 Juanma Juanma 0 Sep 16 12:17 /usr/bin
=============
ls: cannot access '/bin/git': No such file or directory
drwxr-xr-x 1 Juanma Juanma 0 Sep 16 12:17 /bin
=============
ls: cannot access '/c/Windows/System32/git': No such file or directory
drwxr-xr-x 1 Juanma Juanma 0 Sep 12 22:04 /c/Windows/System32
=============
ls: cannot access '/c/Windows/git': No such file or directory
drwxr-xr-x 1 Juanma Juanma 0 Sep 12 01:07 /c/Windows
=============
ls: cannot access '/c/Windows/System32/Wbem/git': No such file or directory
drwxr-xr-x 1 Juanma Juanma 0 Sep 12 01:07 /c/Windows/System32/Wbem
=============
ls: cannot access '/c/Windows/System32/WindowsPowerShell/v1.0//git': No
such file or directory
drwxr-xr-x 1 Juanma Juanma 0 Jun  6 03:09
/c/Windows/System32/WindowsPowerShell/v1.0/
=============
ls: cannot access '/usr/bin/site_perl/git': No such file or directory
drwxr-xr-x 1 Juanma Juanma 0 Apr 25 09:29 /usr/bin/site_perl
=============
ls: cannot access '/usr/bin/vendor_perl/git': No such file or directory
drwxr-xr-x 1 Juanma Juanma 0 Jun  2 19:13 /usr/bin/vendor_perl
=============
ls: cannot access '/usr/bin/core_perl/git': No such file or directory
drwxr-xr-x 1 Juanma Juanma 0 Sep 16 12:16 /usr/bin/core_perl
=============

[-- Attachment #2: Type: text/html, Size: 3188 bytes --]

Re: Building master fails on Windows 10 when testing etc/NEWS file

[Eli Zaretskii]

> From: Juanma Barranquero <lekktu@gmail.com>
> Date: Wed, 18 Sep 2019 13:36:22 +0200
> Cc: Richard Copley <rcopley@gmail.com>, martin rudalics <rudalics@gmx.at>,
>  emacs-devel <emacs-devel@gnu.org>
> 
>   ELC      ../lisp/replace.elc
> Can't find D:\Devel\emacs\repo\trunk\src\bootstrap-emacs.exe to refresh preloaded Lisp files
> 
> the build follows, and then
> 
> Loading leim/leim-list.el (source)...
> Waiting for git...
> Error running git rev-parse: (file-error "Searching for program" "Permission denied" "git")
> Waiting for git...
> Error running git rev-parse --abbrev-ref: (file-error "Searching for program" "Permission denied" "git")

Should be fixed now.  The problem was that openp relied on an
undocumented kludge in file_directory_p regarding the value of errno
it returns.

Re: Building master fails on Windows 10 when testing etc/NEWS file

[Eli Zaretskii]

> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Wed, 18 Sep 2019 04:26:10 -0700
> Cc: Juanma Barranquero <lekktu@gmail.com>, martin rudalics <rudalics@gmx.at>,
>  emacs-devel <emacs-devel@gnu.org>
> 
> Is the MS-Windows tradition to pretend that permission is denied to
> every missing directory? :-)

Yes.

Re: Building master fails on Windows 10 when testing etc/NEWS file

[Eli Zaretskii]

> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Wed, 18 Sep 2019 05:23:20 -0700
> Cc: Richard Copley <rcopley@gmail.com>, martin rudalics <rudalics@gmx.at>,
>  emacs-devel <emacs-devel@gnu.org>
> 
> >    ELC      ../lisp/replace.elc
> > Can't find D:\Devel\emacs\repo\trunk\src\bootstrap-emacs.exe to refresh
> > preloaded Lisp files
> 
> Thanks, I installed the attached to try to fix that.

It can't.  The problem is that openp expects file_directory_p to
return ENOTDIR for a file that exists, but is not a directory.  This
kludge was never documented, and so the DOS_NT implementation didn't
support it.  That totally confused the logic in openp.

Re: Building master fails on Windows 10 when testing etc/NEWS file

[Richard Copley]

[-- Attachment #1: Type: text/plain, Size: 1007 bytes --]

On Wed, 18 Sep 2019 at 13:44, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Paul Eggert <eggert@cs.ucla.edu>
> > Date: Wed, 18 Sep 2019 05:23:20 -0700
> > Cc: Richard Copley <rcopley@gmail.com>, martin rudalics <rudalics@gmx.at
> >,
> >  emacs-devel <emacs-devel@gnu.org>
> >
> > >    ELC      ../lisp/replace.elc
> > > Can't find D:\Devel\emacs\repo\trunk\src\bootstrap-emacs.exe to refresh
> > > preloaded Lisp files
> >
> > Thanks, I installed the attached to try to fix that.
>
> It can't.  The problem is that openp expects file_directory_p to
> return ENOTDIR for a file that exists, but is not a directory.  This
> kludge was never documented, and so the DOS_NT implementation didn't
> support it.  That totally confused the logic in openp.
>

Getting better ... Emacs compiles and installs just fine now, but doesn't
run (if lpackages are loaded during startup).

"emacs -Q" starts OK. Doing [M-x load-library RET package RET] fails with
"Opening stdio stream: No such file or directory, package".

[-- Attachment #2: Type: text/html, Size: 1746 bytes --]

Re: Building master fails on Windows 10 when testing etc/NEWS file

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 458 bytes --]

On Wed, Sep 18, 2019 at 4:27 PM Richard Copley <rcopley@gmail.com> wrote:

> Getting better ... Emacs compiles and installs just fine now, but doesn't
run (if lpackages are loaded during startup).
>
> "emacs -Q" starts OK. Doing [M-x load-library RET package RET] fails with
"Opening stdio stream: No such file or directory, package".

 I've bootstrapped Emacs and I don't see that problem.

emacs -Q
M-x load-library <RET> package <RET>

works as expected.

[-- Attachment #2: Type: text/html, Size: 772 bytes --]

Re: Building master fails on Windows 10 when testing etc/NEWS file

[Eli Zaretskii]

> From: Richard Copley <rcopley@gmail.com>
> Date: Wed, 18 Sep 2019 15:27:10 +0100
> Cc: Paul Eggert <eggert@cs.ucla.edu>, Juanma Barranquero <lekktu@gmail.com>, 
> 	martin rudalics <rudalics@gmx.at>, Emacs Development <emacs-devel@gnu.org>
> 
> "emacs -Q" starts OK. Doing [M-x load-library RET package RET] fails with "Opening stdio stream: No such
> file or directory, package".

Doesn't happen here.  Could it be some artifact of previous failed
builds?  What if you delete package.elc?

Re: Building master fails on Windows 10 when testing etc/NEWS file

[Eli Zaretskii]

> From: Juanma Barranquero <lekktu@gmail.com>
> Date: Wed, 18 Sep 2019 17:01:19 +0200
> Cc: martin rudalics <rudalics@gmx.at>, Eli Zaretskii <eliz@gnu.org>,
>  Paul Eggert <eggert@cs.ucla.edu>, Emacs Development <emacs-devel@gnu.org>
> 
> > "emacs -Q" starts OK. Doing [M-x load-library RET package RET] fails with "Opening stdio stream: No such
> file or directory, package".
> 
>  I've bootstrapped Emacs and I don't see that problem.
> 
> emacs -Q
> M-x load-library <RET> package <RET>
> 
> works as expected.

Richard said "installs", so maybe this problem only happens in an
installed Emacs?  Richard, do you see the same problem if you run
Emacs from its build tree?  To simulate an uninstalled Emacs, you
might need to rename the versioned subdirectory under the share/emacs/
directory where you install Emacs.

Re: Building master fails on Windows 10 when testing etc/NEWS file

[Richard Copley]

[-- Attachment #1: Type: text/plain, Size: 1487 bytes --]

On Wed, 18 Sep 2019 at 16:45, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Juanma Barranquero <lekktu@gmail.com>
> > Date: Wed, 18 Sep 2019 17:01:19 +0200
> > Cc: martin rudalics <rudalics@gmx.at>, Eli Zaretskii <eliz@gnu.org>,
> >  Paul Eggert <eggert@cs.ucla.edu>, Emacs Development <
> emacs-devel@gnu.org>
> >
> > > "emacs -Q" starts OK. Doing [M-x load-library RET package RET] fails
> with "Opening stdio stream: No such
> > file or directory, package".
> >
> >  I've bootstrapped Emacs and I don't see that problem.
> >
> > emacs -Q
> > M-x load-library <RET> package <RET>
> >
> > works as expected.
>
> Richard said "installs", so maybe this problem only happens in an
> installed Emacs?  Richard, do you see the same problem if you run
> Emacs from its build tree?  To simulate an uninstalled Emacs, you
> might need to rename the versioned subdirectory under the share/emacs/
> directory where you install Emacs.
>

That could be the difference. I usually install Emacs, despite the
inconvenience, because then emacs.exe lives alongside all the DLLs it will
need and I can set $PATH and `exec-path' to any convenient value for
subprocesses.

Emacs does launch correctly (at least, it displays a frame) if I run it
from the src directory (I didn't need to move the versioned directory
aside).

I have rebuilt Emacs from a clean repo, and the problem still exists. (If I
launch the installed version I get the message I mentioned, and no frame is
displayed, and Emacs exits.)

[-- Attachment #2: Type: text/html, Size: 2364 bytes --]

Re: Building master fails on Windows 10 when testing etc/NEWS file

[Eli Zaretskii]

> From: Richard Copley <rcopley@gmail.com>
> Date: Wed, 18 Sep 2019 18:00:39 +0100
> Cc: Juanma Barranquero <lekktu@gmail.com>, martin rudalics <rudalics@gmx.at>, Paul Eggert <eggert@cs.ucla.edu>, 
> 	Emacs Development <emacs-devel@gnu.org>
> 
>  Richard said "installs", so maybe this problem only happens in an
>  installed Emacs?  Richard, do you see the same problem if you run
>  Emacs from its build tree?  To simulate an uninstalled Emacs, you
>  might need to rename the versioned subdirectory under the share/emacs/
>  directory where you install Emacs.
> 
> That could be the difference. I usually install Emacs, despite the inconvenience, because then emacs.exe
> lives alongside all the DLLs it will need and I can set $PATH and `exec-path' to any convenient value for
> subprocesses.
> 
> Emacs does launch correctly (at least, it displays a frame) if I run it from the src directory (I didn't need to
> move the versioned directory aside).
> 
> I have rebuilt Emacs from a clean repo, and the problem still exists. (If I launch the installed version I get the
> message I mentioned, and no frame is displayed, and Emacs exits.)

Then please run Emacs under GDB, put a breakpoint on line 1437 of
lread.c (that's where that message comes from), and when it breaks,
show the C and Lisp backtraces.

Re: Building master fails on Windows 10 when testing etc/NEWS file

[Eli Zaretskii]

> Date: Wed, 18 Sep 2019 20:23:52 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: lekktu@gmail.com, eggert@cs.ucla.edu, emacs-devel@gnu.org, rudalics@gmx.at
> 
> Then please run Emacs under GDB, put a breakpoint on line 1437 of
> lread.c (that's where that message comes from), and when it breaks,
> show the C and Lisp backtraces.

Don't bother, I found the culprit.  It is this fragment from Fload:

          if (!load_prefer_newer && is_elc)
            {
              result = stat (SSDATA (efound), &s1);
	      int err = errno;
              if (result == 0)
                {
                  SSET (efound, SBYTES (efound) - 1, 0);
                  result = stat (SSDATA (efound), &s2);
		  err = errno;
                  SSET (efound, SBYTES (efound) - 1, 'c');
		  if (result != 0)                              <<<<<<<<<<<
		    found = Fsubstring (found, make_fixnum (0), <<<<<<<<<<<
					make_fixnum (-1));      <<<<<<<<<<<
                }

The last 3 lines were added as part of the "improved reporting of I/O
errors" changeset.  What it does is replace the .elc file name with
the .el filename if 'stat' fails when called on the .el file.  And in
installed Emacs we compress *.el files, so they are not there.

Paul, why was this snippet added?  It makes no sense to me: why should
we replace an existing .elc file name with a non-existing .el file
name?

Re: Building master fails on Windows 10 when testing etc/NEWS file

[Richard Copley]

[-- Attachment #1: Type: text/plain, Size: 2157 bytes --]

On Wed, 18 Sep 2019 at 19:33, Eli Zaretskii <eliz@gnu.org> wrote:

> > Date: Wed, 18 Sep 2019 20:23:52 +0300
> > From: Eli Zaretskii <eliz@gnu.org>
> > Cc: lekktu@gmail.com, eggert@cs.ucla.edu, emacs-devel@gnu.org,
> rudalics@gmx.at
> >
> > Then please run Emacs under GDB, put a breakpoint on line 1437 of
> > lread.c (that's where that message comes from), and when it breaks,
> > show the C and Lisp backtraces.
>
> Don't bother
>

:)


c:\projects\emacs>gdb --quiet --batch -x src/.gdbinit -ex "break
src/lread.c:1437" -ex run -ex "thread apply all bt" -ex quit --args
"c:\msys64\mingw64\bin\emacs.exe"
Warning: c:\projects\emacs/../lwlib: No such file or directory.
Breakpoint 1 at 0x40014af09: file emacs.c, line 375.
Breakpoint 2 at 0x4002b92e3: file lread.c, line 1437.
warning: FTH: (19220): *** Fault tolerant heap shim applied to current
process. This is usually due to previous crashes. ***
[New Thread 19220.0x4520]
[New Thread 19220.0x44c8]
[New Thread 19220.0x32ec]
[New Thread 19220.0x778]

Thread 1 hit Breakpoint 2, Fload (file=XIL(0x40123a4), noerror=XIL(0),
nomessage=XIL(0x30), nosuffix=XIL(0), must_suffix=XIL(0x30)) at lread.c:1437
1437            report_file_error ("Opening stdio stream", file);

Thread 5 (Thread 19220.0x778):
#0  0x00007ffd293ceae4 in ntdll!ZwDelayExecution () from
C:\WINDOWS\SYSTEM32\ntdll.dll
#1  0x00007ffd26881e41 in SleepEx () from C:\WINDOWS\System32\KernelBase.dll
#2  0x00000004003f026e in timer_loop (arg=0x400929120 <real_itimer>) at
w32proc.c:473
#3  0x00007ffd27e55bb4 in KERNEL32!BaseThreadInitThunk () from
C:\WINDOWS\System32\kernel32.dll
#4  0x00007ffd29389ee1 in ntdll!RtlUserThreadStart () from
C:\WINDOWS\SYSTEM32\ntdll.dll
#5  0x0000000000000000 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

Lisp Backtrace:
"package-activate-all" (0xbfe1c0)
"command-line" (0xbfed38)
"normal-top-level" (0xbff3a0)
Opening stdio stream: No such file or directory, package

Thread 4 (process 0):
The program being debugged exited while in a function called from GDB.
Evaluation of the expression containing the function
(backtrace_top) will be abandoned.

c:\projects\emacs>

[-- Attachment #2: Type: text/html, Size: 3016 bytes --]

Re: Building master fails on Windows 10 when testing etc/NEWS file

[Eli Zaretskii]

> Date: Wed, 18 Sep 2019 21:33:10 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: rudalics@gmx.at, emacs-devel@gnu.org
> 
> 		  if (result != 0)                              <<<<<<<<<<<
> 		    found = Fsubstring (found, make_fixnum (0), <<<<<<<<<<<
> 					make_fixnum (-1));      <<<<<<<<<<<
>                 }
> 
> The last 3 lines were added as part of the "improved reporting of I/O
> errors" changeset.  What it does is replace the .elc file name with
> the .el filename if 'stat' fails when called on the .el file.  And in
> installed Emacs we compress *.el files, so they are not there.
> 
> Paul, why was this snippet added?  It makes no sense to me: why should
> we replace an existing .elc file name with a non-existing .el file
> name?

I guess the answer is so that file_test_errno could report the
offending file name...  But the change failed to pay attention that
'found' is used further down.

So based on this hypothesis I fixed the code to not clobber the file
name found by openp.

Re: Building master fails on Windows 10 when testing etc/NEWS file

[Eli Zaretskii]

> From: Richard Copley <rcopley@gmail.com>
> Date: Wed, 18 Sep 2019 19:35:31 +0100
> Cc: Juanma Barranquero <lekktu@gmail.com>, Paul Eggert <eggert@cs.ucla.edu>,
>  martin rudalics <rudalics@gmx.at>, Emacs Development <emacs-devel@gnu.org>
> 
> Thread 1 hit Breakpoint 2, Fload (file=XIL(0x40123a4), noerror=XIL(0), nomessage=XIL(0x30), nosuffix=XIL
> (0), must_suffix=XIL(0x30)) at lread.c:1437
> 1437            report_file_error ("Opening stdio stream", file);
> 
> Thread 5 (Thread 19220.0x778):
> #0  0x00007ffd293ceae4 in ntdll!ZwDelayExecution () from C:\WINDOWS\SYSTEM32\ntdll.dll
> #1  0x00007ffd26881e41 in SleepEx () from C:\WINDOWS\System32\KernelBase.dll
> #2  0x00000004003f026e in timer_loop (arg=0x400929120 <real_itimer>) at w32proc.c:473
> #3  0x00007ffd27e55bb4 in KERNEL32!BaseThreadInitThunk () from C:\WINDOWS\System32\kernel32.dll
> #4  0x00007ffd29389ee1 in ntdll!RtlUserThreadStart () from C:\WINDOWS\SYSTEM32\ntdll.dll
> #5  0x0000000000000000 in ?? ()
> Backtrace stopped: previous frame inner to this frame (corrupt stack?)

Thanks, but this is the wrong thread...

Anyway, the problem should be fixed by now, please try the latest
master.

Re: Building master fails on Windows 10 when testing etc/NEWS file

[Richard Copley]

[-- Attachment #1: Type: text/plain, Size: 1546 bytes --]

On Wed, 18 Sep 2019 at 19:49, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Richard Copley <rcopley@gmail.com>
> > Date: Wed, 18 Sep 2019 19:35:31 +0100
> > Cc: Juanma Barranquero <lekktu@gmail.com>, Paul Eggert <
> eggert@cs.ucla.edu>,
> >  martin rudalics <rudalics@gmx.at>, Emacs Development <
> emacs-devel@gnu.org>
> >
> > Thread 1 hit Breakpoint 2, Fload (file=XIL(0x40123a4), noerror=XIL(0),
> nomessage=XIL(0x30), nosuffix=XIL
> > (0), must_suffix=XIL(0x30)) at lread.c:1437
> > 1437            report_file_error ("Opening stdio stream", file);
> >
> > Thread 5 (Thread 19220.0x778):
> > #0  0x00007ffd293ceae4 in ntdll!ZwDelayExecution () from
> C:\WINDOWS\SYSTEM32\ntdll.dll
> > #1  0x00007ffd26881e41 in SleepEx () from
> C:\WINDOWS\System32\KernelBase.dll
> > #2  0x00000004003f026e in timer_loop (arg=0x400929120 <real_itimer>) at
> w32proc.c:473
> > #3  0x00007ffd27e55bb4 in KERNEL32!BaseThreadInitThunk () from
> C:\WINDOWS\System32\kernel32.dll
> > #4  0x00007ffd29389ee1 in ntdll!RtlUserThreadStart () from
> C:\WINDOWS\SYSTEM32\ntdll.dll
> > #5  0x0000000000000000 in ?? ()
> > Backtrace stopped: previous frame inner to this frame (corrupt stack?)
>
> Thanks, but this is the wrong thread...
>

The rest is missing because of the part I was smiling about:

Lisp Backtrace:
"package-activate-all" (0xbfe1c0)
"command-line" (0xbfed38)
"normal-top-level" (0xbff3a0)
Opening stdio stream: No such file or directory, package


> Anyway, the problem should be fixed by now, please try the latest
> master.
>

Works for me. Thanks.

[-- Attachment #2: Type: text/html, Size: 2551 bytes --]

Re: Building master fails on Windows 10 when testing etc/NEWS file

[Paul Eggert]

On 9/18/19 11:47 AM, Eli Zaretskii wrote:
> I guess the answer is so that file_test_errno could report the
> offending file name...  But the change failed to pay attention that
> 'found' is used further down.
> 
> So based on this hypothesis I fixed the code to not clobber the file
> name found by openp.

Thanks, that looks right to me.

Re: Building master fails on Windows 10 when testing etc/NEWS file

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 505 bytes --]

On 9/18/19 5:37 AM, Eli Zaretskii wrote:
>> Is the MS-Windows tradition to pretend that permission is denied to
>> every missing directory?
> Yes.

OK, that explains why these problems cropped up on MS-Windows platforms even 
though I didn't see them on GNU/Linux.

Since defining PICKY_EACCES helped to find a configuration-related bug on 
GNU/Linux (Bug#37445) I would like to try it out on GNUish platforms, so I 
installed the attached. We can easily revert if it causes more problems than it 
cures.

[-- Attachment #2: 0001-Default-PICKY_ACCESS-to-false-on-non-MS.patch --]
[-- Type: text/x-patch, Size: 1362 bytes --]

From 30026cfe666e9647aeef73e26df5ffca2fa2c662 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Thu, 19 Sep 2019 00:19:11 -0700
Subject: [PATCH] Default PICKY_ACCESS to false on non-MS

* src/fileio.c (PICKY_EACCES) [!DOS_NT]: Default to false.
---
 src/fileio.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/src/fileio.c b/src/fileio.c
index 5337ea5c80..b2896c1fe1 100644
--- a/src/fileio.c
+++ b/src/fileio.c
@@ -257,12 +257,16 @@ file_attribute_errno (Lisp_Object file, int err)
    be checked further because they may be problems with an ancestor
    directory instead of with the file itself, which means that we
    don't have reliable info about the requested file.  In practice,
-   though, such errors are common enough that signaling them can be
-   annoying even if the errors are real (e.g., Bug#37445).  So return
-   nil for EACCES unless compiling with -DPICKY_EACCES, which is off
-   by default.  */
+   though, DOS_NT platforms set errno to EACCES for missing files like
+   "/var/mail", so signaling EACCES errors would be a mistake there.
+   So return nil for EACCES unless PICKY_EACCES, which is false by
+   default on DOS_NT.  */
 #ifndef PICKY_EACCES
+# ifdef DOS_NT
 enum { PICKY_EACCES = false };
+# else
+enum { PICKY_EACCES = true };
+# endif
 #endif
 
 Lisp_Object
-- 
2.17.1

Re: Building master fails on Windows 10 when testing etc/NEWS file

[martin rudalics]

 >> So based on this hypothesis I fixed the code to not clobber the file
 >> name found by openp.
 >
 > Thanks, that looks right to me.

Builds now here.  Thanks to all who participated in fixing this.

martin

Re: Building master fails on Windows 10 when testing etc/NEWS file

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 1017 bytes --]

Could this be related? It just started happening now.

$ make -C lisp custom-deps
make: Entering directory '/d/Devel/emacs/repo/trunk/lisp'
make[1]: Entering directory '/d/Devel/emacs/repo/trunk/lisp'
  GEN      cus-load.el
  INFO     Scanning files for custom...
  INFO     Scanning files for custom...10%
  INFO     Scanning files for custom...20%
  INFO     Scanning files for custom...30%
  INFO     Scanning files for custom...40%
  INFO     Scanning files for custom...50%
  INFO     Scanning files for custom...60%
  INFO     Scanning files for custom...70%
  INFO     Scanning files for custom...80%
  INFO     Scanning files for custom...90%
  INFO     Scanning files for custom...100%
  INFO     Scanning files for custom...done
  INFO     Generating ./cus-load.el...
Version must be a string
make[1]: *** [Makefile:152: cus-load.el] Error 127
make[1]: Leaving directory '/d/Devel/emacs/repo/trunk/lisp'
make: *** [Makefile:150: custom-deps] Error 2
make: Leaving directory '/d/Devel/emacs/repo/trunk/lisp'

[-- Attachment #2: Type: text/html, Size: 1205 bytes --]

Re: Building master fails on Windows 10 when testing etc/NEWS file

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 222 bytes --]

On Thu, Sep 19, 2019 at 11:53 AM Juanma Barranquero <lekktu@gmail.com>
wrote:
>
> Could this be related? It just started happening now.

Sorry for the noise. Seems to be caused
by  34f1035e878a06ad181ff7fc533cd1fa0a565847

[-- Attachment #2: Type: text/html, Size: 341 bytes --]