Wrong PATH in MSYS2/MINGW64 builds?

Wrong PATH in MSYS2/MINGW64 builds?

[Angelo Graziosi]

Trying to visit a bzip2 file (foo.txt.bz2) with 
mingw-w64-emacs-git-r121447.8a9ba4d-1 installed package, I got

  Uncompression program `bzip2' not found

So, trying

   (getenv "PATH")

from scratch buffer, prints:

"C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\"


I wonder if this is Windows-builds related of we can do a better build. 
BTW I HAVE bzip2 installed (both in MINGW64 and MSYS2 shells):

MINGW64 ~
$ type bzip2
bzip2 is /mingw64/bin/bzip2

MSYS ~
$ type bzip2
bzip2 is /usr/bin/bzip2


Ciao,
  Angelo.

Re: Wrong PATH in MSYS2/MINGW64 builds?

[Óscar Fuentes]

Angelo Graziosi <angelo.graziosi@alice.it> writes:

> Trying to visit a bzip2 file (foo.txt.bz2) with
> mingw-w64-emacs-git-r121447.8a9ba4d-1 installed package, I got
>
>  Uncompression program `bzip2' not found
>
> So, trying
>
>   (getenv "PATH")
>
> from scratch buffer, prints:
>
> "C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\"
>
>
> I wonder if this is Windows-builds related

What is Windows-builds?

> of we can do a better
> build. BTW I HAVE bzip2 installed (both in MINGW64 and MSYS2 shells):
>
> MINGW64 ~
> $ type bzip2
> bzip2 is /mingw64/bin/bzip2

Is your emacs.exe installed on the same directory as mingw64 bzip2.exe ?

> MSYS ~
> $ type bzip2
> bzip2 is /usr/bin/bzip2



>
> Ciao,
>  Angelo.

Re: Wrong PATH in MSYS2/MINGW64 builds?

[Eli Zaretskii]

> Date: Thu, 28 May 2015 15:56:07 +0200
> From: Angelo Graziosi <angelo.graziosi@alice.it>
> Cc: alexey.pawlow@gmail.com
> 
> Trying to visit a bzip2 file (foo.txt.bz2) with 
> mingw-w64-emacs-git-r121447.8a9ba4d-1 installed package, I got
> 
>   Uncompression program `bzip2' not found
> 
> So, trying
> 
>    (getenv "PATH")
> 
> from scratch buffer, prints:
> 
> "C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\"

In which of these directories do you have bzip2.exe?

> BTW I HAVE bzip2 installed (both in MINGW64 and MSYS2 shells):
> 
> MINGW64 ~
> $ type bzip2
> bzip2 is /mingw64/bin/bzip2
> 
> MSYS ~
> $ type bzip2
> bzip2 is /usr/bin/bzip2

This doesn't tell anything, since you obviously typed these commands
from some Unixy shell, and the resulting file name isn't related to
any of the directories shown in PATH above.  A native build of Emacs
doesn't know about all the "mounted" directories created by MSYS and
Cygwin, it searches only the native Windows PATH.  You should make
sure your bzip2.exe is found in one of those directories, or add its
directory to system-wide PATH before starting Emacs.

Re: Wrong PATH in MSYS2/MINGW64 builds?

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

[snip]

> You should make
> sure your bzip2.exe is found in one of those directories, or add its
> directory to system-wide PATH before starting Emacs.

I would expect from Emacs to be able to execute the applications that
reside on the same directory as emacs.exe.

Re: Wrong PATH in MSYS2/MINGW64 builds?

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Thu, 28 May 2015 17:36:04 +0200
> 
> I would expect from Emacs to be able to execute the applications that
> reside on the same directory as emacs.exe.

Why would you expect that?

Re: Wrong PATH in MSYS2/MINGW64 builds?

[Yuri Khan]

On Thu, May 28, 2015 at 10:37 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Óscar Fuentes <ofv@wanadoo.es>
>>
>> I would expect from Emacs to be able to execute the applications that
>> reside on the same directory as emacs.exe.
>
> Why would you expect that?

Because it's the default native behavior on Windows?

===
$ msdn CreateProcess
[...]
If the file name does not contain a directory path, the system
searches for the executable file in the following sequence:

1. The directory from which the application loaded.
2. The current directory for the parent process.
3. The 32-bit Windows system directory. Use the GetSystemDirectory(2)
function to get the path of this directory.
4. The 16-bit Windows system directory. There is no function that
obtains the path of this directory, but it is searched. The name of
this directory is System.
5. The Windows directory. Use the GetWindowsDirectory(2) function to
get the path of this directory.
6. The directories that are listed in the PATH environment variable.
===

https://msdn.microsoft.com/en-us/library/windows/desktop/ms682425%28v=vs.85%29.aspx

Re: Wrong PATH in MSYS2/MINGW64 builds?

[Eli Zaretskii]

> From: Yuri Khan <yuri.v.khan@gmail.com>
> Date: Thu, 28 May 2015 22:52:16 +0600
> Cc: Óscar Fuentes <ofv@wanadoo.es>, 
> 	Emacs developers <emacs-devel@gnu.org>
> 
> On Thu, May 28, 2015 at 10:37 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> >> From: Óscar Fuentes <ofv@wanadoo.es>
> >>
> >> I would expect from Emacs to be able to execute the applications that
> >> reside on the same directory as emacs.exe.
> >
> > Why would you expect that?
> 
> Because it's the default native behavior on Windows?
> 
> ===
> $ msdn CreateProcess

Why would you think Emacs should invoke CreateProcess with just
"bzip2" as the program name?

Please have a look at the implementation of call-process, and you will
see that Emacs searches for the program in platform-independent
application code, long before it gets to low-level Windows-specific
code (which just emulates Posix fork/exec).  By the time we get to
CreateProcess, the program file name is supposed to be already found
and converted to an absolute file name.

IOW, Emacs behaves consistently on all supported platforms, and tries
to keep OS-specific issues and tricks out.  In particular, Emacs
_never_ relies on the current directory of the Emacs process (that's
why it always calls expand-file-name or its equivalent, before passing
file names to low-level APIs).

Re: Wrong PATH in MSYS2/MINGW64 builds?

[Yuri Khan]

On Thu, May 28, 2015 at 11:09 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>> >> From: Óscar Fuentes <ofv@wanadoo.es>
>> >>
>> >> I would expect from Emacs to be able to execute the applications that
>> >> reside on the same directory as emacs.exe.
>> >
>> > Why would you expect that?
>>
>> Because it's the default native behavior on Windows?
>>
> Why would you think Emacs should invoke CreateProcess with just
> "bzip2" as the program name?

I'm not thinking what Emacs "should" or "should not". I'm just
pointing out that being able to run programs that reside in the same
directory as the calling process' executable (as well as several other
directories not necessarily listed on $PATH) is a reasonable
expectation for a Windows user.

Re: Wrong PATH in MSYS2/MINGW64 builds?

[Eli Zaretskii]

> From: Yuri Khan <yuri.v.khan@gmail.com>
> Date: Fri, 29 May 2015 08:05:46 +0600
> Cc: Óscar Fuentes <ofv@wanadoo.es>, 
> 	Emacs developers <emacs-devel@gnu.org>
> 
> On Thu, May 28, 2015 at 11:09 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> >> >> From: Óscar Fuentes <ofv@wanadoo.es>
> >> >>
> >> >> I would expect from Emacs to be able to execute the applications that
> >> >> reside on the same directory as emacs.exe.
> >> >
> >> > Why would you expect that?
> >>
> >> Because it's the default native behavior on Windows?
> >>
> > Why would you think Emacs should invoke CreateProcess with just
> > "bzip2" as the program name?
> 
> I'm not thinking what Emacs "should" or "should not". I'm just
> pointing out that being able to run programs that reside in the same
> directory as the calling process' executable (as well as several other
> directories not necessarily listed on $PATH) is a reasonable
> expectation for a Windows user.

This is a developers' forum, not a users' forum.  People here are
expected to have their expectations modified by their acquaintance
with how Emacs really works.  And since Emacs is originally not a
Windows program, there's no reason to expect it to behave like one in
this regard, since there are no similar features on Posix platforms.

Re: Wrong PATH in MSYS2/MINGW64 builds?

[Angelo Graziosi]

Il 28/05/2015 17:09, Eli Zaretskii ha scritto:
>> Date: Thu, 28 May 2015 15:56:07 +0200
>> From: Angelo Graziosi <angelo.graziosi@alice.it>
>> Cc: alexey.pawlow@gmail.com
>>
>> Trying to visit a bzip2 file (foo.txt.bz2) with
>> mingw-w64-emacs-git-r121447.8a9ba4d-1 installed package, I got
>>
>>    Uncompression program `bzip2' not found
>>
>> So, trying
>>
>>     (getenv "PATH")
>>
>> from scratch buffer, prints:
>>
>> "C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\"
>
> In which of these directories do you have bzip2.exe?
>
>> BTW I HAVE bzip2 installed (both in MINGW64 and MSYS2 shells):
>>
>> MINGW64 ~
>> $ type bzip2
>> bzip2 is /mingw64/bin/bzip2
>>
>> MSYS ~
>> $ type bzip2
>> bzip2 is /usr/bin/bzip2
>
> This doesn't tell anything, since you obviously typed these commands
> from some Unixy shell, and the resulting file name isn't related to
> any of the directories shown in PATH above.  A native build of Emacs
> doesn't know about all the "mounted" directories created by MSYS and
> Cygwin, it searches only the native Windows PATH.  You should make
> sure your bzip2.exe is found in one of those directories, or add its
> directory to system-wide PATH before starting Emacs.
>

OK. In part, I have just replied to Oscar...

The package is a Windows native build. I start it with a link on status 
bar or Start menu with:

   Target :  C:\msys64\mingw64\bin\runemacs.exe
   From   :  C:\msys64\mingw64\bin


On MSYS2/MINGW64 all executable are installed in C:\msys64\mingw64\bin. 
The above commands (type bzip2...) where only the short way to say that 
emacs.exe and bzip2.exe where installed in the same 
C:\msys64\mingw64\bin directory..

As I noticed in replaying to Oscar, getenv "PATH" does not show 
C:\msys64\mingw64\bin in the PATH. For this reason I am asking if this 
is, in some way, to be expected under Windows or if the MSYS2/MINGW64 
build should be still tuned..

 From your answer, it seems the first. I don't like the idea to change 
the system-wide PATH. Maybe someone has some idea about how to patch 
Emacs (on Windows) so that at least the directory where is emacs.exe is 
added to PATH.


Ciao,
  Angelo.

Re: Wrong PATH in MSYS2/MINGW64 builds?

[Eli Zaretskii]

> Date: Thu, 28 May 2015 17:41:45 +0200
> From: Angelo Graziosi <angelo.graziosi@alice.it>
> CC: emacs-devel@gnu.org, alexey.pawlow@gmail.com
> 
> The package is a Windows native build. I start it with a link on status 
> bar or Start menu with:
> 
>    Target :  C:\msys64\mingw64\bin\runemacs.exe
>    From   :  C:\msys64\mingw64\bin
> 
> 
> On MSYS2/MINGW64 all executable are installed in C:\msys64\mingw64\bin. 
> The above commands (type bzip2...) where only the short way to say that 
> emacs.exe and bzip2.exe where installed in the same 
> C:\msys64\mingw64\bin directory..
> 
> As I noticed in replaying to Oscar, getenv "PATH" does not show 
> C:\msys64\mingw64\bin in the PATH. For this reason I am asking if this 
> is, in some way, to be expected under Windows or if the MSYS2/MINGW64 
> build should be still tuned..

If bzip2.exe is not on PATH, I don't expect this to work.

>  From your answer, it seems the first. I don't like the idea to change 
> the system-wide PATH.

??? Isn't that what you would do on any other OS?  Also, bzip2 is
useful on its own right, unrelated to Emacs.

> Maybe someone has some idea about how to patch Emacs (on Windows) so
> that at least the directory where is emacs.exe is added to PATH.

I think your problem is with exec-path (which is derived from PATH),
not with PATH directly.

IME, it's a bad idea to have exec-path inside Emacs be different from
PATH outside Emacs, because then you'd have subtle differences between
Emacs looking for programs and programs it invokes (such as the shell,
for example) looking for programs.  So my advice is not to patch Emacs
that way.

Re: Wrong PATH in MSYS2/MINGW64 builds?

[Angelo Graziosi]

Il 28/05/2015 18:49, Eli Zaretskii ha scritto:
>
> I think your problem is with exec-path (which is derived from PATH),
> not with PATH directly.
>
> IME, it's a bad idea to have exec-path inside Emacs be different from
> PATH outside Emacs, because then you'd have subtle differences between
> Emacs looking for programs and programs it invokes (such as the shell,
> for example) looking for programs.  So my advice is not to patch Emacs
> that way.
>

Shouldn't the option

   --libexecdir="${MINGW_PREFIX}/bin"

(added to "configure") fix this issue (being 
MINGW_PREFIX=/c/msys64/mingw64)?

Ciao,
  Angelo.

Re: Wrong PATH in MSYS2/MINGW64 builds?

[Eli Zaretskii]

> Date: Thu, 28 May 2015 23:51:59 +0200
> From: Angelo Graziosi <angelo.graziosi@alice.it>
> CC: emacs-devel@gnu.org, alexey.pawlow@gmail.com
> 
> > I think your problem is with exec-path (which is derived from PATH),
> > not with PATH directly.
> >
> > IME, it's a bad idea to have exec-path inside Emacs be different from
> > PATH outside Emacs, because then you'd have subtle differences between
> > Emacs looking for programs and programs it invokes (such as the shell,
> > for example) looking for programs.  So my advice is not to patch Emacs
> > that way.
> >
> 
> Shouldn't the option
> 
>    --libexecdir="${MINGW_PREFIX}/bin"
> 
> (added to "configure") fix this issue (being 
> MINGW_PREFIX=/c/msys64/mingw64)?

The libexec directory is for auxiliary programs that come with Emacs,
about which we _know_ how Emacs will invoke them.  (Example: hexl.)
That directory will indeed be added to exec-path, but it will then
create the very problem I warned against: the shell will not search
that directory, and so Emacs features that invoke shell commands (via
shell-command etc.) might mysteriously fail.  IOW, visiting
bzip2-compressed files might start working for you, but other features
using bzip2 via shell commands might still fail.

So no, this is not recommended.

Re: Wrong PATH in MSYS2/MINGW64 builds?

[Angelo Graziosi]

Óscar Fuentes wrote:
> Angelo Graziosi writes:
>>
>> I wonder if this is Windows-builds related
>
> What is Windows-builds?

The build done on MinGW64 shell from MSYS2...

>
>> of we can do a better
>> build. BTW I HAVE bzip2 installed (both in MINGW64 and MSYS2 shells):
>>
>> MINGW64 ~
>> $ type bzip2
>> bzip2 is /mingw64/bin/bzip2
>
> Is your emacs.exe installed on the same directory as mingw64 bzip2.exe ?

Yes,

  MINGW64 ~
$ type emacs
emacs è /mingw64/bin/emacs

  MINGW64 ~
$ type bzip2
bzip2 è /mingw64/bin/bzip2

But if (getenv "PATH") prints

"C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\"

it seems to me obvious that bzip2 isn't found. "C:\msys64\mingw64\bin" 
is not in the PATH...

BTW, I start Emacs with a link on status bar and its target is 
"C:\msys64\mingw64\bin\runemacs.exe".

Ciao,
  Angelo.