Working rgrep in windows emacs in 2024? (GNU emacs 29.1, windows 11 Enterprise)

Working rgrep in windows emacs in 2024? (GNU emacs 29.1, windows 11 Enterprise)

[Steinar Bang]

I've used emacs for many years on many platforms, including windows.

One thing I use a lot on linux (and MacOS when I had that), is rgrep and
'C-c p s g' in projectile.

And rgrep is something I have never managed to get working on windows.

Now I have a new windows machine so I make a new attempt.

It's not that I haven't tried before:

I've messed with setting the PATH environment variable (from my .emacs)
   (if windows-emacs
        (setenv "PATH" (concat "c:\\ProgramFiles\\ezwinports\\bin;C:\\ProgramFiles\\GnuWin32\\bin;" (getenv "PATH"))))
and I've tried adding to the exec-path
    (when windows-emacs
        (push "C:/ProgramFiles/GnuWin32/bin" exec-path)
        (push "c:/ProgramFiles/ezwinports/bin" exec-path))

But neither of the above settings have ever given me a working rgrep.

I also have the following grep-related setting:
(setq grep-command "grep -n "
      grep-find-ignored-directories '("SCCS" "RCS" "CVS" "MCVS" ".src" ".svn" ".git" ".hg" ".bzr" "_MTN" "_darcs" "{arch}" "node_modules" "target"))

What has been at the end of the added paths (not right now, or "not
yet", in my (or "my employer's") new Windows 11 laptop) in the PATH and
exec-path settings above, should be obvious to those that have tried
tinkering with this before.

But in case they aren't known, explanation follows:

Ezwinports is Emacs maintainer Eli Zaretskii's old port of linux/unix
command line tools to windows:
 https://www.gnu.org/software/emacs/manual/html_node/efaq-w32/EZWinPorts.html

Do these still work for 64bit windows?
Are they recommended over GnuWin32?
Are they deprecated for GnuWin32?

Then there is GnuWin32
 https://gnuwin32.sourceforge.net/

It doesn't look maintained, judging from the dates on the front page...?

The grep is from February 2009
 https://gnuwin32.sourceforge.net/packages/grep.htm
The find is fron April 2005
 https://gnuwin32.sourceforge.net/packages/findutils.htm

Granted there probably haven't been much active development going on in
these utils but there have to have been bugfixes and security fixes and
adapting to API changes to the underlying APIs for all of these
years...?

And being this old they probably aren't 64bit (yes they have "win32" in
the name, but that could be misleading).

Does anyone else have a working rgrep in a recent emacs on a recent
windows on amd64, they would like to share?

Thanks!


- Steinar

Re: Working rgrep in windows emacs in 2024? (GNU emacs 29.1, windows 11 Enterprise)

[Steinar Bang]

>>>>> Steinar Bang <sb@dod.no>:

> Does anyone else have a working rgrep in a recent emacs on a recent
> windows on amd64, they would like to share?

Is the WSL (Windows Subsystem for Linux) the way to go today?

Or is stuff found in WSL not reachable from a regular windows application?

Re: Working rgrep in windows emacs in 2024? (GNU emacs 29.1, windows 11 Enterprise)

[Steinar Bang]

>>>>> Steinar Bang <sb@dod.no>:
> But in case they aren't known, explanation follows:

> Ezwinports is Emacs maintainer Eli Zaretskii's old port of linux/unix
> command line tools to windows:
>  https://www.gnu.org/software/emacs/manual/html_node/efaq-w32/EZWinPorts.html

> Do these still work for 64bit windows?
> Are they recommended over GnuWin32?
> Are they deprecated for GnuWin32?

Ezwinports is newer and more maintained than GnuWin32, at least if I am
to judge from the README.txt
 https://sourceforge.net/projects/ezwinports/files/

The README.txt file was updated on Januar 19 2024 and mentions 2021
early on.

So I'll try ezwinports (unless somebody brings something newer to the table?).

Re: Working rgrep in windows emacs in 2024? (GNU emacs 29.1, windows 11 Enterprise)

[Steinar Bang]

>>>>> Steinar Bang <sb@dod.no>:

> Ezwinports is newer and more maintained than GnuWin32, at least if I am
> to judge from the README.txt
>  https://sourceforge.net/projects/ezwinports/files/

> The README.txt file was updated on Januar 19 2024 and mentions 2021
> early on.

> So I'll try ezwinports (unless somebody brings something newer to the table?).

I have downloaded the grep w32 utilities from ezwinports and the find
w64 utilities, unpacked them, and copied the files in bin directory of
both unpacked zip files, into a directory found in the PATH (as set with
setenv in a previous post in the thread), and in exec-path (as set with
a different setting):
 c:\Windows\ezwinports

But I couldn't find the required dlls mentioned in the ezwinports
README.

The
 https://osdn.net/projects/mingw/downloads/72215/libgcc-9.2.0-3-mingw32-dll-1.tar.xz/
and
 https://osdn.net/projects/mingw/downloads/72210/libstdc%2B%2B-9.2.0-3-mingw32-dll-6.tar.xz/
URLs get 504 Bad Gateway and broken SSL

And I can't figure out where they are, starting at
 https://osdn.net/projects/mingw/releases/
 

Re: Working rgrep in windows emacs in 2024? (GNU emacs 29.1, windows 11 Enterprise)

[Eli Zaretskii]

> From: Steinar Bang <sb@dod.no>
> Date: Tue, 13 Feb 2024 13:55:27 +0100
> 
> I have downloaded the grep w32 utilities from ezwinports and the find
> w64 utilities, unpacked them, and copied the files in bin directory of
> both unpacked zip files, into a directory found in the PATH (as set with
> setenv in a previous post in the thread), and in exec-path (as set with
> a different setting):
>  c:\Windows\ezwinports
> 
> But I couldn't find the required dlls mentioned in the ezwinports
> README.
> 
> The
>  https://osdn.net/projects/mingw/downloads/72215/libgcc-9.2.0-3-mingw32-dll-1.tar.xz/
> and
>  https://osdn.net/projects/mingw/downloads/72210/libstdc%2B%2B-9.2.0-3-mingw32-dll-6.tar.xz/
> URLs get 504 Bad Gateway and broken SSL

I still see the files:

  https://osdn.net/projects/mingw/downloads/72215/libgcc-9.2.0-3-mingw32-dll-1.tar.xz/
  https://osdn.net/projects/mingw/downloads/72210/libstdc%2B%2B-9.2.0-3-mingw32-dll-6.tar.xz/

The site is a bit slow to load, and you could get a warning about an
expired license, but if you insist, you will get to the DLLs.

> And I can't figure out where they are, starting at
>  https://osdn.net/projects/mingw/releases/

They are under "MinGW.OSDN Compiler Collection (GCC)".

I'm sorry about the hassle, but I cannot distribute the binaries of
these DLLs without also distributing the humongous GCC and libstdc++
source tarballs.

Re: Working rgrep in windows emacs in 2024? (GNU emacs 29.1, windows 11 Enterprise)

[Steinar Bang]

>>>>> Eli Zaretskii <eliz@gnu.org>:

> I still see the files:

>   https://osdn.net/projects/mingw/downloads/72215/libgcc-9.2.0-3-mingw32-dll-1.tar.xz/
>   https://osdn.net/projects/mingw/downloads/72210/libstdc%2B%2B-9.2.0-3-mingw32-dll-6.tar.xz/

> The site is a bit slow to load, and you could get a warning about an
> expired license, but if you insist, you will get to the DLLs.

I tried a bit harder and insisted on the first one (where I got an
expired certificate warning, I think...?).

The second one just downloaded (but took its time doing so... it also,
apparently quietly and slowly downloaded yesterday...?)

Anyway: both now unpacked and copied to C:\ProgramFiles\ezwinports
(Note: directory exactly as shown above, i.e. no space in the
ProgramFiles directory name)

Thanks!

>> And I can't figure out where they are, starting at
>> https://osdn.net/projects/mingw/releases/

> They are under "MinGW.OSDN Compiler Collection (GCC)".

> I'm sorry about the hassle, but I cannot distribute the binaries of
> these DLLs without also distributing the humongous GCC and libstdc++
> source tarballs.

No worries! Thanks again!

Re: Working rgrep in windows emacs in 2024? (GNU emacs 29.1, windows 11 Enterprise)

[Eli Zaretskii]

> From: Steinar Bang <sb@dod.no>
> Date: Tue, 13 Feb 2024 12:20:50 +0100
> 
> I've used emacs for many years on many platforms, including windows.
> 
> One thing I use a lot on linux (and MacOS when I had that), is rgrep and
> 'C-c p s g' in projectile.
> 
> And rgrep is something I have never managed to get working on windows.

FWIW, rgrep works for me on MS-Windows.

> I've messed with setting the PATH environment variable (from my .emacs)
>    (if windows-emacs
>         (setenv "PATH" (concat "c:\\ProgramFiles\\ezwinports\\bin;C:\\ProgramFiles\\GnuWin32\\bin;" (getenv "PATH"))))
> and I've tried adding to the exec-path
>     (when windows-emacs
>         (push "C:/ProgramFiles/GnuWin32/bin" exec-path)
>         (push "c:/ProgramFiles/ezwinports/bin" exec-path))

I don't do that because the directory with ports of GNU software,
including 'find' and 'grep', is on my system-wide PATH.

Does Emacs find find.exe and grep.exe if you try it with
executable-find, like this:

  M-: (executable-find "find") RET
  M-: (executable-find "grep") RET

> But neither of the above settings have ever given me a working rgrep.

Which means what?  What do you see if you try invoking rgrep?  Please
show everything you type at rgrep's prompts and the output and/or
error messages you get as result.

> Ezwinports is Emacs maintainer Eli Zaretskii's old port of linux/unix
> command line tools to windows:
>  https://www.gnu.org/software/emacs/manual/html_node/efaq-w32/EZWinPorts.html
> 
> Do these still work for 64bit windows?

Yes.  I use them every day on 64-bit Windows 11.

> Are they recommended over GnuWin32?

Yes.

> Then there is GnuWin32
>  https://gnuwin32.sourceforge.net/
> 
> It doesn't look maintained, judging from the dates on the front page...?

Yes.

> The grep is from February 2009
>  https://gnuwin32.sourceforge.net/packages/grep.htm
> The find is fron April 2005
>  https://gnuwin32.sourceforge.net/packages/findutils.htm

Don't use those, use the ones from ezwinports: they are old, but they
do the job.  Or install newer ports from the MSYS2 site (but be sure
to install MinGW64 ports, not MSYS ports).  I don't use those, but
many others do.

> Does anyone else have a working rgrep in a recent emacs on a recent
> windows on amd64, they would like to share?

I just tried

  M-x rgrep RET DOS_NT RET .[ch] RET /path/to/emacs RET

(where /path/to/emacs was replaced with the actual top-level directory
of the Emacs source tree), and pronto saw the list of 178 matches in
the current Emacs tree.

So yes, it works here, but I'm not sure what can I share, except what
I wrote above.

Re: Working rgrep in windows emacs in 2024? (GNU emacs 29.1, windows 11 Enterprise)

[Steinar Bang]

>>>>> Eli Zaretskii <eliz@gnu.org>:

>> I've messed with setting the PATH environment variable (from my .emacs)
>> (if windows-emacs
>> (setenv "PATH" (concat "c:\\ProgramFiles\\ezwinports\\bin;C:\\ProgramFiles\\GnuWin32\\bin;" (getenv "PATH"))))
>> and I've tried adding to the exec-path
>> (when windows-emacs
>> (push "C:/ProgramFiles/GnuWin32/bin" exec-path)
>> (push "c:/ProgramFiles/ezwinports/bin" exec-path))

> I don't do that because the directory with ports of GNU software,
> including 'find' and 'grep', is on my system-wide PATH.

I tried that now, by removing the above from .emacs, editing PATH for my
user to put C:\ProgramFiles\ezwinports first.  But that still failed
    -*- mode: grep; default-directory: "~/projects/altinn3/app-localtest/" -*-
    Grep started at Wed Feb 14 08:47:34
    
    find -H . -type d "(" -path "*/SCCS" -o -path "*/RCS" -o -path "*/CVS" -o -path "*/MCVS" -o -path "*/.src" -o -path "*/.svn" -o -path "*/.git" -o -path "*/.hg" -o -path "*/.bzr" -o -path "*/_MTN" -o -path "*/_darcs" -o -path "*/{arch}" -o -path "*/node_modules" -o -path "*/target" ")" -prune -o ^"^!^" -type d "(" -name ".#*" -o -name "*.o" -o -name "*~" -o -name "*.bin" -o -name "*.bak" -o -name "*.obj" -o -name "*.map" -o -name "*.ico" -o -name "*.pif" -o -name "*.lnk" -o -name "*.a" -o -name "*.ln" -o -name "*.blg" -o -name "*.bbl" -o -name "*.dll" -o -name "*.drv" -o -name "*.vxd" -o -name "*.386" -o -name "*.elc" -o -name "*.lof" -o -name "*.glo" -o -name "*.idx" -o -name "*.lot" -o -name "*.fmt" -o -name "*.tfm" -o -name "*.class" -o -name "*.fas" -o -name "*.lib" -o -name "*.mem" -o -name "*.x86f" -o -name "*.sparcf" -o -name "*.dfsl" -o -name "*.pfsl" -o -name "*.d64fsl" -o -name "*.p64fsl" -o -name "*.lx64fsl" -o -name "*.lx32fsl" -o -name "*.dx64fsl" -o -name "*.dx32fsl" -o -name "*.fx64fsl" -o -name "*.fx32fsl" -o -name "*.sx64fsl" -o -name "*.sx32fsl" -o -name "*.wx64fsl" -o -name "*.wx32fsl" -o -name "*.fasl" -o -name "*.ufsl" -o -name "*.fsl" -o -name "*.dxl" -o -name "*.lo" -o -name "*.la" -o -name "*.gmo" -o -name "*.mo" -o -name "*.toc" -o -name "*.aux" -o -name "*.cp" -o -name "*.fn" -o -name "*.ky" -o -name "*.pg" -o -name "*.tp" -o -name "*.vr" -o -name "*.cps" -o -name "*.fns" -o -name "*.kys" -o -name "*.pgs" -o -name "*.tps" -o -name "*.vrs" -o -name "*.pyc" -o -name "*.pyo" ")" -prune -o  -type f "(" -iname "*.yml" ")" -exec grep --color=always -n --null -e "LocalAppMode" "{}" NUL ";"
    FIND: Parameter format not correct

    Grep exited abnormally with code 2 at Wed Feb 14 08:47:34

Looking at the output from getenv it looks like the system setting for
PATH comes before mine:
    C:\Program Files\PuTTY;C:\Program Files\Common Files\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files (x86)\Pulse Secure\VC142.CRT\X64\;C:\Program Files (x86)\Pulse Secure\VC142.CRT\X86\;C:\Program Files (x86)\Common Files\Pulse Secure\TNC Client Plugin\;C:\Program Files\WindowsPowerShell\Scripts\HP.ClientScriptLibrary;C:\Program Files\Git\cmd;C:\Program Files\PuTTY\;C:\Program Files\Calibre2\;C:\Program Files\dotnet\;C:\Program Files\Rancher Desktop\resources\resources\win32\bin\;C:\Program Files\Rancher Desktop\resources\resources\linux\bin\;C:\ProgramFiles\ezwinports;C:\Users\sb\AppData\Local\Microsoft\WindowsApps;C:\Users\sb\apps\apache-maven-3.9.6\bin;C:\Users\sb\apps\bin;C:\Users\sb\.dotnet\tools;C:\Users\sb\AppData\Local\Programs\Microsoft VS Code\bin;

It could be that tried putting the settings in the emacs config to avoid
messing with the system PATH setting...?  Or that I was lacking
privileges to change that?

Both config settings (i.e. both setenv on the PATH, and push on
exec-path) are from 2015, and the last one (push on exec-path) has
a commit message starting with "Attempt at", so I've never had success
here, I think.

> Does Emacs find find.exe and grep.exe if you try it with
> executable-find, like this:

>   M-: (executable-find "find") RET

On emacs with .emacs config removed, and C:\ProgramFiles\ezwinports
added to PATH:  "c:/Windows/system32/find.exe"
(so that's why it didn't work)

On emacs without the PATH setting and the .emacs config in place:
"c:/Windows/system32/find.exe"

So the .emacs config doesn't work and probably never have.

>   M-: (executable-find "grep") RET

On emacs with .emacs config removed, and C:\ProgramFiles\ezwinports
added to PATH:  "c:/Windows/system32/find.exe"

So that was promising.

On emacs without the PATH setting and the .emacs config in place:
nil

So, yeah, that has never worked.

>> But neither of the above settings have ever given me a working rgrep.

> Which means what?  What do you see if you try invoking rgrep?  Please
> show everything you type at rgrep's prompts and the output and/or
> error messages you get as result.

Sorry about that! Forgot!

Included above.

But I guess that means that if I can put C:\ProgramFiles\ezwinports
first in the SYSTEM PATH then rgrep in emacs should work?

I'll try (somewhat locked down corporate machine, so it can sometimes be
hard to find out what I am allowed and not allowed to do...).

>> Ezwinports is Emacs maintainer Eli Zaretskii's old port of linux/unix
>> command line tools to windows:
>> https://www.gnu.org/software/emacs/manual/html_node/efaq-w32/EZWinPorts.html
>> 
>> Do these still work for 64bit windows?

> Yes.  I use them every day on 64-bit Windows 11.

Just a side note: the find was both a win64 and a win32 version, and I
picked the win64 version.  But the grep was just a win32 version.

Does that matter/have any unforseen consequences?

Re: Working rgrep in windows emacs in 2024? (GNU emacs 29.1, windows 11 Enterprise)

[Steinar Bang]

>>>>> Steinar Bang <sb@dod.no>:

> But I guess that means that if I can put C:\ProgramFiles\ezwinports
> first in the SYSTEM PATH then rgrep in emacs should work?

> I'll try (somewhat locked down corporate machine, so it can sometimes be
> hard to find out what I am allowed and not allowed to do...).

First dialog that popped up had system variable edit buttons greyed out.

But when I went in "the old way", so to speak, i.e. via "System
properties" and "Environment variables", I got the same dialog but with
the system variables enabled.

Then I put C:\ProgramFiles\ezwinports at the start of the system
enviroment variable "Path".

When I started a fresh emacs rgrep worked just as well as it does on
linuxen and macs.

Wow! That took 9 years, but I finally got there! :-)

Thanks again, Eli!

Re: Working rgrep in windows emacs in 2024? (GNU emacs 29.1, windows 11 Enterprise)

[Eli Zaretskii]

> From: Steinar Bang <sb@dod.no>
> Date: Wed, 14 Feb 2024 10:00:43 +0100
> 
> But when I went in "the old way", so to speak, i.e. via "System
> properties" and "Environment variables", I got the same dialog but with
> the system variables enabled.
> 
> Then I put C:\ProgramFiles\ezwinports at the start of the system
> enviroment variable "Path".
> 
> When I started a fresh emacs rgrep worked just as well as it does on
> linuxen and macs.

That's exactly the way to go.  The error message you show in your
previous message:

    FIND: Parameter format not correct

indicates that the find.exe program which comes with Windows (a
completely different utility with a different purpose and command-line
syntax) was being called instead of GNU Find.

> Wow! That took 9 years, but I finally got there! :-)

Congrats!

> Thanks again, Eli!

You're welcome.

Re: Working rgrep in windows emacs in 2024? (GNU emacs 29.1, windows 11 Enterprise)

[Eli Zaretskii]

> From: Steinar Bang <sb@dod.no>
> Date: Wed, 14 Feb 2024 09:26:27 +0100
> 
> Just a side note: the find was both a win64 and a win32 version, and I
> picked the win64 version.  But the grep was just a win32 version.
> 
> Does that matter/have any unforseen consequences?

It shouldn't matter.  On Windows, 64-programs can invoke 32-programs,
and vice versa.

Re: Working rgrep in windows emacs in 2024? (GNU emacs 29.1, windows 11 Enterprise)

[Gregor Zattler]

Hi Eli,
* Eli Zaretskii <eliz@gnu.org> [2024-02-13; 15:28 +02]:
> I just tried
>
>   M-x rgrep RET DOS_NT RET .[ch] RET /path/to/emacs RET
>
> (where /path/to/emacs was replaced with the actual top-level directory
> of the Emacs source tree), and pronto saw the list of 178 matches in
> the current Emacs tree.

That was actually

M-x rgrep RET DOS_NT RET *.[ch] RET /path/to/emacs RET
                         ^
wasn't it?  With your original receipt I did not get
matches, with the additional asterisk, it's 180 matches
for me.

Ciao; Gregor

Re: Working rgrep in windows emacs in 2024? (GNU emacs 29.1, windows 11 Enterprise)

[Eli Zaretskii]

> From: Gregor Zattler <telegraph@gmx.net>
> Date: Thu, 15 Feb 2024 10:40:09 +0100
> 
> Hi Eli,
> * Eli Zaretskii <eliz@gnu.org> [2024-02-13; 15:28 +02]:
> > I just tried
> >
> >   M-x rgrep RET DOS_NT RET .[ch] RET /path/to/emacs RET
> >
> > (where /path/to/emacs was replaced with the actual top-level directory
> > of the Emacs source tree), and pronto saw the list of 178 matches in
> > the current Emacs tree.
> 
> That was actually
> 
> M-x rgrep RET DOS_NT RET *.[ch] RET /path/to/emacs RET
>                          ^
> wasn't it?

Yes, sorry.