bug#6665: 24.0.50; rgrep does not work on Windows

bug#6665: 24.0.50; rgrep does not work on Windows

[Christoph]

rgrep does not work on Windows. I am using the GnuWin32 versions of
find and grep and find returns an error when used by Emacs' rgrep.

Start Emacs with `emacs -Q'. 

In *scratch* buffer evaluate the following to point to GNU find and grep executables:
(setq find-program "\"C:/Program Files (x86)/GnuWin32/bin/find.exe\"")
(setq grep-program "\"C:/Program Files (x86)/GnuWin32/bin/grep.exe\"")

Run `M-x rgrep'. Search for "emacs" in files "all" in base dir
"d:/devel/emacs/emacs-bzr/trunk/src/oo-spd/i386/" (or something similar).

*grep* buffer output:
-*- mode: grep; default-directory: "d:/devel/emacs/emacs-bzr/trunk/src/oo-spd/i386/" -*-
Grep started at Sat Jul 17 23:15:55

"C:/Program Files (x86)/GnuWin32/bin/find.exe" . "(" -path "*/SCCS" -o -path "*/RCS" -o -path "*/CVS" -o -path "*/MCVS" -o -path "*/.svn" -o -path "*/.git" -o -path "*/.hg" -o -path "*/.bzr" -o -path "*/_MTN" -o -path "*/_darcs" -o -path "*/{arch}" ")" -prune -o "(" -name ".#*" -o -name "*.o" -o -name "*~" -o -name "*.bin" -o -name "*.bak" -o -name "*.obj" -o -name "*.map" -o -name "*.ico" -o -name "*.pif" -o -name "*.lnk" -o -name "*.a" -o -name "*.ln" -o -name "*.blg" -o -name "*.bbl" -o -name "*.dll" -o -name "*.drv" -o -name "*.vxd" -o -name "*.386" -o -name "*.elc" -o -name "*.lof" -o -name "*.glo" -o -name "*.idx" -o -name "*.lot" -o -name "*.fmt" -o -name "*.tfm" -o -name "*.class" -o -name "*.fas" -o -name "*.lib" -o -name "*.mem" -o -name "*.x86f" -o -name "*.sparcf" -o -name "*.fasl" -o -name "*.ufsl" -o -name "*.fsl" -o -name "*.dxl" -o -name "*.pfsl" -o -name "*.dfsl" -o -name "*.p64fsl" -o -name "*.d64fsl" -o -name "*.dx64fsl" -o -name "*.lo" -o -name "*.la" -o -name "*.gmo" -o -name "*.mo" -o -name "*.toc" -o -name "*.aux" -o -name "*.cp" -o -name "*.fn" -o -name "*.ky" -o -name "*.pg" -o -name "*.tp" -o -name "*.vr" -o -name "*.cps" -o -name "*.fns" -o -name "*.kys" -o -name "*.pgs" -o -name "*.tps" -o -name "*.vrs" -o -name "*.pyc" -o -name "*.pyo" ")" -prune -o  -type f "(" -iname "*" -o -iname ".*" ")" -exec "C:/Program Files (x86)/GnuWin32/bin/grep.exe" -i -n "emacs" {} NUL ";"
C:/Program Files (x86)/GnuWin32/bin/find.exe: paths must precede expression
Usage: C:/Program Files (x86)/GnuWin32/bin/find.exe [-H] [-L] [-P] [path...] [expression]

Grep finished with no matches found at Sat Jul 17 23:15:55

`M-x grep' and `M-x lgrep', as well as `M-x grep-find' work fine.

http://linux.die.net/man/1/find talks about this particular error in section
"Non-bugs", but as far as I can tell the files are enclosed in quotes correctly.


In GNU Emacs 24.0.50.1 (i386-mingw-nt6.1.7600)
 of 2010-07-17 on MARVIN
Windowing system distributor `Microsoft Corp.', version 6.1.7600
configured using `configure --with-gcc (3.4) --cflags -IC:/Progra~2/GnuWin32/include -ID:/devel/emacs/libXpm-3.5.8/include -ID:/devel/emacs/libXpm-3.5.8/src'

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: ENU
  value of $XMODIFIERS: nil
  locale-coding-system: cp1252
  default enable-multibyte-characters: t

Major mode: Grep

Minor modes in effect:
  ido-everywhere: t
  yas/global-mode: t
  global-auto-revert-mode: t
  delete-selection-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t

Recent input:
C-x RET r g r e p <return> <return> <return> <return> 
C-x RET <up> <return> t e s t <return> <return> <return> 
C-x RET <up> <return> <up> <return> <return> <return> 
<down-mouse-1> <mouse-1> q C-x k <return> C-x k <return> 
<up> <up> <up> <up> <down-mouse-1> <mouse-1> <help-echo> 
<backspace> <down-mouse-1> <mouse-1> C-x RET g r e 
p - f i n d <return> s e t q SPC * e <backspace> . 
e l <return> C-x k <return> y e s <return> C-x RET 
r g r e p <return> <return> <return> <return> C-x o 
C-SPC C-SPC C-n C-n C-n C-n C-n C-n C-n M-w C-x RET 
r e p o r t <tab> <return>

Recent messages:
call-interactively: Beginning of buffer
Quit
Grep finished with no matches found
Mark set
Mark activated

Load-path shadows:
None found.

Features:
(shadow sort gnus-cite mail-extr gnus-msg gnus-art mm-uu mml2015 pgg
pgg-parse pgg-def epg-config mm-view smime password-cache dig mailcap
gnus-sum nnoo gnus-group time-date gnus-undo nnmail mail-source
format-spec gnus-start gnus-spec gnus-int gnus-range gnus-win message
rfc822 mml mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
rfc2047 rfc2045 ietf-drums mailabbrev gmm-utils mailheader gnus gnus-ems
nnheader gnus-util mail-utils mm-util mail-prsvr emacsbug find-dired
server package re-builder ack-emacs ibuffer nav nav-tags python-21
python nav-bufs xcscope anything-config warnings semantic/util-modes
semantic/util semantic semantic/tag semantic/lex semantic/fw loaddefs
eieio byte-opt bytecomp byte-compile mode-local cedet imenu bookmark pp
ffap dired rx thingatpt anything woman man cc-mode cc-fonts cc-menus
cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs grep-o-matic grep
compile comint browse-kill-ring+ browse-kill-ring second-sel ido
yasnippet dropdown-list derived easy-mmode assoc etags-table etags ring
remember zenburn color-theme edmacro kmacro wid-edit cl cl-19 sendmail
regexp-opt reporter easymenu uniquify advice help-fns advice-preload
autorevert delsel tooltip ediff-hook vc-hooks lisp-float-type mwheel
dos-w32 disp-table ls-lisp w32-win w32-vars tool-bar dnd fontset image
fringe lisp-mode register page menu-bar rfn-eshadow timer select
scroll-bar mldrag mouse jit-lock font-lock syntax facemenu font-core
frame cham georgian utf-8-lang misc-lang vietnamese tibetan thai
tai-viet lao korean japanese hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese case-table epa-hook jka-cmpr-hook help
simple abbrev button minibuffer faces cus-face files text-properties
overlay md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process multi-tty emacs)

bug#6665: 24.0.50; rgrep does not work on Windows

[Eli Zaretskii]

> From: Christoph <cschol2112@googlemail.com>
> Date: Sun, 18 Jul 2010 08:32:56 -0600
> Cc: 
> 
> rgrep does not work on Windows.

Actually, it does; you just don't like the results ;-)

> In *scratch* buffer evaluate the following to point to GNU find and grep executables:
> (setq find-program "\"C:/Program Files (x86)/GnuWin32/bin/find.exe\"")
> (setq grep-program "\"C:/Program Files (x86)/GnuWin32/bin/grep.exe\"")

Why do you need these?  Doesn't Emacs find them?

> "C:/Program Files (x86)/GnuWin32/bin/find.exe" . "(" -path "*/SCCS" -o -path "*/RCS" -o -path "*/CVS" -o -path "*/MCVS" -o -path "*/.svn" -o -path "*/.git" -o -path "*/.hg" -o -path "*/.bzr" -o -path "*/_MTN" -o -path "*/_darcs" -o -path "*/{arch}" ")" -prune -o "(" -name ".#*" -o -name "*.o" -o -name "*~" -o -name "*.bin" -o -name "*.bak" -o -name "*.obj" -o -name "*.map" -o -name "*.ico" -o -name "*.pif" -o -name "*.lnk" -o -name "*.a" -o -name "*.ln" -o -name "*.blg" -o -name "*.bbl" -o -name "*.dll" -o -name "*.drv" -o -name "*.vxd" -o -name "*.386" -o -name "*.elc" -o -name "*.lof" -o -name "*.glo" -o -name "*.idx" -o -name "*.lot" -o -name "*.fmt" -o -name "*.tfm" -o -name "*.class" -o -name "*.fas" -o -name "*.lib" -o -name "*.mem" -o -name "*.x86f" -o -name "*.sparcf" -o -name "*
 .fasl" -o -name "*.ufsl" -o -name "*.fsl" -o -name "*.dxl" -o -name "*.pfsl" -o -name "*.dfsl" -o -name "*.p64fsl" -o -name "*.d64fsl" -o -name "*.dx64fsl" -o -name "*.lo" -o -name "*.la" -o -name "
> C:/Program Files (x86)/GnuWin32/bin/find.exe: paths must precede expression

If you paste all this long command line to the shell window, does the
program work then?  (You may need to mirror the slashes for that.)

Also, does the problem go away if you replace "(" with "-(" and ")"
with "-)"?

Does the problem go away if you copy the two programs to a directory
whose name does not include blanks and parentheses, and invoke them
from there?

bug#6665: 24.0.50; rgrep does not work on Windows

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Christoph <cschol2112@googlemail.com>
>> Date: Sun, 18 Jul 2010 08:32:56 -0600
>> Cc: 
>> 
>> rgrep does not work on Windows.
>
> Actually, it does; you just don't like the results ;-)
>
>> In *scratch* buffer evaluate the following to point to GNU find and grep executables:
>> (setq find-program "\"C:/Program Files (x86)/GnuWin32/bin/find.exe\"")
>> (setq grep-program "\"C:/Program Files (x86)/GnuWin32/bin/grep.exe\"")
>
> Why do you need these?  Doesn't Emacs find them?

Unless GnuWin32 comes before than %windir%/system32 on the PATH, Emacs
will execute Windows' find.exe. Or the OP is trying to use GnuWin32
without adding it to the PATH.

[snip]

bug#6665: 24.0.50; rgrep does not work on Windows

[Christoph]

On 7/18/2010 11:14 AM, Eli Zaretskii wrote:

>> rgrep does not work on Windows.
>
> Actually, it does; you just don't like the results ;-)

You know, I saw the title in the bug tracker and wondered if someone 
would pick on that. ;)

>> In *scratch* buffer evaluate the following to point to GNU find and grep executables:
>> (setq find-program "\"C:/Program Files (x86)/GnuWin32/bin/find.exe\"")
>> (setq grep-program "\"C:/Program Files (x86)/GnuWin32/bin/grep.exe\"")
>
> Why do you need these?  Doesn't Emacs find them?

No. I usually append the directory of the executables to PATH and set 
the exec path accordingly in my .emacs. This is more portable than 
adding it to the path permanently on every computer. The setqs above 
were just to make it work with `emacs -Q'.

I need GNU find because otherwise Windows find complains about incorrect 
parameter format (maybe a separate issue?).

> If you paste all this long command line to the shell window, does the
> program work then?  (You may need to mirror the slashes for that.)

No. I pasted it into cmd.exe and it gives the exact same error.

> Also, does the problem go away if you replace "(" with "-(" and ")"
> with "-)"?

Same result.

> Does the problem go away if you copy the two programs to a directory
> whose name does not include blanks and parentheses, and invoke them
> from there?

No. The spaces are not the problem. I enclosed the path in quotes so it 
finds the correct executable at this path (I omitted the quotes before 
and it would only find C:\Program).

Christoph

bug#6665: 24.0.50; rgrep does not work on Windows

[Eli Zaretskii]

> Date: Sun, 18 Jul 2010 22:00:16 -0600
> From: Christoph <cschol2112@googlemail.com>
> CC: 6665@debbugs.gnu.org
> 
> > If you paste all this long command line to the shell window, does the
> > program work then?  (You may need to mirror the slashes for that.)
> 
> No. I pasted it into cmd.exe and it gives the exact same error.
> 
> > Also, does the problem go away if you replace "(" with "-(" and ")"
> > with "-)"?
> 
> Same result.
> 
> > Does the problem go away if you copy the two programs to a directory
> > whose name does not include blanks and parentheses, and invoke them
> > from there?
> 
> No. The spaces are not the problem. I enclosed the path in quotes so it 
> finds the correct executable at this path (I omitted the quotes before 
> and it would only find C:\Program).

It's something with your port of Find.  My port is something I did
myself (the original GnuWin32 one was terribly slow and buggy), and it
does work, both with your command line from the shell and inside
Emacs.

I think this bug report should be closed, as it is not something Emacs
can fix.

bug#6665: 24.0.50; rgrep does not work on Windows

[Christoph]

On 7/19/2010 12:02 PM, Eli Zaretskii wrote:

> It's something with your port of Find.  My port is something I did
> myself (the original GnuWin32 one was terribly slow and buggy), and it
> does work, both with your command line from the shell and inside
> Emacs.

Yes, indeed it is.

I tried using cygwin's find instead and it works fine.

Like this:
(setq find-program "\"C:/cygwin/bin/find.exe\"")
(setq grep-program "\"C:/Program Files (x86)/GnuWin32/bin/grep.exe\"")

> I think this bug report should be closed, as it is not something Emacs
> can fix.

Agreed.

Could we add a note somewhere that documents this? Would the 
etc/PROBLEMS file be the right place?

Christoph

bug#6665: 24.0.50; rgrep does not work on Windows

[Eli Zaretskii]

> Date: Mon, 19 Jul 2010 16:56:03 -0600
> From: Christoph <cschol2112@googlemail.com>
> CC: 6665@debbugs.gnu.org
> 
> Could we add a note somewhere that documents this? Would the 
> etc/PROBLEMS file be the right place?

Sure.

bug#6665: rgrep does not work on Windows

[Colin Fraizer]

On July 19, 2010, Eli Zaretskii wrote:

>I think this bug report should be closed, as it is not something Emacs
can fix.

I believe Mr. Zaretskii is correct that Emacs is not the cause of the
problem. It appears to be a bug in the GnuWin32 and/or MSys port of
"find". *However*, I would argue that rgrep constructs a bogus find
command anyway and that *that* should be fixed. Specifically, it
creates

[NOTE: I replaced text with the string "[...]".]

find . "(" -path "*/SCCS" [...] ")" -prune -o "(" -name ".#*" [...] ")"
-prune -o -type f "(" -iname "*.el" ")" -exec grep -nH "mystring" {} ";"

That second "-prune" is really bogus.  It is supposed to mean "don't descend
into
the matched file as a directory", but it's not matching directories.
It is matching regular filenames.

I beleive a better string to generate is

find . "(" -path "*/SCCS" [...] ")" -prune -o -not "(" -name ".#*" [...] ")"
-and -type f "(" -iname "*.el" ")" -exec grep -nH "mystring" {} ";"

That is, I would:
1. Add "-not" before the list of excluded files.
2. Remove the second "-prune".
3. Change the "-o" after the list of excluded files to "-and".

I believe this should work on all systems (even with the buggy version of
"find") and doesn't rely on "-prune" to mean "-noop".

[I'm new here, so please excuse any breaches of rules or etiquette. I'll be
happy to conform to your norms (or to leave) if you instruct me.]

Best regards,
--Colin Fraizer
Indianapolis, Indiana, USA, Earth 8-)

My (very-lightly-tested) version of rgrep follows:
(defun rgrep (regexp &optional files dir confirm)
  "Recursively grep for REGEXP in FILES in directory tree rooted at DIR.
The search is limited to file names matching shell pattern FILES.
FILES may use abbreviations defined in `grep-files-aliases', e.g.
entering `ch' is equivalent to `*.[ch]'.

With \\[universal-argument] prefix, you can edit the constructed shell
command line
before it is executed.
With two \\[universal-argument] prefixes, directly edit and run
`grep-find-command'.

Collect output in a buffer.  While find runs asynchronously, you
can use \\[next-error] (M-x next-error), or
\\<grep-mode-map>\\[compile-goto-error] \
in the grep output buffer,
to go to the lines where grep found matches.

This command shares argument histories with \\[lgrep] and \\[grep-find]."
  (interactive
   (progn
     (grep-compute-defaults)
     (cond
      ((and grep-find-command (equal current-prefix-arg '(16)))
       (list (read-from-minibuffer "Run: " grep-find-command
				   nil nil 'grep-find-history)))
      ((not grep-find-template)
       (error "grep.el: No `grep-find-template' available"))
      (t (let* ((regexp (grep-read-regexp))
		(files (grep-read-files regexp))
		(dir (read-directory-name "Base directory: "
					  nil default-directory t))
		(confirm (equal current-prefix-arg '(4))))
	   (list regexp files dir confirm))))))
  (when (and (stringp regexp) (> (length regexp) 0))
    (unless (and dir (file-directory-p dir) (file-readable-p dir))
      (setq dir default-directory))
    (if (null files)
	(if (not (string= regexp grep-find-command))
	    (compilation-start regexp 'grep-mode))
      (setq dir (file-name-as-directory (expand-file-name dir)))
      (require 'find-dired)		; for `find-name-arg'
      (let ((command (grep-expand-template
		      grep-find-template
		      regexp
		      (concat (shell-quote-argument "(")
			      " " find-name-arg " "
			      (mapconcat #'shell-quote-argument
					 (split-string files)
					 (concat " -o " find-name-arg " "))
			      " "
			      (shell-quote-argument ")"))
		      dir
		      (concat
		       (and grep-find-ignored-directories
			    (concat (shell-quote-argument "(")
				    ;; we should use shell-quote-argument
here
				    " -path "
				    (mapconcat
				     #'(lambda (ignore)
					 (cond ((stringp ignore)
						(shell-quote-argument
						 (concat "*/" ignore)))
					       ((consp ignore)
						(and (funcall (car ignore)
dir)
						     (shell-quote-argument
						      (concat "*/"
							      (cdr
ignore)))))))
				     grep-find-ignored-directories
				     " -o -path ")
				    " "
				    (shell-quote-argument ")")
				    " -prune -o "))
		       (and grep-find-ignored-files
			    (concat (shell-quote-argument "-not")
                        " "
                        (shell-quote-argument "(")
                        ;; we should use shell-quote-argument here
                        " -name "
				    (mapconcat
				     #'(lambda (ignore)
					 (cond ((stringp ignore)
						(shell-quote-argument
ignore))
					       ((consp ignore)
						(and (funcall (car ignore)
dir)
						     (shell-quote-argument
						      (cdr ignore))))))
				     grep-find-ignored-files
				     " -o -name ")
				    " "
				    (shell-quote-argument ")")
				    " -and "))))))
	(when command
	  (if confirm
	      (setq command
		    (read-from-minibuffer "Confirm: "
					  command nil nil
'grep-find-history))
	    (add-to-history 'grep-find-history command))
	  (let ((default-directory dir))
	    (compilation-start command 'grep-mode))
	  ;; Set default-directory if we started rgrep in the *grep* buffer.
	  (if (eq next-error-last-buffer (current-buffer))
	      (setq default-directory dir)))))))

bug#6665: rgrep does not work on Windows

[Andreas Schwab]

"Colin Fraizer" <colin@cfraizer.com> writes:

> find . "(" -path "*/SCCS" [...] ")" -prune -o "(" -name ".#*" [...] ")"
> -prune -o -type f "(" -iname "*.el" ")" -exec grep -nH "mystring" {} ";"
>
> That second "-prune" is really bogus.  It is supposed to mean "don't descend
> into
> the matched file as a directory", but it's not matching directories.
> It is matching regular filenames.

It can match directories as well, just like the first expression can
match non-directories.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#6665: rgrep does not work on Windows

[Lennart Borgman]

On Wed, Nov 17, 2010 at 1:26 PM, Colin Fraizer <colin@cfraizer.com> wrote:
>
> On July 19, 2010, Eli Zaretskii wrote:
>
>>I think this bug report should be closed, as it is not something Emacs
> can fix.
>
> I believe Mr. Zaretskii is correct that Emacs is not the cause of the
> problem. It appears to be a bug in the GnuWin32 and/or MSys port of
> "find". *However*, I would argue that rgrep constructs a bogus find
> command anyway and that *that* should be fixed. Specifically, it
> creates

We discussed some time ago the possibility to use "grep -r" for rgrep.
It was rejected, but I have implemented this in the library w32-grep
which is part of my patched Emacs (i.e. EmacsW32).

bug#6665: rgrep does not work on Windows

[Thierry Volpiatto]

Lennart Borgman <lennart.borgman@gmail.com> writes:

> On Wed, Nov 17, 2010 at 1:26 PM, Colin Fraizer <colin@cfraizer.com> wrote:
>>
>> On July 19, 2010, Eli Zaretskii wrote:
>>
>>>I think this bug report should be closed, as it is not something Emacs
>> can fix.
>>
>> I believe Mr. Zaretskii is correct that Emacs is not the cause of the
>> problem. It appears to be a bug in the GnuWin32 and/or MSys port of
>> "find". *However*, I would argue that rgrep constructs a bogus find
>> command anyway and that *that* should be fixed. Specifically, it
>> creates
>
> We discussed some time ago the possibility to use "grep -r" for rgrep.
> It was rejected, but I have implemented this in the library w32-grep
> which is part of my patched Emacs (i.e. EmacsW32).
FYI i have fixed in anything the use of grep with the Gnuwin32 version
of grep.(See anything-config.el)

So it work now both in GNU/Linux and windows.

It is recursive by default like grep -r or -d recurse.

By the nature of the implementation of grep in anything, you
can select quickly a set of files/dirs not only in current directory but
anywhere in your system files, with the use of wildcard or not.
That increase the performance because you can search exactly in what you
want.

It is usable without knowledge of grep command (just write in the prompt
what you search for).

Of course, it is incremental unlike the vanilla emacs grep.
With that i don't feel the need of grep/find.

You can access it at anytime from f5 a C-x C-f (aka M-x anything-find-files)
or with M-x anything-do-grep (f5 a M-g s).

-- 
A+ Thierry
Get my Gnupg key:
gpg --keyserver pgp.mit.edu --recv-keys 59F29997