bug#29680: 24.5; find-grep not finding a file or missing the grep

bug#29680: 24.5; find-grep not finding a file or missing the grep

[Donald H Locker]

The particular bug is not easy to reproduce (yet) and I don't have an
example I can actually send.

`M-x find-grep` (aka `grep-find`), then:

     find . -type f -name '*-2017-12-12_09.22.57.log' -exec grep -nHE 
'ERROR:' {} \;

finds a number of ERROR: lines in various `.log` files at various depths 
in the directory hierarchy where I am working.

One particular file with an 'ERROR:'

     ./PFF/PFF/PFF.bat-2017-12-12_09.22.57.log

is not found unless I change down a directory level, starting in 
`./PFF`, or unless the parent directory name is [at least] one character 
longer or shorter (but see below.) e.g. the ERROR: line is found in

     ./PFF/PFFm/PFF.bat-2017-12-12_09.22.57.log

or in

     ./PFF/PFF-/PFF.bat-2017-12-12_09.22.57.log

but never in

     ./PFF/PFF/PFF.bat-2017-12-12_09.22.57.log

whether the original `./PFF/PFF` directory is renamed or copied to a 
[longer-named] parent (when copied, I have both `./PFF/PFF` and 
`./PFF/PFFx` directories.)

Then it gets weird - if I have a copy of the `./PFF/PFF` directory named 
`./PFF/PF` or `./PFF/P`, the `ERROR:` lines in both copies of the `.log` 
file are found (one in each of the directories;) if the duplicate 
directory is named with three or more characters (e.g. `./PFF/PFM`) the 
original `./PFF/PFF` log file is not found (or grepped; not sure yet 
which.) Simply creating a `./PFF/PF` directory does not work; it has to 
be populated with [at least some] of the same files found in `./PFF/PFF`

It might be worth mentioning that the `./PFF` directory does have two 
other subdirs in addition to `./PFF/PFF` (`./PFF/PFF_FILE_HANDLING` and 
`./PFF/PFF_CAPTURE_FF`) so it doesn't appear that the problem is that 
`./PFF/PFF` is all alone.

     find . -type f -name '*-2017-12-12_09.22.57.log' -exec grep -nHE 
'ERROR:' {} \;

at the command line works as expected.



In GNU Emacs 24.5.1 (i686-pc-mingw32)
  of 2015-04-11 on LEG570
Windowing system distributor `Microsoft Corp.', version 6.1.7601
Configured using:
  `configure --prefix=/c/usr --host=i686-pc-mingw32'

Important settings:
   value of $LANG: ENU
   locale-coding-system: cp1252

Major mode: Dired by date

Minor modes in effect:
   shell-dirtrack-mode: t
   cscope-minor-mode: t
   show-paren-mode: t
   savehist-mode: t
   desktop-save-mode: t
   tooltip-mode: t
   electric-indent-mode: t
   mouse-wheel-mode: t
   tool-bar-mode: t
   menu-bar-mode: t
   file-name-shadow-mode: t
   global-font-lock-mode: t
   font-lock-mode: t
   blink-cursor-mode: t
   auto-composition-mode: t
   auto-encryption-mode: t
   auto-compression-mode: t
   buffer-read-only: t
   line-number-mode: t
   transient-mark-mode: t

Recent messages:
Wrote 
c:/workingSVN/cr17935-dolly-port-1/testing/VectorCAST/openecu-unit-test/environment/PFF/PFF/PFF.bat
Directory has changed on disk; type g to update Dired
Mark saved where search started
Mark set
Deleting...done
Mark saved where search started
Mark set
Mark saved where search started
Making completion list... [6 times]
Note: file is write protected

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils find-dired misearch multi-isearch pp apropos
cus-edit wid-edit dired-aux help-mode shell pcomplete csharp-mode js
advice byte-opt bytecomp byte-compile cl-extra cconv imenu thingatpt
arc-mode archive-mode nroff-mode ruler-mode mule-util hl-line hexl eldoc
help-fns vc-git python json make-mode cperl-mode info nxml-uchnm rng-xsd
xsd-regexp rng-cmpct rng-nxml rng-valid rng-loc rng-uri rng-parse
nxml-parse rng-match rng-dt rng-util rng-pttrn nxml-ns nxml-mode
nxml-outln nxml-rap nxml-util nxml-glyph nxml-enc xmltok cc-langs
cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine
cc-vars cc-defs conf-mode bat-mode vc-hg sh-script smie executable
sgml-mode vc-dispatcher vc-svn dired xcscope easymenu saveplace edmacro
kmacro paren savehist grep compile comint ansi-color ring desktop
frameset cl-loaddefs cl-lib cus-start cus-load time-date tooltip
electric uniquify ediff-hook vc-hooks lisp-float-type mwheel dos-w32
ls-lisp w32-common-fns disp-table w32-win w32-vars tool-bar dnd fontset
image regexp-opt fringe tabulated-list newcomment lisp-mode prog-mode
register page menu-bar rfn-eshadow timer select scroll-bar mouse
jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
w32notify w32 multi-tty emacs)

Memory information:
((conses 8 897528 60889)
  (symbols 32 32816 0)
  (miscs 32 22333 927)
  (strings 16 76656 18285)
  (string-bytes 1 2399020)
  (vectors 8 34189)
  (vector-slots 4 1012343 26868)
  (floats 8 734 707)
  (intervals 28 141955 4946)
  (buffers 508 605))

-- 
*Plain Text* email -- it's an accessibility issue
() no proprietary attachments; no html mail
/\ <http://www.georgedillon.com/web/html_email_is_evil.shtml>

bug#29680: another grep-find anomalous behaviour

[Donald H Locker]

-*- mode: grep; default-directory: 
"c:/workingSVN/2017-08-21-xxxxx-v2g-test-model/Model_withEVSE/" -*-
Grep started at Fri Dec 15 07:12:23

find . -type f -name '*.log' -exec grep -nHE 'CurrentDemand' {} \;
/usr/bin/find: paths must precede expression
Usage: /usr/bin/find [path...] [expression]

Grep exited abnormally with code 1 at Fri Dec 15 07:12:23

######### from the next higher directory - works as expected #########

-*- mode: grep; default-directory: 
"c:/workingSVN/2017-08-21-xxxxx-v2g-test-model/" -*-
Grep started at Fri Dec 15 07:12:23

find . -type f -name '*.log' -exec grep -nHE 'CurrentDemand' {} \;
[lines found] ...
Grep finished (matches found) at Fri Dec 15 07:20:57

######### notes ##########

Command (copied and pasted from the emacs grep buffer) works correctly 
from bash (cygwin-64, mintty) in either of those directories.

GNU Emacs 24.5.1 (i686-pc-mingw32)
  of 2015-04-11 on LEG570
Windows-7, 64-bit, patched up-to-date

Donald.
-- 
*Plain Text* email -- it's an accessibility issue
() no proprietary attachments; no html mail
/\ <http://www.georgedillon.com/web/html_email_is_evil.shtml>

bug#29680: another grep-find anomalous behaviour

[Eli Zaretskii]

> From: Donald H Locker <dhlocker@comcast.net>
> Date: Fri, 15 Dec 2017 07:25:33 -0500
> 
> -*- mode: grep; default-directory: 
> "c:/workingSVN/2017-08-21-xxxxx-v2g-test-model/Model_withEVSE/" -*-
> Grep started at Fri Dec 15 07:12:23
> 
> find . -type f -name '*.log' -exec grep -nHE 'CurrentDemand' {} \;
> /usr/bin/find: paths must precede expression
> Usage: /usr/bin/find [path...] [expression]
> 
> Grep exited abnormally with code 1 at Fri Dec 15 07:12:23

Do you have the sub-shell customized to invoke a Unixy shell or
something?  Windows shells don't support quoting 'like this', so you
need to use "*.log" instead.  And, depending on which port of GNU
Findutils you have installed, even "*.log" could get expanded, even
though it's quoted, because Windows Vista changed its interpretation
of quoting wrt previous versions of Windows.

Also, I wonder how come Emacs doesn't propose to the the command with
"+" as it did for me.

FWIW, your command (with a different search string) works for me, but
it isn't surprising, since in your case it sometimes works and
sometimes doesn't.

> GNU Emacs 24.5.1 (i686-pc-mingw32)

Emacs 24 is quite old.  Could you try the latest pretest of Emacs 26,
please?

bug#29680: another grep-find anomalous behaviour

[Eli Zaretskii]

> Date: Fri, 15 Dec 2017 18:49:02 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 29680@debbugs.gnu.org
> 
> Also, I wonder how come Emacs doesn't propose to the the command with
> "+" as it did for me.

I meant to write "propose to end the command".  Sorry.

bug#29680: another grep-find anomalous behaviour

[Donald H Locker]

Thank you, Eli.

The underlying "unixy" environment is cygwin; updated a few weeks ago. 
(Windows command tools are junk.)

The behaviour is very repeatable, though - at one level in the directory 
hierarchy, the command fails to even execute find; at one higher level 
in the directory hierarchy, the command succeeds and finds the sought 
strings in the file. Note that in the Model_withEVSE directory, "find ." 
doesn't seem to think that '.' is a directory, while it is perfectly 
happy when started in the directory above that. (word wrap seems to have 
made those lines hard to see clearly.)

I'll see if I can drag myself away from Emacs-24.

Donald.
-- 
*Plain Text* email -- it's an accessibility issue
() no proprietary attachments; no html mail
/\ <http://www.georgedillon.com/web/html_email_is_evil.shtml>

On 15-Dec-2017 11:49, Eli Zaretskii wrote:
>> From: Donald H Locker <dhlocker@comcast.net>
>> Date: Fri, 15 Dec 2017 07:25:33 -0500
>>
>> -*- mode: grep; default-directory:
>> "c:/workingSVN/2017-08-21-xxxxx-v2g-test-model/Model_withEVSE/" -*-
>> Grep started at Fri Dec 15 07:12:23
>>
>> find . -type f -name '*.log' -exec grep -nHE 'CurrentDemand' {} \;
>> /usr/bin/find: paths must precede expression
>> Usage: /usr/bin/find [path...] [expression]
>>
>> Grep exited abnormally with code 1 at Fri Dec 15 07:12:23
> 
> Do you have the sub-shell customized to invoke a Unixy shell or
> something?  Windows shells don't support quoting 'like this', so you
> need to use "*.log" instead.  And, depending on which port of GNU
> Findutils you have installed, even "*.log" could get expanded, even
> though it's quoted, because Windows Vista changed its interpretation
> of quoting wrt previous versions of Windows.
> 
> Also, I wonder how come Emacs doesn't propose to the the command with
> "+" as it did for me.
> 
> FWIW, your command (with a different search string) works for me, but
> it isn't surprising, since in your case it sometimes works and
> sometimes doesn't.
> 
>> GNU Emacs 24.5.1 (i686-pc-mingw32)
> 
> Emacs 24 is quite old.  Could you try the latest pretest of Emacs 26,
> please?
> 

bug#29680: another grep-find anomalous behaviour

[Eli Zaretskii]

> Cc: 29680@debbugs.gnu.org
> From: Donald H Locker <dhlocker@comcast.net>
> Date: Fri, 15 Dec 2017 13:46:32 -0500
> 
> The underlying "unixy" environment is cygwin; updated a few weeks ago. 

What customizations do you have to go with that setup?  Any
customizations of shell-file-name or similar variables?

> The behaviour is very repeatable, though - at one level in the directory 
> hierarchy, the command fails to even execute find; at one higher level 
> in the directory hierarchy, the command succeeds and finds the sought 
> strings in the file.

It's hard to reason about your case without knowing which files are
present in each directory.  Would you mind to concoct a small test
case, where all directories and files in the tree to be searched by
'find' are explicitly spelled out?

> Note that in the Model_withEVSE directory, "find ."  doesn't seem to
> think that '.' is a directory

I think your interpretation of the error message is mistaken.  It says
"paths must precede expression", but that's because somehow '*.log' is
expanded into more than a single argument.  I suspect that the command
works in a directory with no files whose names match *.log, and
doesn't work where there are such files.  So I'd suggest to continue
looking into the quoting issue.  One possibility is to replace 'find'
with a program or a batch file which will just echo its command-line
arguments, and then see how Emacs invokes it.

bug#29680: another grep-find anomalous behaviour

[Donald H Locker]

It would appear that quoting of the '*.log' part of the command is at 
fault.  Changing the line from

   -name '*.log'
to

   -name \\*.log

allows the find to proceed normally. There are several files with .log 
extensions in the directory from which the find-grep fails.

To answer the other comments, televant customisations (all in 
(custom-set-variables ...)) are:

;; ignore line-wraps, please, in the following. in my .emacs, each is
;; really all on one line and I haven't figured out all of Thunderbird's
;; options yet

'(explicit-shell-file-name "c:/cygwin64/bin/bash")

'(exec-path
    (quote
     ("C:/cygwin64/bin" "C:/Program Files/Common Files/Microsoft 
Shared/Microsoft Online Services" "C:/Program Files (x86)/Common 
Files/Microsoft Shared/Microsoft Online Services" "C:/Windows/system32" 
"C:/Windows" "C:/Windows/System32/Wbem" 
"C:/Windows/System32/WindowsPowerShell/v1.0/" "C:/Program 
Files/Intel/WiFi/bin/" "C:/Program Files/Common 
Files/Intel/WirelessCommon/" "C:/Program Files/TortoiseSVN/bin" 
"c:/Users/dlocker/AppData/Roaming/local/bin/emacs-24.5-bin-i686-mingw32/libexec/emacs/24.5/i686-pc-mingw32" 
"c:/ProgramData/Oracle/Java/javapath")))

'(grep-command "grep -nHE ")

'(grep-find-command (quote ("find . -type f -exec grep -nHE  {} \\;" . 32)))

-- 
*Plain Text* email -- it's an accessibility issue
() no proprietary attachments; no html mail
/\ <http://www.georgedillon.com/web/html_email_is_evil.shtml>

On 15-Dec-2017 15:42, Eli Zaretskii wrote:
>> Cc: 29680@debbugs.gnu.org
>> From: Donald H Locker <dhlocker@comcast.net>
>> Date: Fri, 15 Dec 2017 13:46:32 -0500
>>
>> The underlying "unixy" environment is cygwin; updated a few weeks ago.
> 
> What customizations do you have to go with that setup?  Any
> customizations of shell-file-name or similar variables?
> 
>> The behaviour is very repeatable, though - at one level in the directory
>> hierarchy, the command fails to even execute find; at one higher level
>> in the directory hierarchy, the command succeeds and finds the sought
>> strings in the file.
> 
> It's hard to reason about your case without knowing which files are
> present in each directory.  Would you mind to concoct a small test
> case, where all directories and files in the tree to be searched by
> 'find' are explicitly spelled out?
> 
>> Note that in the Model_withEVSE directory, "find ."  doesn't seem to
>> think that '.' is a directory
> 
> I think your interpretation of the error message is mistaken.  It says
> "paths must precede expression", but that's because somehow '*.log' is
> expanded into more than a single argument.  I suspect that the command
> works in a directory with no files whose names match *.log, and
> doesn't work where there are such files.  So I'd suggest to continue
> looking into the quoting issue.  One possibility is to replace 'find'
> with a program or a batch file which will just echo its command-line
> arguments, and then see how Emacs invokes it.
> 

bug#29680: another grep-find anomalous behaviour

[Eli Zaretskii]

> Cc: 29680@debbugs.gnu.org
> From: Donald H Locker <dhlocker@comcast.net>
> Date: Mon, 18 Dec 2017 07:35:22 -0500
> 
> It would appear that quoting of the '*.log' part of the command is at 
> fault.  Changing the line from
> 
>    -name '*.log'
> to
> 
>    -name \\*.log
> 
> allows the find to proceed normally. There are several files with .log 
> extensions in the directory from which the find-grep fails.

Makes sense.

It's probably some subtle snafu with command-line quoting.  Were the
'qoutes' typed by you, or did Emacs produce them?

In any case, if you can spot where this quoting goes wrong, please
tell the details.  The combination of a native Windows Emacs and
Cygwin shell/utilities is relatively less well tested and has
subtleties, so I'm not surprised to hear about such problems.

bug#29680: another grep-find anomalous behaviour

[Donald H Locker]

I'm going to suggest this bug be held in abeyance until I test a few 
other issues; this may well be a cygwin behaviour, in retrospect.

I recently had to change all of my "egrep" to "grep -E" to address other 
issues, likely related to cygwin updates. My system log indicates 
updates of cygwin on 03Nov2017, 27Nov2017, 05Dec2017, and 15Dec2017. 
(This is the largest cluster of updates I have done within the last 
year.) Curiously, the 27Nov update included a w32api-runtime update, and 
this approximately coincides with the beginning of some of the troubles 
I am having. (I use find-grep _a_lot_ and don't remember ever having any 
problems until recently. I probably should have looked at my system log 
before raising a bug about emacs.)

To answer the question:

My typing was the source of the 'quotes' and the \\quoted-* (the default 
command presented in the minibuffer was

    find . -type f -exec grep -nHE  {} \;

I supply the "-name 'blahblah'" and the "regexp".)

Donald.
-- 
*Plain Text* email -- it's an accessibility issue
() no proprietary attachments; no html mail
/\ <http://www.georgedillon.com/web/html_email_is_evil.shtml>

On 18-Dec-2017 11:21, Eli Zaretskii wrote:
>> Cc: 29680@debbugs.gnu.org
>> From: Donald H Locker <dhlocker@comcast.net>
>> Date: Mon, 18 Dec 2017 07:35:22 -0500
>>
>> It would appear that quoting of the '*.log' part of the command is at
>> fault.  Changing the line from
>>
>>     -name '*.log'
>> to
>>
>>     -name \\*.log
>>
>> allows the find to proceed normally. There are several files with .log
>> extensions in the directory from which the find-grep fails.
> 
> Makes sense.
> 
> It's probably some subtle snafu with command-line quoting.  Were the
> 'qoutes' typed by you, or did Emacs produce them?
> 
> In any case, if you can spot where this quoting goes wrong, please
> tell the details.  The combination of a native Windows Emacs and
> Cygwin shell/utilities is relatively less well tested and has
> subtleties, so I'm not surprised to hear about such problems.
>