bug#23451: 25.0.93; Clarify the dependency on find/grep for platforms not having those tools

bug#23451: 25.0.93; Clarify the dependency on find/grep for platforms not having those tools

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 5657 bytes --]

From: kmodi@ulcf41.cld.analog.com (Kaushal.Modi)
To: bug-gnu-emacs@gnu.org
Subject:
--text follows this line--

The changes to the A/Q bindings in dired as discussed and confirmed are not
immediately compatible on Windows

Ref: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=23426#49

Even understanding that users would need to install GNU find & grep on
their Windows system to use the new implementations bound to A/Q in dired,
I believe that we should have the following:

- NOT bind A/Q at all if the right dependencies are not found. I tried the
A binding on Windows, it looked like it was grepping for the strings I
entered and returned an empty *xref* window. The same search on same files
worked as expected in RHEL (to be honest I love this new feature on RHEL,
and I might start using the A binding). Currently the implementation on
Windows gives an appearance that something was searched for and no results
were found. That is misleading!

Possible solution?

(when (correct-version-of-find-and-grep-found-p)
   (define-key dired-mode-map (kbd "A") #'dired-do-find-regexp)
   (define-key dired-mode-map (kbd "A") #'dired-do-find-regexp-and-replace))

- Another alternative would be (if we want to keep A/Q bindings) that a
user-error or error be thrown if the correct external dependencies are not
installed. The user should be let known that they need to install the GNU
find/grep executables for their platform in order to use those commands. In
the current implementation, the user will just assume that they searched
something and nothing got returned.

- The requirement to have find/grep installed should also go to backward
incompatible changes section in NEWS.

(I got an idea of "incompatible change" section in NEWS from this recent
commit:
http://git.savannah.gnu.org/cgit/emacs.git/commit/?h=emacs-25&id=c68a09107c1f7459c626d38be5e0e991912e57ec
 )

I would suggest that this bug be made blocking for the release of 25.1.

For Windows users, the bindings change for A/Q keys in dired is not
apparent to the user. At the very least, an error should be thrown if the
correct external dependencies (GNU version of find/grep) are not found on
the system PATH.



In GNU Emacs 25.0.93.4 (x86_64-unknown-linux-gnu, GTK+ Version 2.24.23)
 of 2016-05-04 built on ..
Repository revision: adc80b7e238e09b1b8c392ecf902d2b978d9016d
Windowing system distributor 'The X.Org Foundation', version 11.0.60900000
System Description: Red Hat Enterprise Linux Workstation release 6.6
(Santiago)

Configured using:
 'configure --with-modules
 --prefix=/home/kmodi/usr_local/apps/6/emacs/emacs-25
 'CPPFLAGS=-fgnu89-inline -I/home/kmodi/usr_local/6/include
 -I/usr/include/freetype2 -I/usr/include' 'CFLAGS=-ggdb3 -O0'
 'CXXFLAGS=-ggdb3 -O0' 'LDFLAGS=-L/home/kmodi/usr_local/6/lib
 -L/home/kmodi/usr_local/6/lib64 -ggdb3'
 PKG_CONFIG_PATH=/home/kmodi/usr_local/6/lib/pkgconfig:/home/kmodi/usr_local/6/lib64/pkgconfig:/cad/adi/apps/gnu/linux/x86_64/6/lib/pkgconfig:/cad/adi/apps/gnu/linux/x86_64/6/lib64/pkgconfig:/usr/lib/pkgconfig:/usr/lib64/pkgconfig:/usr/share/pkgconfig:/lib/pkgconfig:/lib64/pkgconfig'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GCONF GSETTINGS
NOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK2 X11 MODULES

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=none
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Making completion list...

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message dired format-spec rfc822 mml
mml-sec password-cache epg epg-config gnus-util mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util help-fns help-mode easymenu
cl-loaddefs pcase cl-lib mail-prsvr mail-utils time-date mule-util
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt
fringe tabulated-list newcomment elisp-mode lisp-mode prog-mode register
page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core frame cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese charscript case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer cl-preloaded nadvice
loaddefs button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote dbusbind inotify dynamic-setting
system-font-setting font-render-setting move-toolbar gtk x-toolkit x
multi-tty make-network-process emacs)

Memory information:
((conses 16 87685 9438)
 (symbols 48 19756 0)
 (miscs 40 40 172)
 (strings 32 14551 3894)
 (string-bytes 1 435236)
 (vectors 16 12373)
 (vector-slots 8 433753 3061)
 (floats 8 168 95)
 (intervals 56 243 0)
 (buffers 976 12)
 (heap 1024 36573 680))

-- 

-- 
Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 7508 bytes --]

bug#23451: 25.0.93; Clarify the dependency on find/grep for platforms not having those tools

[Dmitry Gutov]

On 5/4/16 9:02 PM, Kaushal Modi wrote:
> Another alternative would be (if we want to keep A/Q bindings) that a
> user-error or error be thrown if the correct external dependencies are not
> installed. The user should be let known that they need to install the GNU
> find/grep executables for their platform in order to use those commands.

I've made a step toward this in commit 3bc3dc4: if the status is not 
zero and the process made some output, we signal a user error with 
whatever output we have.

It's not exactly installation instructions, but it will at least tell 
the user that something is wrong.

Previously, I had some problems using this approach, but failed to 
document them properly. Let's see if someone manages to find them again. 
Hopefully, the only thing missing before was the (/= (point-min) 
(point-max)) check.

bug#23451: 25.0.93; Clarify the dependency on find/grep for platforms not having those tools

[Dmitry Gutov]

On 5/29/17 3:07 AM, Dmitry Gutov wrote:

> Previously, I had some problems using this approach, but failed to 
> document them properly. Let's see if someone manages to find them again.

Aaand here it is:

1. Enter the Emacs checkout, pull the latest commit, 'make' to make sure 
the new change is preloaded.

2. emacs -Q.

3. M-x project-find-regexp, enter 'turn-on-eldoc-mode'.

4. See the user error about status 1, and the message containing four 
grep hits. Doing this search with Ag confirms the same set of hits (with 
normal exit status).

Could someone please explain why find-grep exits with status 1 here?

It's not the command line invocation, or else we'd be getting this exit 
status for other similar inputs.

bug#23451: 25.0.93; Clarify the dependency on find/grep for platforms not having those tools

[npostavs]

Dmitry Gutov <dgutov@yandex.ru> writes:

> 4. See the user error about status 1, and the message containing four
> grep hits. Doing this search with Ag confirms the same set of hits
> (with normal exit status).
>
> Could someone please explain why find-grep exits with status 1 here?

I see the same exit status when running find ... -exec grep ... + from
the shell.  It seems that -exec <cmd> + will cause 'find' to exit with
status 1 if the <cmd> exits with status 1:

    ~$ find . -exec false '{}' + ; echo $?
    1
    ~$ find . -exec true '{}' + ; echo $?
    0

So when <cmd> is grep, and the number of files is greater than the
command line length limit, the exit status is effectively random (it
depends on 'find' decides to group the batches of files it passes to
'grep').

bug#23451: 25.0.93; Clarify the dependency on find/grep for platforms not having those tools

[Eli Zaretskii]

On May 29, 2017 5:58:08 AM GMT+03:00, npostavs@users.sourceforge.net wrote:
>Dmitry Gutov <dgutov@yandex.ru> writes:
>
>> 4. See the user error about status 1, and the message containing four
>> grep hits. Doing this search with Ag confirms the same set of hits
>> (with normal exit status).
>>
>> Could someone please explain why find-grep exits with status 1 here?
>
>I see the same exit status when running find ... -exec grep ... + from
>the shell.  It seems that -exec <cmd> + will cause 'find' to exit with
>status 1 if the <cmd> exits with status 1:
>
>    ~$ find . -exec false '{}' + ; echo $?
>    1
>    ~$ find . -exec true '{}' + ; echo $?
>    0
>
>So when <cmd> is grep, and the number of files is greater than the
>command line length limit, the exit status is effectively random (it
>depends on 'find' decides to group the batches of files it passes to
>'grep').

Isn't this simply the consequence of grep returning non-zero status
for files that didn't match?

bug#23451: 25.0.93; Clarify the dependency on find/grep for platforms not having those tools

[Dmitry Gutov]

On 5/29/17 7:19 AM, Eli Zaretskii wrote:

> Isn't this simply the consequence of grep returning non-zero status
> for files that didn't match?

The consequence of batching, I'd say. I wasn't 100% clear that Grep is 
called more than once with this invocation style.

bug#23451: 25.0.93; Clarify the dependency on find/grep for platforms not having those tools

[Dmitry Gutov]

On 5/29/17 5:58 AM, npostavs@users.sourceforge.net wrote:

> So when <cmd> is grep, and the number of files is greater than the
> command line length limit, the exit status is effectively random (it
> depends on 'find' decides to group the batches of files it passes to
> 'grep').

Can we make Grep exit with 0 no matter if it matched something or not? 
So that non-zero means an actual problem.

There is no suitable command-line option, it seems, but maybe there's a 
shell-based option? Like 'grep ... || exit 0', but I don't think we want 
to swallow exit statuses higher than 1.

Alternatively, I suppose we can learn to determine that the user has no 
working find and grep inside grep-compute-defaults, and possibly avoid 
setting grep-find-template in that case.

bug#23451: 25.0.93; Clarify the dependency on find/grep for platforms not having those tools

[npostavs]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 5/29/17 5:58 AM, npostavs@users.sourceforge.net wrote:
>
>> So when <cmd> is grep, and the number of files is greater than the
>> command line length limit, the exit status is effectively random (it
>> depends on 'find' decides to group the batches of files it passes to
>> 'grep').
>
> Can we make Grep exit with 0 no matter if it matched something or not?
> So that non-zero means an actual problem.
>
> There is no suitable command-line option, it seems, but maybe there's
> a shell-based option? Like 'grep ... || exit 0', but I don't think we
> want to swallow exit statuses higher than 1.

This seems to work:

    find ... -exec sh -c 'grep -i -E -nH -e turn-on-eldoc-mode "$@" ; [ $? -le 1 ]' sh '{}' +

bug#23451: 25.0.93; Clarify the dependency on find/grep for platforms not having those tools

[Eli Zaretskii]

> From: npostavs@users.sourceforge.net
> Date: Mon, 29 May 2017 08:43:58 -0400
> Cc: 23451@debbugs.gnu.org, Kaushal Modi <kaushal.modi@gmail.com>
> 
> This seems to work:
> 
>     find ... -exec sh -c 'grep -i -E -nH -e turn-on-eldoc-mode "$@" ; [ $? -le 1 ]' sh '{}' +

Who said sh is available?

I think we should simply test explicitly for 'find' and 'grep' being
available, before we run the command.  Otherwise, it sounds like we
will increase the complexity and the fragility of the feature, to very
little gain.

bug#23451: 25.0.93; Clarify the dependency on find/grep for platforms not having those tools

[Dmitry Gutov]

On 5/29/17 3:43 PM, npostavs@users.sourceforge.net wrote:

> This seems to work:
> 
>      find ... -exec sh -c 'grep -i -E -nH -e turn-on-eldoc-mode "$@" ; [ $? -le 1 ]' sh '{}' +

Thank you for the attempt, but maybe I was asking too much. Seems like 
we can't rely on sh being available on all supported platforms.

I'm inclined to go with a regexp-based approach for now. "program not 
found" errors rarely look like Grep output.

bug#23451: 25.0.93; Clarify the dependency on find/grep for platforms not having those tools

[Dmitry Gutov]

On 5/29/17 8:35 PM, Eli Zaretskii wrote:

> I think we should simply test explicitly for 'find' and 'grep' being
> available, before we run the command.

Windows comes with find.exe, though, one we can't use.

 > Otherwise, it sounds like we
 > will increase the complexity and the fragility of the feature, to very
 > little gain.

Agreed, I was hoping for a solution that would more or less fit one of 
the templates defined in grep.el.

See commit 4886b2e for something entirely different.

bug#23451: 25.0.93; Clarify the dependency on find/grep for platforms not having those tools

[Eli Zaretskii]

> Cc: 23451@debbugs.gnu.org, kaushal.modi@gmail.com
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Tue, 30 May 2017 01:00:18 +0300
> 
> On 5/29/17 8:35 PM, Eli Zaretskii wrote:
> 
> > I think we should simply test explicitly for 'find' and 'grep' being
> > available, before we run the command.
> 
> Windows comes with find.exe, though, one we can't use.

Yes, I meant to test it's the find we expect, not the one which comes
with Windows out of the box.  Which means not just executable-find,
but something a bit more sophisticated.

bug#23451: 25.0.93; Clarify the dependency on find/grep for platforms not having those tools

[Eli Zaretskii]

> Cc: 23451@debbugs.gnu.org, kaushal.modi@gmail.com
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Tue, 30 May 2017 01:00:18 +0300
> 
> See commit 4886b2e for something entirely different.

Thanks, looks good to me.

bug#23451: 25.0.93; Clarify the dependency on find/grep for platforms not having those tools

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 476 bytes --]

On Tue, May 30, 2017 at 4:11 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > Cc: 23451@debbugs.gnu.org, kaushal.modi@gmail.com
> > From: Dmitry Gutov <dgutov@yandex.ru>
> > Date: Tue, 30 May 2017 01:00:18 +0300
> >
> > See commit 4886b2e for something entirely different.
>
> Thanks, looks good to me.
>

Closing this bug was long overdue .. was fixed in
http://git.savannah.gnu.org/cgit/emacs.git/commit/?h=emacs-26&id=4886b2ed52249597d1ea638f20c0ceb689075e72

-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 1182 bytes --]