bug#68864: 30.0.50; project-find-regexp fails on Alpine

bug#68864: 30.0.50; project-find-regexp fails on Alpine

[john muhl via Bug reports for GNU Emacs, the Swiss army knife of text editors]

The grep on Alpine does not support the --null option.

  $ grep --null test *
  grep: unrecognized option: null
  BusyBox v1.36.1 (2024-01-16 17:10:30 UTC) multi-call binary.

To reproduce:

  emacs -Q
  M-: (project-find-regexp "test")

Debugger entered--Lisp error: (user-error "Search failed with status 123: grep: unrecognized option: null")
  signal(user-error ("Search failed with status 123: grep: unrecognized option: null"))
  user-error("Search failed with status %d: %s" 123 "grep: unrecognized option: null")
  xref-matches-in-files("test" ("/home/build/emacs/.clang-format" ...))
  project--find-regexp-in-files("test" ("/home/build/emacs/.clang-format" ...))
  apply(project--find-regexp-in-files ("test" ("/home/build/emacs/.clang-format" ...))
  #f(compiled-function (&rest args2) #<bytecode -0x47b2fa0835514c4>)()
  xref--show-xref-buffer(#f(compiled-function (&rest args2) #<bytecode -0x47b2fa0835514c4>) ((window . #<window 1 on *scratch*>) (display-action) (auto-jump)))
  xref--show-xrefs(#f(compiled-function (&rest args2) #<bytecode -0x47b2fa0835514c4>) nil)
  xref-show-xrefs(#f(compiled-function (&rest args2) #<bytecode -0x47b2fa0835514c4>) nil)
  project-find-regexp("test")
  eval((project-find-regexp "test") t)
  #f(compiled-function () #<bytecode -0xca5776b8f06f719>)()
  #f(compiled-function () #<bytecode -0xb488c244da63e0f>)()
  handler-bind-1(#f(compiled-function () #<bytecode -0xb488c244da63e0f>) (error) eval-expression--debug)
  eval-expression((project-find-regexp "test") nil nil 127)
  funcall-interactively(eval-expression (project-find-regexp "test") nil nil 127)
  call-interactively(eval-expression nil nil)
  command-execute(eval-expression)


In GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-musl) of 2024-02-01 built
 on build
Repository revision: cd2c45a3890601e1bc498c81e64791fead6efc86
Repository branch: master
System Description: Alpine Linux edge

Configured using:
 'configure --without-all --without-x'

Configured features:
GMP PDUMPER

Important settings:
  value of $LC_COLLATE: C
  value of $LANG: C.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dnd dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg
rfc6068 epg-config gnus-util time-date subr-x mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils help-fns
radix-tree cl-print debug backtrace help-mode find-func files-x vc-git
diff-mode easy-mmode vc-dispatcher cl-seq grep compile
text-property-search comint regexp-opt ansi-osc ansi-color tool-bar xref
ring project cl-loaddefs cl-lib term/tmux term/xterm xterm byte-opt gv
bytecomp byte-compile rmc iso-transl tooltip cconv eldoc paren electric
uniquify ediff-hook vc-hooks lisp-float-type elisp-mode tabulated-list
replace newcomment text-mode lisp-mode prog-mode register page tab-bar
menu-bar rfn-eshadow isearch easymenu timer select mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer nadvice seq
simple cl-generic indonesian philippine cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite emoji-zwj charscript charprop case-table
epa-hook jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button
loaddefs theme-loaddefs faces cus-face macroexp files window
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget keymap hashtable-print-readable backquote multi-tty
make-network-process emacs)

Memory information:
((conses 16 85101 9311) (symbols 48 7887 0) (strings 32 28128 1973)
 (string-bytes 1 941684) (vectors 16 11619)
 (vector-slots 8 103879 5456) (floats 8 34 13825)
 (intervals 56 1202 24) (buffers 976 14))

bug#68864: 30.0.50; project-find-regexp fails on Alpine

[Dmitry Gutov]

On 01/02/2024 05:38, john muhl via Bug reports for GNU Emacs, the Swiss 
army knife of text editors wrote:
> The grep on Alpine does not support the --null option.
> 
>    $ grep --null test *
>    grep: unrecognized option: null
>    BusyBox v1.36.1 (2024-01-16 17:10:30 UTC) multi-call binary.
> 
> To reproduce:
> 
>    emacs -Q
>    M-: (project-find-regexp "test")
> 
> Debugger entered--Lisp error: (user-error "Search failed with status 123: grep: unrecognized option: null")
>    signal(user-error ("Search failed with status 123: grep: unrecognized option: null"))

Hi!

That's a problem: apparently it does indeed not support --null or -Z: 
https://boxmatrix.info/wiki/Property:grep

There is another flag we could use, which seems to have a similar enough 
effect: -z. But from what I can tell, it would make OpenBSD unsupported: 
https://man.openbsd.org/grep

Perhaps it would be best to just file a feature request for busybox's 
support for --null/-Z. Better ideas welcome.

In the meantime, you can customize the entry for 'grep' in 
xref-search-program-alist to use -z.

bug#68864: 30.0.50; project-find-regexp fails on Alpine

[john muhl via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Dmitry Gutov <dmitry@gutov.dev> writes:

> On 01/02/2024 05:38, john muhl via Bug reports for GNU Emacs, the
> Swiss army knife of text editors wrote:
>> The grep on Alpine does not support the --null option.
>>    $ grep --null test *
>>    grep: unrecognized option: null
>>    BusyBox v1.36.1 (2024-01-16 17:10:30 UTC) multi-call binary.
>> To reproduce:
>>    emacs -Q
>>    M-: (project-find-regexp "test")
>> Debugger entered--Lisp error: (user-error "Search failed with status
>> 123: grep: unrecognized option: null")
>>    signal(user-error ("Search failed with status 123: grep: unrecognized option: null"))
>
> Hi!
>
> That's a problem: apparently it does indeed not support --null or -Z:
> https://boxmatrix.info/wiki/Property:grep
>
> There is another flag we could use, which seems to have a similar
> enough effect: -z. But from what I can tell, it would make OpenBSD
> unsupported: https://man.openbsd.org/grep

Even -z is a compile time option:

  $ busybox grep -h
  BusyBox v1.36.1 (2024-01-16 17:10:30 UTC) multi-call binary.

  Usage: grep [-HhnlLoqvsrRiwFE] [-m N] [-A|B|C N] …

https://git.busybox.net/busybox/tree/findutils/grep.c#n85

I couldn’t get busybox to compile with the EXTRA_COMPAT option
so can’t say if it helps or not.

> Perhaps it would be best to just file a feature request for busybox's
> support for --null/-Z. Better ideas welcome.
>
> In the meantime, you can customize the entry for 'grep' in
> xref-search-program-alist to use -z.

Would it be possible to have xref check
grep-use-null-filename-separator and only use --null when
available? I tried with this crude patch and the results look
good enough:

  --- a/lisp/progmodes/xref.el
  +++ b/lisp/progmodes/xref.el
  @@ -1855,7 +1855,11 @@ xref-search-program-alist
       `((grep
          .
          ;; '-s' because 'git ls-files' can output broken symlinks.
  -       ,(concat "xargs -0 " xargs-max-chars "grep <C> --null -snHE -e <R>"))
  +       ,(concat "xargs -0 " xargs-max-chars "grep <C>"
  +                (when (and (grep-compute-defaults)
  +                           grep-use-null-filename-separator)
  +                  " --null")
  +                " -snHE -e <R>"))
         (ripgrep
          .
          ;; '!*/' is there to filter out dirs (e.g. submodules).

bug#68864: 30.0.50; project-find-regexp fails on Alpine

[Dmitry Gutov]

On 01/02/2024 19:13, john muhl wrote:
> Dmitry Gutov <dmitry@gutov.dev> writes:
> 
>> On 01/02/2024 05:38, john muhl via Bug reports for GNU Emacs, the
>> Swiss army knife of text editors wrote:
>>> The grep on Alpine does not support the --null option.
>>>     $ grep --null test *
>>>     grep: unrecognized option: null
>>>     BusyBox v1.36.1 (2024-01-16 17:10:30 UTC) multi-call binary.
>>> To reproduce:
>>>     emacs -Q
>>>     M-: (project-find-regexp "test")
>>> Debugger entered--Lisp error: (user-error "Search failed with status
>>> 123: grep: unrecognized option: null")
>>>     signal(user-error ("Search failed with status 123: grep: unrecognized option: null"))
>>
>> Hi!
>>
>> That's a problem: apparently it does indeed not support --null or -Z:
>> https://boxmatrix.info/wiki/Property:grep
>>
>> There is another flag we could use, which seems to have a similar
>> enough effect: -z. But from what I can tell, it would make OpenBSD
>> unsupported: https://man.openbsd.org/grep
> 
> Even -z is a compile time option:
> 
>    $ busybox grep -h
>    BusyBox v1.36.1 (2024-01-16 17:10:30 UTC) multi-call binary.
> 
>    Usage: grep [-HhnlLoqvsrRiwFE] [-m N] [-A|B|C N] …
> 
> https://git.busybox.net/busybox/tree/findutils/grep.c#n85

That's... unexpected.

> I couldn’t get busybox to compile with the EXTRA_COMPAT option
> so can’t say if it helps or not.
> 
>> Perhaps it would be best to just file a feature request for busybox's
>> support for --null/-Z. Better ideas welcome.
>>
>> In the meantime, you can customize the entry for 'grep' in
>> xref-search-program-alist to use -z.
> 
> Would it be possible to have xref check
> grep-use-null-filename-separator and only use --null when
> available? I tried with this crude patch and the results look
> good enough:
> 
>    --- a/lisp/progmodes/xref.el
>    +++ b/lisp/progmodes/xref.el
>    @@ -1855,7 +1855,11 @@ xref-search-program-alist
>         `((grep
>            .
>            ;; '-s' because 'git ls-files' can output broken symlinks.
>    -       ,(concat "xargs -0 " xargs-max-chars "grep <C> --null -snHE -e <R>"))
>    +       ,(concat "xargs -0 " xargs-max-chars "grep <C>"
>    +                (when (and (grep-compute-defaults)
>    +                           grep-use-null-filename-separator)
>    +                  " --null")
>    +                " -snHE -e <R>"))
>           (ripgrep
>            .
>            ;; '!*/' is there to filter out dirs (e.g. submodules).

That would slow down the loading of xref.el, which seems unfortunate, 
even if it's only by 0.05s (just measured on my machine; on many others 
it will be more).

What we could do, I suppose, is check whether 
grep-use-null-filename-separator is set to some explicit value (not 
'auto-detect'), and if so, apply it. That would work if xref.el is 
loaded late, or if you customized the variable explicitly.

I was also thinking of adding another template parameter like <N>, but 
it would be used for 'grep'--would seem odd.

We also could just remove "--null" inside xref-matches-in-files from 
grep's command line if the variable is set to this or that. It would be 
unfortunate to encounter some false positives, though (though I can't 
imagine them now).