bug#54017: add regexp translation option to read-regexp

bug#54017: add regexp translation option to read-regexp

[emacsq via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Suppose, the user wants to use ( ) for capturing and
\( \) for matching parens. He provides a function
which does this translation.

If this new option is set and is a function which does
the translation then read-regexp will

- accept the format which the user prefers, in this case
( ... ) for capturing.

- store regexps in this format in history, so the user
can work with the user's preferred format even when
retrieving history entries.

- return the elisp format regexp by calling the function
provided by the user which does the translation.

bug#54017: add regexp translation option to read-regexp

[Juri Linkov]

> Suppose, the user wants to use ( ) for capturing and
> \( \) for matching parens. He provides a function
> which does this translation.
>
> If this new option is set and is a function which does
> the translation then read-regexp will
>
> - accept the format which the user prefers, in this case
> ( ... ) for capturing.
>
> - store regexps in this format in history, so the user
> can work with the user's preferred format even when
> retrieving history entries.
>
> - return the elisp format regexp by calling the function
> provided by the user which does the translation.

This user option already exists.  It's name is 'search-default-mode'.
For stand-alone query-replace it's called 'replace-regexp-function'.

So we just need someone to write these regexp translation functions.

bug#54017: add regexp translation option to read-regexp

[emacsq via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> This user option already exists. It's name is 'search-default-mode'.

Does that option affect every command where read regexp is used?
Because this bug is about that.

So regexp translation for occur, flush/keep-lines, highlight-regexp,
dired-mark-files-regexp, etc., so anywhere where read-regexp is used.

bug#54017: add regexp translation option to read-regexp

[Juri Linkov]

>> This user option already exists. It's name is 'search-default-mode'.
>
> Does that option affect every command where read regexp is used?
> Because this bug is about that.
>
> So regexp translation for occur, flush/keep-lines, highlight-regexp,
> dired-mark-files-regexp, etc., so anywhere where read-regexp is used.

Sorry, I misread your feature request, I thought it's continuation
of adding custom regexp types to search/replace.  You are right
that read-regexp misses support for custom regexp formats.

It's easy to add regexp translation to the return value of read-regexp.
But there is one complication: the default value returned by
read-regexp-defaults-function might be a function like
find-tag-default-as-regexp that returns a regexp in the
default format created by regexp-quote.

Do you agree that a pair of two translation functions should be provided:
one to translate a custom regexp syntax to the default regexp syntax
(to be used on the return value of read-regexp), and another translation
from the default regexp syntax to the custom regexp syntax (to be used
to translate the default regexp value added to the minibuffer)?

bug#54017: add regexp translation option to read-regexp

[Juri Linkov]

[-- Attachment #1: Type: text/plain, Size: 240 bytes --]

> It's easy to add regexp translation to the return value of read-regexp.

This patch allows using Rx syntax, so for example, after running 'occur',
you can type an Rx expression:

  List lines matching regexp: (rx (or "e.g." "i.e.") " ")


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: regexp-from-function.patch --]
[-- Type: text/x-diff, Size: 1695 bytes --]

diff --git a/lisp/emacs-lisp/rx.el b/lisp/emacs-lisp/rx.el
index aa2486b47e..b1e726d025 100644
--- a/lisp/emacs-lisp/rx.el
+++ b/lisp/emacs-lisp/rx.el
@@ -1479,6 +1479,10 @@ rx
 ;; Obsolete internal symbol, used in old versions of the `flycheck' package.
 (define-obsolete-function-alias 'rx-submatch-n 'rx-to-string "27.1")
 
+(defun regexp-from-rx (string)
+  "This translation function can be used by `regexp-from-function'."
+  (rx--to-expr (cons 'seq (cdr (read string)))))
+
 (provide 'rx)
 
 ;;; rx.el ends here
diff --git a/lisp/replace.el b/lisp/replace.el
index 06be597855..a8b850a9ed 100644
--- a/lisp/replace.el
+++ b/lisp/replace.el
@@ -819,6 +819,15 @@ occur-highlight-overlays
 (defvar occur-collect-regexp-history '("\\1")
   "History of regexp for occur's collect operation.")
 
+(defcustom regexp-from-function nil
+  "Function to translate from a custom regexp to the default regexp syntax."
+  :type '(choice
+          (const :tag "No translation" nil)
+          (function-item :tag "RX" regexp-from-rx)
+          (function :tag "Your choice of function"))
+  :group 'matching
+  :version "29.1")
+
 (defcustom read-regexp-defaults-function nil
   "Function that provides default regexp(s) for `read-regexp'.
 This function should take no arguments and return one of: nil, a
@@ -923,7 +932,9 @@ read-regexp
 	  (when default
 	    (add-to-history (or history 'regexp-history) default)))
       ;; Otherwise, add non-empty input to the history and return input.
-      (prog1 input
+      (prog1 (if (functionp regexp-from-function)
+                 (funcall regexp-from-function input)
+               input)
 	(add-to-history (or history 'regexp-history) input)))))
 
 

bug#54017: add regexp translation option to read-regexp

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Date: Thu, 17 Feb 2022 10:24:14 +0200
> Cc: 54017@debbugs.gnu.org
> 
> It's easy to add regexp translation to the return value of read-regexp.
> But there is one complication: the default value returned by
> read-regexp-defaults-function might be a function like
> find-tag-default-as-regexp that returns a regexp in the
> default format created by regexp-quote.
> 
> Do you agree that a pair of two translation functions should be provided:
> one to translate a custom regexp syntax to the default regexp syntax
> (to be used on the return value of read-regexp), and another translation
> from the default regexp syntax to the custom regexp syntax (to be used
> to translate the default regexp value added to the minibuffer)?

Before we install something like this, we need to discuss the relevant
use cases and agree that it makes sense for us to support them.

I originally interpreted the OP's request as asking for a feature
where the user could use one of the regexp styles used by other
programs, such as BRE or ERE or maybe PCRE.  If this is indeed the
intent, then I'd rather we implemented support for only those specific
styles (or some subset of them).  It doesn't make sense to me to
support arbitrary translations of regular expressions, because I see
no valid use cases for such a general feature, and am unaware of any
other applications which support regular expressions that allow such
arbitrary translations.

bug#54017: add regexp translation option to read-regexp

[emacsq via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> I originally interpreted the OP's request as asking for a feature
> where the user could use one of the regexp styles used by other
> programs, such as BRE or ERE or maybe PCRE.

If it provides me a more friendly regexp syntax where I don't have to
qoute parens, alternations then I'm OK with it, though I guess
providing a full translation for certain styles is much more work,
so giving an option for translation and letting the user write his
own function according to his taste may be the simpler solution which
can even support different custom regexp styles later.

bug#54017: add regexp translation option to read-regexp

[emacsq via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> a pair of two translation functions should be provided:
> one to translate a custom regexp syntax to the default regexp syntax
>(to be used on the return value of read-regexp), and another translation
> from the default regexp syntax to the custom regexp syntax (to be used
> to translate the default regexp value added to the minibuffer)?

The purpose of regexp translation is that all user facing regexps by
read-regexp should use the translated format, so if there is a need
for a function which converts to the convenient format then the
user should provide that as well. Maybe there should be one function only
with  an argument specifying the direction of the translation.

BTW, ideally the variables you mentioned ('search-default-mode' and
'replace-regexp-function') should be merged with this option, because
it's unlikely the user wants one regexp syntax for one command and a
different syntax for an other one, so there should be a single
translation function which works everywhere where user facing
regexps occur.

bug#54017: add regexp translation option to read-regexp

[Juri Linkov]

> Before we install something like this, we need to discuss the relevant
> use cases and agree that it makes sense for us to support them.
>
> I originally interpreted the OP's request as asking for a feature
> where the user could use one of the regexp styles used by other
> programs, such as BRE or ERE or maybe PCRE.  If this is indeed the
> intent, then I'd rather we implemented support for only those specific
> styles (or some subset of them).  It doesn't make sense to me to
> support arbitrary translations of regular expressions, because I see
> no valid use cases for such a general feature, and am unaware of any
> other applications which support regular expressions that allow such
> arbitrary translations.

Then need to wait when someone will step forward to write these
BRE/ERE/PCRE translation functions.

bug#54017: add regexp translation option to read-regexp

[emacsq via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> Then need to wait when someone will step forward to write these
> BRE/ERE/PCRE translation functions.

This is a bigger job and is there a current need for it?

People just want to make their life easier and change the way
how the syntax of some of the symbols often used by them works
in interactive input (e.g. grouping).

If we wait for those complete translation functions then we
might have to wait for a long time, unless there are people
who want complete translations right now, but there were no
such volunteers in this thread for that task.

Adding an option for a custom translation function solves the
problem right know, everyone can change the symbols they are
most annoyed by and this function can be the base mechanism
for complete translations later if someone ever wants to do
those.

bug#54017: add regexp translation option to read-regexp

[Eli Zaretskii]

> Date: Sun, 20 Feb 2022 19:06:53 +0000
> From: emacsq <laszlomail@protonmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>, 54017@debbugs.gnu.org
> 
> > Then need to wait when someone will step forward to write these
> > BRE/ERE/PCRE translation functions.
> 
> This is a bigger job and is there a current need for it?
> 
> People just want to make their life easier and change the way
> how the syntax of some of the symbols often used by them works
> in interactive input (e.g. grouping).

Like I said: I don't think we should support arbitrary
transformations of regular expressions' syntax.  It makes no sense to
me.


And we don't need to support _all_ of the above syntaxes, we could
support just the BRE or just the ERE.

bug#54017: [External] : bug#54017: add regexp translation option to read-regexp

[Drew Adams]

(Probably at least somewhat off-topic.)

I'd like to see Emacs regexp handling support
a way to either (1) specify a new, multiline
"dot" syntax that's equivalent to \(.\|[\n]\)
or (2) have a (dynamic) variable, or a mode,
that you can toggle to switch the behavior of
ordinary dot to multiline dot.
___

In Icicles I have a command that toggles the
behavior in the minibuffer.  Users see `.',
but under the covers \(.\|[\n]\) is used.
Editing (e.g. char deletion) automatically
treats that regexp as if it were just the
single char `.'.

When a `.' in the minibuffer actually stands
for \(.\|[\n]\) it gets highlighted, to
remind you of its behavior.

This is admittedly just a hack.  It would be
much better for Emacs itself to provide for
multiline (i.e. true any-char) dot matching.