bug#31953: feature request - `highlight-rx` interactively

bug#31953: feature request - `highlight-rx` interactively

[Van L]

Hello.

Towards the transition to rx from regexp
may I suggest `highlight-rx` for `highlight-regexp`.

A usecase is to mark a region or string of few words
and to call `highlight-rx` which does a verbatim highlight.

bug#31953: feature request - `highlight-rx` interactively

[Lars Ingebrigtsen]

Van L <van@scratch.space> writes:

> Towards the transition to rx from regexp
> may I suggest `highlight-rx` for `highlight-regexp`.
>
> A usecase is to mark a region or string of few words
> and to call `highlight-rx` which does a verbatim highlight.

Sorry, I'm not sure I understand what you're requesting here.

highlight-regexp is an alias for hi-lock-face-buffer which just does
some highlighting, as far as I can see.  

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#31953: [VanL] Re: bug#31953: feature request - `highlight-rx` interactively

[Lars Ingebrigtsen]

-------------------- Start of forwarded message --------------------
From: VanL <van@scratch.space>
To: Lars Ingebrigtsen <larsi@gnus.org>
Subject: Re: bug#31953: feature request - `highlight-rx` interactively
Date: Thu, 10 Oct 2019 18:15:38 +1100


>> A usecase is to mark a region or string of few words
>> and to call `highlight-rx` which does a verbatim highlight.
>
>Sorry, I'm not sure I understand what you're requesting here.
>
>highlight-regexp is an alias for hi-lock-face-buffer which just does
>some highlighting, as far as I can see.  

I was wanting to migrate from regexp to rx syntax for highlighting.
-------------------- End of forwarded message --------------------

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#31953: [VanL] Re: bug#31953: feature request - `highlight-rx` interactively

[Lars Ingebrigtsen]

(Please keep the debbugs address in the Cc list -- otherwise the bug
tracker won't get the message.)

>>> A usecase is to mark a region or string of few words
>>> and to call `highlight-rx` which does a verbatim highlight.
>>
>>Sorry, I'm not sure I understand what you're requesting here.
>>
>>highlight-regexp is an alias for hi-lock-face-buffer which just does
>>some highlighting, as far as I can see.  
>
> I was wanting to migrate from regexp to rx syntax for highlighting.

`rx' returns a normal regexp, so you can use whatever you want, can't
you?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#31953: [VanL] Re: bug#31953: feature request - `highlight-rx` interactively

[Phil Sainty]

On 2019-10-15 13:35, Lars Ingebrigtsen wrote:
>> I was wanting to migrate from regexp to rx syntax for highlighting.
> 
> `rx' returns a normal regexp, so you can use whatever you want, can't
> you?

I think the idea is to be able to *interactively* enter rx syntax
and have it highlighted (as opposed to `highlight-regexp' which will
obviously treat what you enter interactively as an actual regexp,
rather than something to be transformed into one).

i.e. These would be equivalent:

M-x highlight-rx RET (or "foo" "bar") RET
M-x highlight-regexp RET \(foo\|bar\) RET

bug#31953: [VanL] Re: bug#31953: feature request - `highlight-rx` interactively

[Lars Ingebrigtsen]

Phil Sainty <psainty@orcon.net.nz> writes:

> I think the idea is to be able to *interactively* enter rx syntax
> and have it highlighted (as opposed to `highlight-regexp' which will
> obviously treat what you enter interactively as an actual regexp,
> rather than something to be transformed into one).
>
> i.e. These would be equivalent:
>
> M-x highlight-rx RET (or "foo" "bar") RET
> M-x highlight-regexp RET \(foo\|bar\) RET

Oh, I see.

I think it would be rather odd to have just one regexp-reading command
that uses the rx syntax interactively.  If this is something we want
(and I don't really envision people willingly typing in rx forms
interactively), then it should be fixed in `read-regexp'.  It could, for
instance, have a keystroke that changes the input mode.

But do we think that people want to type in rx forms?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#31953: [VanL] Re: bug#31953: feature request - `highlight-rx` interactively

[Michael Heerdegen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> But do we think that people want to type in rx forms?

I would welcome such a change.

But IMO instead of

  M-x highlight-rx RET (or "foo" "bar") RET

it's not much harder to enter (rx (or "foo" "bar")) - so I wonder if we
could also just allow to enter any Elisp expression?

Michael.

bug#31953: [VanL] Re: bug#31953: feature request - `highlight-rx` interactively

[Lars Ingebrigtsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> But IMO instead of
>
>   M-x highlight-rx RET (or "foo" "bar") RET
>
> it's not much harder to enter (rx (or "foo" "bar")) - so I wonder if we
> could also just allow to enter any Elisp expression?

It's ambiguous, though -- you may want to search for "(rx (or "foo" "bar"))".

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#31953: [VanL] Re: bug#31953: feature request - `highlight-rx` interactively

[Michael Heerdegen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> It's ambiguous, though -- you may want to search for "(rx (or "foo"
> "bar"))".

Unless I misunderstood, you would enter

  "(rx (or \"foo\" \"bar\"))"

instead of

  (rx (or "foo" "bar"))


Michael.

bug#31953: [VanL] Re: bug#31953: feature request - `highlight-rx` interactively

[Drew Adams]

> I think it would be rather odd to have just one regexp-reading command
> that uses the rx syntax interactively.  If this is something we want
> (and I don't really envision people willingly typing in rx forms
> interactively), then it should be fixed in `read-regexp'.  It could,
> for instance, have a keystroke that changes the input mode.
> 
> But do we think that people want to type in rx forms?

(Apologies for jumping in here, ignoring the thread.)

FWIW -

Many applications, including some good ones, provide
dialog boxes (modal or not) for things like search &
replace that let you check a number of settings that
configure the kind of search (and replace).

Emacs generally favors progressive interaction: if
you need to specify multiple things then you are
prompted multiple times, or you use a prefix arg
together with perhaps responding to a single prompt.

But there's no reason that Emacs could not or should
not also offer such dialog boxes - including for
things like Isearch.  (If I had more free time
perhaps I'd wrestle something up with widgets.)

The other thing is that originally Isearch was simple,
in terms of the possibilities it offered.  That's
changed over time (char folding and lots of other
things).  The ability to configure multiple search
settings at once, with a straightforward dialog box,
and then reuse the same settings or modify them, is
a feature waiting to happen.

And I think configuring rx searches would be a great
candidate for this kind of thing.  I don't use RX,
mainly because I'm used to regexps and it think it
would be slower to use RX.  But not necessarily so,
if it's just a question of checking a few boxes.

It should be quick to configure a given search.
And it's super clear because the dialog box can lay
it all out and even provide helpful tooltips for
fields/actions.  And once configured just repeat,
with or without changing some setting.

I say go for it - even without a dialog box.

Do people really "want to type in rx forms?"  Maybe
not.  But later on we could give them a dialog box
that takes the place of typing in rx syntax ("forms").

bug#31953: [VanL] Re: bug#31953: feature request - `highlight-rx` interactively

[VanL]

Phil Sainty <psainty@orcon.net.nz> writes:

> On 2019-10-15 13:35, Lars Ingebrigtsen wrote:
>>> I was wanting to migrate from regexp to rx syntax for highlighting.
>>
>> `rx' returns a normal regexp, so you can use whatever you want, can't
>> you?
>
> I think the idea is to be able to *interactively* enter rx syntax
> and have it highlighted (as opposed to `highlight-regexp' which will
> obviously treat what you enter interactively as an actual regexp,
> rather than something to be transformed into one).
>
> i.e. These would be equivalent:
>
> M-x highlight-rx RET (or "foo" "bar") RET
> M-x highlight-regexp RET \(foo\|bar\) RET

What I can tell from the emacs-devel list and the git log is the rx
feature is being worked on.  I'd like to use rx everywhere regexp occurs
and highlight-rx would be the first easy use case.  At the limit there
are things expressible in regexp that rx won't be able to.  And, in that
case, I'd like to see in the rx documentation a sign post to regexp at
depth for that.  And, if there are patterns rx and/or regexp are unable
to express then providing a see also for those would be a help.

bug#31953: [VanL] Re: bug#31953: feature request - `highlight-rx` interactively

[Juri Linkov]

>> I think the idea is to be able to *interactively* enter rx syntax
>> and have it highlighted (as opposed to `highlight-regexp' which will
>> obviously treat what you enter interactively as an actual regexp,
>> rather than something to be transformed into one).
>>
>> i.e. These would be equivalent:
>>
>> M-x highlight-rx RET (or "foo" "bar") RET
>> M-x highlight-regexp RET \(foo\|bar\) RET
>
> What I can tell from the emacs-devel list and the git log is the rx
> feature is being worked on.  I'd like to use rx everywhere regexp occurs
> and highlight-rx would be the first easy use case.

Do you mean using rx also in all search and replace commands?

But the documentation says:

     The ‘rx’ notation is mainly useful in Lisp code; it cannot be used in
  most interactive situations where a regexp is requested, such as when
  running ‘query-replace-regexp’ or in variable customisation.

> At the limit there are things expressible in regexp that rx won't be
> able to.  And, in that case, I'd like to see in the rx documentation
> a sign post to regexp at depth for that.  And, if there are patterns
> rx and/or regexp are unable to express then providing a see also for
> those would be a help.

There is still some shortcomings in the current rx shorthands:

Such verbose constructs as ‘zero-or-more’ and ‘one-or-more’
are hard to remember and too long to type.  But fortunately
there are shorter synonyms ‘*’ and ‘+’ and other.  But they have
inconsistencies:

- ‘or’ has a synonym ‘|’, but ‘and’ has no synonym ‘&’

- ‘A{N}’ is the same as ‘=’, ‘A{N,}’ is ‘>=’, but ‘A{N,M}’ for some
  reason is ‘**’.  It would be more mnemonic something like ‘==’

- ‘line-start’ and ‘line-end’ have no shorter synonyms ‘^’ and ‘$’
  maybe because adding them would also require adding ‘\`’ and ‘\'’
  for ‘buffer-start’ and ‘buffer-end’, but ‘\`’ and ‘\'’ are very ugly.
  Fortunately there are already good synonyms ‘bol’ and ‘eol’
  that are easy to remember

- there is still problem with ‘buffer-start’ and ‘buffer-end’:
  their shorter synonyms are ‘bot’ and ‘eot’ that have no logic at all.
  Like there are already functions ‘bolp’, ‘eolp’, ‘bobp’ and ‘eobp’
  the best synonyms for buffer matches would be ‘bob’ and ‘eob’

- following the same logic ‘word-start’ could have a synonym ‘bow’
  and ‘word-end’ - ‘eow’.  I checked the implementation, and see
  there synonyms already are supported, but not documented.  Why?

bug#31953: [VanL] Re: bug#31953: feature request - `highlight-rx` interactively

[Noam Postavsky]

Juri Linkov <juri@linkov.net> writes:

> - ‘or’ has a synonym ‘|’, but ‘and’ has no synonym ‘&’

`and' is not the counterpart of `or', it's a synonym for `seq' or `:'.
We should probably deprecate `and', since this is a bit confusing.

bug#31953: [VanL] Re: bug#31953: feature request - `highlight-rx` interactively

[Juri Linkov]

>> - ‘or’ has a synonym ‘|’, but ‘and’ has no synonym ‘&’
>
> `and' is not the counterpart of `or', it's a synonym for `seq' or `:'.
> We should probably deprecate `and', since this is a bit confusing.

I consider ‘rx’ as an analogue of Definite Clause Grammar.

I'm so accustomed to Prolog paradigms used for defining grammars in DCG
that I can't use `rx' without its analogous constructs, e.g. such DCG:

  rx --> a, b; c.

where ‘,’ denotes conjunction, and ‘;’ denotes disjunction,
would naturally translate in ‘rx’ to corresponding and/or:

  (rx (| (& a b) c))

For example, this package parses a regexp and returns a DCG structure:
https://github.com/SWI-Prolog/packages-regex/blob/master/regex.dcg.pl

bug#31953: feature request - `highlight-rx` interactively

[VanL]

Juri Linkov <juri@linkov.net> writes:

>>> i.e. These would be equivalent:
>>>
>>> M-x highlight-rx RET (or "foo" "bar") RET
>>> M-x highlight-regexp RET \(foo\|bar\) RET
>>
>> What I can tell from the emacs-devel list and the git log is the rx
>> feature is being worked on.  I'd like to use rx everywhere regexp occurs
>> and highlight-rx would be the first easy use case.
>
> Do you mean using rx also in all search and replace commands?

I guess so if that makes sense to do farther down the track.
The advantage to using rx is to avoid those repeating
pathological toothpick sequences that are difficult to track.
For now, highlight-rx for searching is enough.

> But the documentation says:
>
>      The ‘rx’ notation is mainly useful in Lisp code; it cannot be used in
>   most interactive situations where a regexp is requested, such as when
>   running ‘query-replace-regexp’ or in variable customisation.

For search and replace, I don't know if it would be possible to
use in org-mode in the future, say, a table of three columns
having

1. input pattern
2. output 'desired' pattern
3. rx pattern which is generated automatically for 2

>> At the limit there are things expressible in regexp that rx won't be
>> able to.  And, in that case, I'd like to see in the rx documentation
>> a sign post to regexp at depth for that.  And, if there are patterns
>> rx and/or regexp are unable to express then providing a see also for
>> those would be a help.
>
> There is still some shortcomings in the current rx shorthands: ...

I guess the rx keywords will evolve to fit better with
experience.  Statistics are beginning to be collected on the
reported bugs.  Perhaps, rx can be used there where it makes
sense.

bug#31953: feature request - `highlight-rx` interactively

[Juri Linkov]

> If this is something we want (and I don't really envision people
> willingly typing in rx forms interactively), then it should be fixed
> in `read-regexp'.

Yep, `read-regexp' would be the right place to implement this.
Then like `read-from-minibuffer' has the arg `READ':

  If fourth arg READ is non-nil, interpret the result as a Lisp object
    and return that object:
    in other words, do ‘(car (read-from-string INPUT-STRING))’

`read-regexp' could have the same arg.

bug#31953: feature request - `highlight-rx` interactively

[Juri Linkov]

> Towards the transition to rx from regexp
> may I suggest `highlight-rx` for `highlight-regexp`.
>
> A usecase is to mark a region or string of few words
> and to call `highlight-rx` which does a verbatim highlight.

Nothing stops you from using rx in regexp commands already.

The first example shows how to use rx to find all Lisp comments:

 M-s o      ;; occur
 M-:        ;; eval-expression, then type or yank the following rx:
 (insert (rx bol (* whitespace) ";" (* nonl)))
 <return>   ;; exit-minibuffer inserts the converted regexp to the minibuffer
 <return>   ;; exit-minibuffer uses this regexp

The second example shows how to use rx to remove all Lisp comments:

 C-M-%      ;; query-replace-regexp
 M-:        ;; eval-expression
 M-p        ;; previous-history-element inserts the previous rx from history
 <return>   ;; exit-minibuffer inserts the converted regexp to the minibuffer
 <return>   ;; exit-minibuffer uses an empty string as replacement
 <return>   ;; exit-minibuffer replaces all comments with empty string

in 'emacs -Q' with (setq enable-recursive-minibuffers t)
The same can be used for 'highlight-regexp' as well.

bug#31953: feature request - `highlight-rx` interactively

[VanL]

>> Towards the transition to rx from regexp
>> may I suggest `highlight-rx` for `highlight-regexp`.
>>
>> A usecase is to mark a region or string of few words
>> and to call `highlight-rx` which does a verbatim highlight.
>
> Nothing stops you from using rx in regexp commands already.
>
> The first example shows how to use rx to find all Lisp comments:
>
>  M-s o      ;; occur
>  M-:        ;; eval-expression, then type or yank the following rx:
>  (insert (rx bol (* whitespace) ";" (* nonl)))
>  <return>   ;; exit-minibuffer inserts the converted regexp to the minibuffer
>  <return>   ;; exit-minibuffer uses this regexp
>

Ah, got it.

> The second example shows how to use rx to remove all Lisp comments:
>
>  C-M-%      ;; query-replace-regexp
>  M-:        ;; eval-expression
>  M-p        ;; previous-history-element inserts the previous rx from history
>  <return>   ;; exit-minibuffer inserts the converted regexp to the minibuffer
>  <return>   ;; exit-minibuffer uses an empty string as replacement
<  !          ;; exit-minibuffer replaces all comments with empty string
>
> in 'emacs -Q' with (setq enable-recursive-minibuffers t)
> The same can be used for 'highlight-regexp' as well.

Thanks for the examples.

bug#31953: feature request - `highlight-rx` interactively

[VanL]

> Nothing stops you from using rx in regexp
> commands already.

From a human factor's usability standpoint, the UI would
be better initially intuitively as sourcetrail's is. [1]



--
Footnotes: 
[1]  https://invidio.us/watch?v=Cfu6f0uyzc8
     Sourcetrail - Introduction