bug#35354: 26.2; `dired-do-query-replace-regexp': How to replace only word-delimited matches?

bug#35354: 26.2; `dired-do-query-replace-regexp': How to replace only word-delimited matches?

[Drew Adams]

`dired-do-query-replace-regexp' has taken the key binding of
`dired-do-find-regexp-and-replace'.  I was expecting that it would do
all that the latter does plus more.

How does a user use `dired-do-query-replace-regexp' to replace only
word-delimited matches?

If this isn't currently possible, please make is so.  Thx.

In GNU Emacs 26.2 (build 1, x86_64-w64-mingw32)
 of 2019-04-13
Repository revision: fd1b34bfba8f3f6298df47c8e10b61530426f749
Windowing system distributor `Microsoft Corp.', version 10.0.17134
Configured using:
 `configure --without-dbus --host=x86_64-w64-mingw32
 --without-compress-install 'CFLAGS=-O2 -static -g3''

bug#35354: 26.2; `dired-do-query-replace-regexp': How to replace only word-delimited matches?

[Dmitry Gutov]

On 21.04.2019 6:20, Drew Adams wrote:
> `dired-do-query-replace-regexp' has taken the key binding of
> `dired-do-find-regexp-and-replace'.  I was expecting that it would do
> all that the latter does plus more.

I don't really understand how we can both expect FROM to be a regexp and 
have a separate argument DELIMITED.

> How does a user use `dired-do-query-replace-regexp' to replace only
> word-delimited matches?

How about \<input\>?

bug#35354: 26.2; `dired-do-query-replace-regexp': How to replace only word-delimited matches?

[Drew Adams]

> > `dired-do-query-replace-regexp' has taken the key binding of
> > `dired-do-find-regexp-and-replace'.

(I obviously said that backward.  The latter took the
binding of the former.)

> > I was expecting that it would do
> > all that the latter does plus more.
> 
> I don't really understand how we can both expect FROM
> to be a regexp and have a separate argument DELIMITED.

Why is that a problem?

That's exactly what `dired-do-query-replace-regexp' does.
Its FROM is a REGEXP.  Doc string:

 Do ‘query-replace-regexp’ of FROM with TO, on all marked files.
 Third arg DELIMITED (prefix arg) means replace only
 word-delimited matches.
 If you exit (C-g, RET or q), you can resume the query replace
 with the command M-x tags-loop-continue.

(Of course, before the loss of key bindings, the last line
read simply "with the command M-,.)

> > How does a user use `dired-do-query-replace-regexp' to
> > replace only word-delimited matches?

(Again, I meant `dired-do-find-regexp-and-replace'.)

> How about \<input\>?

How do you specify delimited _replacement_?  It's about
the 3rd element returned by `query-replace-read-args',
which you're not currently using anywhere (even though
it is provided).

It's no different than it is for `query-replace-regexp'
or other commands that use `query-replace-read-args'.
But `xref-query-replace-in-results' doesn't accept that
3rd element, DELIMITED.  This bug report asks it to do so.

Also, args FROM and TO for `xref-query-replace-in-results'
aren't even described in its doc string.  They should be.

The doc string of `dired-do-query-replace-regexp' refers
to `query-replace-regexp', so users can tell that FROM is
in fact a REGEXP.  They deserve similar help for
`xref-query-replace-in-results', no?

`dired-do-query-replace-regexp' lets you use a prefix arg
to specify word-delimiting.  A prefix arg isn't even used
for `dired-do-find-regexp-and-replace'.  It would be simple
to do the same, provided `xref-query-replace-in-results'
could handle the DELIMITED arg.

bug#35354: 26.2; `dired-do-query-replace-regexp': How to replace only word-delimited matches?

[Dmitry Gutov]

On 22.04.2019 2:57, Drew Adams wrote:

> How do you specify delimited_replacement_?

Is there a practical difference in practice between this and using a 
regexp with word boundaries as FROM?

bug#35354: 26.2; `dired-do-query-replace-regexp': How to replace only word-delimited matches?

[Drew Adams]

> > How do you specify delimited_replacement_?
> 
> Is there a practical difference in practice between this and using a
> regexp with word boundaries as FROM?

A practical difference in practice?

See `query-replace-regexp'.  Why do you think it
has argument DELIMITED?

Same question for `replace-regexp', `replace-search'
(with arg REGEXP-FLAG), `query-replace-regexp-eval',
`perform-replace', and `tags-query-replace'.

All of those allow a "from" arg that can be a regexp,
and they all accept a DELIMITED arg as well.

bug#35354: 26.2; `dired-do-query-replace-regexp': How to replace only word-delimited matches?

[npostavs]

>>>>> "Drew" == Drew Adams <drew.adams@oracle.com> writes:
>>>>> "Dmitry" == Dmitry Gutov <dgutov@yandex.ru> writes:

    Drew> I was expecting that it would do all that the latter does plus
    Drew> more.

    Dmitry> I don't really understand how we can both expect FROM to be a
    Dmitry> regexp and have a separate argument DELIMITED.

    Drew> Why is that a problem?

It seems to give some possibly unexpected results.  For example, with
this buffer contents:

    word fooooo bar word
    word foo bar word

C-u C-M-% foo.*bar RET replaces only the "foo bar" on the second line,
not the "fooooo bar" on the first line.  That is to say, it's unclear
(to me at least) what DELIMITED means for regexps which can match both
word and non-word characters.

    Drew> How does a user use `dired-do[-find]-query-replace-regexp' to replace
    Drew> only word-delimited matches?

    Dmitry> How about \<input\>?

    Drew> How do you specify delimited _replacement_?

I understand how the match can be delimited, but I don't understand how
the replacement can be delimited.

    Dmitry> Is there a practical difference between this and
    Dmitry> using a regexp with word boundaries as FROM?

    Drew> See `query-replace-regexp'.  Why do you think it has argument
    Drew> DELIMITED?

I think it might be just to save the user from typing `\<' and `\>'.
Did you have another reason in mind?  If so, please say it plainly,
thanks.

bug#35354: 26.2; `dired-do-query-replace-regexp': How to replace only word-delimited matches?

[Lars Ingebrigtsen]

npostavs@gmail.com writes:

> I think it might be just to save the user from typing `\<' and `\>'.

Seems like that to me, too, so I don't think there's anything to do
here, and I'm closing this bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#35354: 26.2; `dired-do-query-replace-regexp': How to replace only word-delimited matches?

[Drew Adams]

> > I think it might be just to save the user
> > from typing `\<' and `\>'.
> 
> Seems like that to me, too, so I don't think there's
> anything to do here, and I'm closing this bug report.

So all of these points are ignored?

1.
  "Your example points out a regression.  It was introduced
   in Emacs 24.5.  In Emacs 20-23 (and probably pre-20) it
   acts correctly (per the doc): both lines of your example
   are treated the same: fooooo bar is matched.  Starting
   with Emacs 24.5 the doc is no longer respected."

2.
  "for the `query-replace' case it is useful as such.
   For the `query-replace-regexp' case it is not as useful.

   But even in that case it can be useful in this sense:
   You can _reuse_ a regexp in a different search, and just
   use a prefix arg to temporarily restrict the matches to
   having word boundaries - no need to edit the regexp.
   Not a big win, granted, but if you need to go back and
   forth then it could be a convenience."

3.
  "The point of this bug report is that if we replace the
   command used by `Q' with another then we should maybe
   have it do at least as much.

   DELIMITED has been in Emacs for the functions that use
   it (in replace.el) for a very long time."

4. Everything said about `dired-do-query-replace-regexp'
and `dired-do-find-regexp-and-replace'.

bug#35354: 26.2; `dired-do-query-replace-regexp': How to replace only word-delimited matches?

[Drew Adams]

> >>>>> "Drew" == Drew Adams <drew.adams@oracle.com> writes:
> >>>>> "Dmitry" == Dmitry Gutov <dgutov@yandex.ru> writes:
> 
>     Drew> I was expecting that it would do all that the latter does plus
>     Drew> more.
> 
>     Dmitry> I don't really understand how we can both expect FROM to be a
>     Dmitry> regexp and have a separate argument DELIMITED.
> 
>     Drew> Why is that a problem?
> 
> It seems to give some possibly unexpected results.  For example, with
> this buffer contents:
> 
>     word fooooo bar word
>     word foo bar word
> 
> C-u C-M-% foo.*bar RET replaces only the "foo bar" on the second line,
> not the "fooooo bar" on the first line.  That is to say, it's unclear
> (to me at least) what DELIMITED means for regexps which can match both
> word and non-word characters.

The doc says that only the start and end of the match
need be word boundaries.  So I guess that Dmitry is
right that in the case where the input is interpreted
as a regexp non-nil DELIMITED can only be a convenience
for not having to use \v.

I was thinking that DELIMITED acted as a second filter,
requiring not only word boundaries at beginning and end
but also that all chars matched be word-constituent.
That's clearly not the case though, and the doc doesn't
suggest it should be.

Your example points out a regression.  It was introduced
in Emacs 24.5.  In Emacs 20-23 (and probably pre-20) it
acts correctly (per the doc): both lines of your example
are treated the same: fooooo bar is matched.  Starting
with Emacs 24.5 the doc is no longer respected.

Was the change intentional, and someone forgot to update
the doc?  I doubt it.  This change/regression coincides
with the introduction of `replace-search' and
`replace-highlight' and the use of `isearch-word', which
later became ` isearch-regexp-function'.  Before that,
`perform-replace' just did this:

    (if delimited-flag
	(setq search-function 're-search-forward
	      search-string (concat "\\b"
				    (if regexp-flag from-string
				      (regexp-quote from-string))
				    "\\b")))

> I understand how the match can be delimited, but I don't understand how
> the replacement can be delimited.

Right.  I was wrongly thinking that non-nil DELIMITED
meant that the otherwise-matches were in addition
filtered by also requiring their chars to be word
constituents.

>     Dmitry> Is there a practical difference between this and
>     Dmitry> using a regexp with word boundaries as FROM?
> 
>     Drew> See `query-replace-regexp'.  Why do you think it has argument
>     Drew> DELIMITED?
> 
> I think it might be just to save the user from typing `\<' and `\>'.
> Did you have another reason in mind?  If so, please say it plainly,
> thanks.

I think you must be right.  Clearly for the `query-replace'
case it is useful as such.  For the `query-replace-regexp'
case it is not as useful.

But even in that case it can be useful in this sense:
You can _reuse_ a regexp in a different search, and just
use a prefix arg to temporarily restrict the matches to
having word boundaries - no need to edit the regexp.
Not a big win, granted, but if you need to go back and
forth then it could be a convenience.

The point of this bug report is that if we replace the
command used by `Q' with another then we should maybe
have it do at least as much.

DELIMITED has been in Emacs for the functions that use
it (in replace.el) for a very long time.