Rename refactoring, or something like that

Rename refactoring, or something like that

[Dmitry Gutov]

We have "something like that" now, implemented as an xref command acting 
on a list of search results. To try it out:

1. M-x xref-find-references, or M-x project-find-regexp. Input something 
real, get a list of matches.

2. In the results buffer, press `r'.

3. Somewhat counter-intuitively, press RET to agree to `.*', then enter 
the replacement string, and press RET again.

4. Press `y' or `n' a few times, to choose which of the matches to 
actually replace. M-x save-some-buffers, then `!', to save the changes.

5. Feel hugely motivated, and write a better UI for all of that.

Re: Rename refactoring, or something like that

[joakim]

Dmitry Gutov <dgutov@yandex.ru> writes:

> We have "something like that" now, implemented as an xref command
> acting on a list of search results. To try it out:
>
> 1. M-x xref-find-references, or M-x project-find-regexp. Input
> something real, get a list of matches.
>
> 2. In the results buffer, press `r'.
>
> 3. Somewhat counter-intuitively, press RET to agree to `.*', then
> enter the replacement string, and press RET again.
>
> 4. Press `y' or `n' a few times, to choose which of the matches to
> actually replace. M-x save-some-buffers, then `!', to save the
> changes.
>
> 5. Feel hugely motivated, and write a better UI for all of that.
>

Does it work like one of the grep-edit implementations?

-- 
Joakim Verona

Re: Rename refactoring, or something like that

[Dmitry Gutov]

On 11/16/2015 12:03 PM, joakim@verona.se wrote:

> Does it work like one of the grep-edit implementations?

It doesn't.

A better UI than perform-replace (was: Rename refactoring, or something like that)

[Oleh Krehel]

Dmitry Gutov <dgutov@yandex.ru> writes:

> 5. Feel hugely motivated, and write a better UI for all of that.

I think a better UI for `perform-replace' is warranted.  The current
thing is very basic:

- No good way to see how many matches there are.
- No good way to get an overview of matches per buffer.
- No good way to pause the replacement procedure.
- No good way to undo a replacement.

An idea to improve this would be with a permanent *replace* buffer,
similar to `dired' or *Buffer List*. This buffer would be visible during
the replacement operation, together with the actual buffers that contain
the candidates. Here are some ideas for key bindings and the general
interface:

0. The *replace* buffer would be organized in branches by buffer, just
like *xref* is currently.

1. "y" would mark the current item for replacement, which corresponds to
the current line in the *replace* buffer, and advance to the next line.

2. "n" would mark the current item for non-replacement, and advance to
the next line. This is to distinguish the items for which no decision
was made yet: they don't have a `y' or `n' mark.

3. "Y" would mark all items from the current one until the end of the
current buffer branch. "N" would work in a similar way.

4. "c" could change the replacement text for a single item. The mark
would change appropriately.

5. "C" could change the replacement text for all following items.

6. "x" would execute all requested changes. 

7. "e" would launch an `ediff' session to review the changes that "x"
would do.

8. After "x", the *replace* buffer stays behind until the user kills it.
It should serve to review the changes. Possibly "u" could be used to
revert a performed change for the current item.

9. "C-n", "C-p" and any other navigational commands should work as
usual. Except changing the current item in *replace* should result in
that item being displayed in its native buffer - exactly what "C-o"
currently does in *xref*.

To re-describe things in a shorter way, *replace* should be a staging
area for the `perform-replace' operation. With the ability to quickly
view and manipulate the separate replacements, the ability to defer it
indefinitely, and the ability to finalize the operation with "x".

I post this idea here for feedback. Maybe there's a better way to do
this. Maybe there are ideas to further improve my *replace* proposal.
Maybe there's a wish to have this functionality in the core.  Maybe
someone even wants to implement it right away (I can do it eventually,
if there are no volunteers). Let me know.

    Oleh

Re: Rename refactoring, or something like that

[Óscar Fuentes]

Dmitry Gutov <dgutov@yandex.ru> writes:

> We have "something like that" now, implemented as an xref command
> acting on a list of search results. To try it out:
>
> 1. M-x xref-find-references, or M-x project-find-regexp. Input
> something real, get a list of matches.
>
> 2. In the results buffer, press `r'.
>
> 3. Somewhat counter-intuitively, press RET to agree to `.*', then
> enter the replacement string, and press RET again.
>
> 4. Press `y' or `n' a few times, to choose which of the matches to
> actually replace. M-x save-some-buffers, then `!', to save the
> changes.

How is this different from grepping and then using grep-edit.el?

> 5. Feel hugely motivated, and write a better UI for all of that.

I suspect that the UI is the least of our concerns.

Re: Rename refactoring, or something like that

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sun, 15 Nov 2015 06:43:24 +0200
> 
> We have "something like that" now, implemented as an xref command acting 
> on a list of search results. To try it out:

On master or on the release branch?

Speaking of which: is this a known failure:

  Testing elisp-mode-tests.elc
  Running 35 tests (2015-11-16 17:58:49+0200)
     passed   1/35  elisp-completes-anything-quoted
  Test xref-elisp-test-find-defs-defun-c-defvar-c backtrace:
    ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc
    ert-run-test([cl-struct-ert-test xref-elisp-test-find-defs-defun-c-d
    ert-run-or-rerun-test([cl-struct-ert--stats t [[cl-struct-ert-test e
    ert-run-tests(t #[385 "\306☻\307\"\203G \211\211G\310U\203¶ \211@\20
    ert-run-tests-batch(nil)
    ert-run-tests-batch-and-exit()
    command-line-1(("-L" ";." "-l" "ert" "-l" "elisp-mode-tests.elc" "-f
    command-line()
    normal-top-level()
  Test xref-elisp-test-find-defs-defun-c-defvar-c condition:
      (void-function elisp-xref-find)
     FAILED  24/35  xref-elisp-test-find-defs-defun-c-defvar-c
     passed  25/35  xref-elisp-test-find-defs-defun-defvar-el
     passed  26/35  xref-elisp-test-find-defs-defun-el
  Test xref-elisp-test-find-defs-defun-el-defvar-c backtrace:
    ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc
    ert-run-test([cl-struct-ert-test xref-elisp-test-find-defs-defun-el-
    ert-run-or-rerun-test([cl-struct-ert--stats t [[cl-struct-ert-test e
    ert-run-tests(t #[385 "\306☻\307\"\203G \211\211G\310U\203¶ \211@\20
    ert-run-tests-batch(nil)
    ert-run-tests-batch-and-exit()
    command-line-1(("-L" ";." "-l" "ert" "-l" "elisp-mode-tests.elc" "-f
    command-line()
    normal-top-level()
  Test xref-elisp-test-find-defs-defun-el-defvar-c condition:
      (void-function elisp-xref-find)
     FAILED  27/35  xref-elisp-test-find-defs-defun-el-defvar-c

Re: Rename refactoring, or something like that

[Dmitry Gutov]

On 11/16/2015 06:01 PM, Eli Zaretskii wrote:

> On master or on the release branch?

Both.

> Speaking of which: is this a known failure:

Sorry; I'll take care of it today.

Re: Rename refactoring, or something like that

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 16 Nov 2015 18:03:54 +0200
> 
> On 11/16/2015 06:01 PM, Eli Zaretskii wrote:
> 
> > On master or on the release branch?
> 
> Both.

OK, I will try that when I have time.

> > Speaking of which: is this a known failure:
> 
> Sorry; I'll take care of it today.

Thanks.

Re: A better UI than perform-replace

[Dmitry Gutov]

On 11/16/2015 01:47 PM, Oleh Krehel wrote:

> I think a better UI for `perform-replace' is warranted.  The current
> thing is very basic:
>
> - No good way to see how many matches there are.
> - No good way to get an overview of matches per buffer.
> - No good way to pause the replacement procedure.
> - No good way to undo a replacement.

Indeed.

> An idea to improve this would be with a permanent *replace* buffer,
> similar to `dired' or *Buffer List*. This buffer would be visible during
> the replacement operation, together with the actual buffers that contain
> the candidates. Here are some ideas for key bindings and the general
> interface:

Maybe a universal "replace buffer" is the way to go. I'd expected to 
have an xref-specific replace buffer first, though.

Or even modify the existing xref buffer inline when you press `r', to 
add some widgets to signify marks (maybe just a couple of columns), 
you'd be able to select the lines you want to rename, perform the 
rename, and then maybe see the xref results refreshed, with only those 
hits that you haven't changes, remaining (maybe you want to do something 
else with them).

But don't let me stop you from working on a generic feature - after all, 
query-replace is used in many places.

> 0. The *replace* buffer would be organized in branches by buffer, just
> like *xref* is currently.
>
> 1. "y" would mark the current item for replacement, which corresponds to
> the current line in the *replace* buffer, and advance to the next line.

Or `m', which we use for marks, usually.

> 2. "n" would mark the current item for non-replacement, and advance to
> the next line. This is to distinguish the items for which no decision
> was made yet: they don't have a `y' or `n' mark.

I see the keystroke-saving benefits, but maybe we should start with the 
usual approach of p/n for up/down, and m/u to mark/unmark. `% m' to mark 
by regexp, `U' to unmark all, `t' to toggle all marks. Press `m' while 
cursor is on a group header -> and its marks all its elements, and 
similar for `u'.

> 3. "Y" would mark all items from the current one until the end of the
> current buffer branch. "N" would work in a similar way.
>
> 4. "c" could change the replacement text for a single item. The mark
> would change appropriately.
>
> 5. "C" could change the replacement text for all following items.

That might be cool to try, too.

> 6. "x" would execute all requested changes.
>
> 7. "e" would launch an `ediff' session to review the changes that "x"
> would do.

Wouldn't hurt.

> 8. After "x", the *replace* buffer stays behind until the user kills it.
> It should serve to review the changes. Possibly "u" could be used to
> revert a performed change for the current item.

Sounds good.

> 9. "C-n", "C-p" and any other navigational commands should work as
> usual. Except changing the current item in *replace* should result in
> that item being displayed in its native buffer - exactly what "C-o"
> currently does in *xref*.

That description seems incomplete. Either we shouldn't mess with what's 
displayed in other buffers (and maybe implement a direct counterpart to 
"C-o"), or the *replace* buffer should always display the current item 
in the "other window". But I'm inclined toward the former.

As another alternative, we could should a few lines of context for the 
current match inside the *replace* buffer, if the user asks (e.g. with 
`+' increasing the context height, and `-' reducing it).

> To re-describe things in a shorter way, *replace* should be a staging
> area for the `perform-replace' operation. With the ability to quickly
> view and manipulate the separate replacements, the ability to defer it
> indefinitely, and the ability to finalize the operation with "x".

With the ability to "defer it", there'll also come a need to check 
whether the inputs (search matches?) are up-to-date. That puts some 
restrictions on what it can work with.

> I post this idea here for feedback. Maybe there's a better way to do
> this. Maybe there are ideas to further improve my *replace* proposal.

Another thing I haven't seen mentioned in this list, is the ability to 
rename a file, as a part of search-replace. It may not be needed if one 
performs a textual search, but for the "rename" refactoring operation, 
if I rename a class in Ruby or Java, I want to have its file to be 
renamed automatically as well.

So ideally, the replace buffer would allow to preview that operation as 
well. I'm still thinking how to better support that in xref data structures.

 > Maybe there's a wish to have this functionality in the core.

Where else?

Re: Rename refactoring, or something like that

[Dmitry Gutov]

On 11/16/2015 03:13 PM, Óscar Fuentes wrote:

> How is this different from grepping and then using grep-edit.el?

You get to perform the replacements in the results of 
xref-find-references. Or, really, in any "search results" that are fed 
to xref.

Another difference is that grep-edit is not in the core.

There will be more in future, but we're not there yet.

> I suspect that the UI is the least of our concerns.

What's your biggest concern, then?

RE: A better UI than perform-replace (was: Rename refactoring, or something like that)

[Drew Adams]

> I think a better UI for `perform-replace' is warranted.

Any such should not be a replacement but an addition to what
we have now.  Think ibuffer and buffer-menu.

Re: Rename refactoring, or something like that

[John Yates]

[-- Attachment #1: Type: text/plain, Size: 668 bytes --]

As part of an improved UI I would hope that if I rename an identifier to an
identifier of a different length there will be an easy way to visit the
sites of the renames to fix up formatting (primarily adjusting whitespace
and/or lines that have become too long).

If the rename is not based on a strong semantic framework (e.g. in the C++
world something that can truly grok all of the name resolution rules) then
there will need to be some means of incorporating human input (a la
query-and-replace).  If such querying occurs on a site by site basis then
there could be a query respone that says "yes, do the rename but then pause
and let me do some editing".

/john

[-- Attachment #2: Type: text/html, Size: 996 bytes --]

Re: Rename refactoring, or something like that

[Dmitry Gutov]

On 11/17/2015 01:10 PM, John Yates wrote:

> If the rename is not based on a strong semantic framework (e.g. in the
> C++ world something that can truly grok all of the name resolution
> rules) then there will need to be some means of incorporating human
> input (a la query-and-replace).  If such querying occurs on a site by
> site basis then there could be a query respone that says "yes, do the
> rename but then pause and let me do some editing".

That an interesting suggestion. Not the highest priority, IMHO, but it 
should be possible to add such a feature to any renames interface we 
settle on, later.

Re: Rename refactoring, or something like that

[John Yates]

[-- Attachment #1: Type: text/plain, Size: 485 bytes --]

On Tue, Nov 17, 2015 at 8:14 AM, Dmitry Gutov <dgutov@yandex.ru> wrote:

> That an interesting suggestion. Not the highest priority, IMHO
>

Agreed, not the highest priority.  Just wanted to get the suggestion on the
record.

Further embellishments: in the initial buffer setting up the rename it
might help to be able to say only edit if
  [] Line length now exceeds <threshold>
  [] Rename precedes an end of line comment

Again, very low priority but UI ideas just the same.

/john

[-- Attachment #2: Type: text/html, Size: 898 bytes --]

Re: A better UI than perform-replace (was: Rename refactoring, or something like that)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > - No good way to pause the replacement procedure.

You can do that easily by typing any ordinary editing
command, or ESC.  To resume, use C-x ESC ESC RET.

  > - No good way to undo a replacement.

Yes there is: type C-_ (C-- on an X terminal).

Nonethless, I can see how your new mode might be useful.  But the
*replace* buffer might also get in the way.

I think we should poll the users before changing the default behavior.
We could ask them to suggest changes in the new feature
as well as to say whether they want it as the default.

  > To re-describe things in a shorter way, *replace* should be a staging
  > area for the `perform-replace' operation.

I see a possible pitfall here.  It seems to envision finding all the
matches before operating on any of them.

Currently, if you pause and resume the query replace loop, it resumes
starting from point and it finds all matches in the current text after
point.  So if during he pause you edited something below, creating or
eliminating matches, it operates on the matches that exist after those
changes.

But if the list of matches was made in advance, it won't understand
that the buffer has changed.  It can double-check the old matches
to make sure they still match, but it will ignore newly introduced
matches.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: A better UI than perform-replace

[John Wiegley]

>>>>> Drew Adams <drew.adams@oracle.com> writes:

>> I think a better UI for `perform-replace' is warranted.
> Any such should not be a replacement but an addition to what
> we have now.  Think ibuffer and buffer-menu.

Agreed. And nothing is stopping such a new replace feature from being
developed in ELPA, so we can all try it out as it develops. If it really
improves in every way upon the status quo, it could become the new default
binding. But the old code should remain as it is, for those who might prefer
it.

John

Re: A better UI than perform-replace

[Juri Linkov]

> I think a better UI for `perform-replace' is warranted.  The current
> thing is very basic:
>
> - No good way to see how many matches there are.
> - No good way to get an overview of matches per buffer.

It's easy to run 'occur' to display all matches and total match count.

> - No good way to pause the replacement procedure.

C-r enters recursive edit (C-M-c to get out again).

> - No good way to undo a replacement.

Undo is implemented in bug#21684.

> An idea to improve this would be with a permanent *replace* buffer,
> similar to `dired' or *Buffer List*. This buffer would be visible during
> the replacement operation, together with the actual buffers that contain
> the candidates.

Good idea.  And as addressing your first concern above suggests,
the most suitable buffer would be *Occur* itself.  It already
displays all matches, and what is missing is just what you propose:
new keybindings and Dired-like indication on the left edge.

In fact, we already have 'occur-edit-mode' acting on *Occur* buffer.
So what we could do is to add a similar mode 'occur-replace-mode'
that will perform 'automatic-all' on 'C-c C-c'.

RE: A better UI than perform-replace

[Drew Adams]

> Good idea.  And as addressing your first concern above suggests,
> the most suitable buffer would be *Occur* itself.  It already
> displays all matches, and what is missing is just what you propose:
> new keybindings and Dired-like indication on the left edge.
> 
> In fact, we already have 'occur-edit-mode' acting on *Occur* buffer.
> So what we could do is to add a similar mode 'occur-replace-mode'
> that will perform 'automatic-all' on 'C-c C-c'.

+1.

And Juri will have a patch for it in about...45 minutes. ;-)

Re: A better UI than perform-replace

[Andreas Schwab]

Oleh Krehel <ohwoeowho@gmail.com> writes:

> - No good way to pause the replacement procedure.

You can use C-r to enter recusive edit.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: A better UI than perform-replace

[Dmitry Gutov]

On 11/18/2015 02:55 AM, Juri Linkov wrote:

> It's easy to run 'occur' to display all matches and total match count.

Should we have a way to turn a multi-file query-replace into a 
multi-occur editing buffer?

Re: A better UI than perform-replace

[Juri Linkov]

> Should we have a way to turn a multi-file query-replace into a multi-occur
> editing buffer?

I think we need to keep both the current multi-file query-replace and
multi-occur, and extend the latter with query-replace capabilities,
(thus a better mode name would be 'occur-query-replace-mode') because
the current multi-file query-replace is already used in several places
such as tags-query-replace.  Then for xref you could take any of them.

PS: I noticed a FIXME in xref--query-replace-1, and I wonder why can't you
use `multi-query-replace-map' the same way as it's used in tags-query-replace?

Re: A better UI than perform-replace

[Dmitry Gutov]

On 11/19/2015 02:57 AM, Juri Linkov wrote:

> I think we need to keep both the current multi-file query-replace and
> multi-occur, and extend the latter with query-replace capabilities,
> (thus a better mode name would be 'occur-query-replace-mode') because
> the current multi-file query-replace is already used in several places
> such as tags-query-replace.  Then for xref you could take any of them.

Yes, we'll need a separate occur-??? query-replace function that takes a 
set of files as an argument.

It would be cool if both query-replace functions had the same calling 
convention, and we could choose the user-preferred one based on a global 
variable.

> PS: I noticed a FIXME in xref--query-replace-1, and I wonder why can't you
> use `multi-query-replace-map' the same way as it's used in tags-query-replace?

Last time I tried, it wasn't a drop-in replacement: with 
`multi-query-replace-map', `perform-replace' really expects to be called 
once per buffer, and it would've been really inconvenient in the initial 
implementation. Less so with the current one, though.

Re: A better UI than perform-replace

[John Yates]

[-- Attachment #1: Type: text/plain, Size: 740 bytes --]

On Tue, Nov 17, 2015 at 7:55 PM, Juri Linkov <juri@linkov.net> wrote:

> > - No good way to pause the replacement procedure.
>
> C-r enters recursive edit (C-M-c to get out again).
>

On Wed, Nov 18, 2015 at 3:50 AM, Andreas Schwab <schwab@suse.de> wrote:

> > - No good way to pause the replacement procedure.
>
> You can use C-r to enter recusive edit.
>

I am not a power users.  It would never occurr to me that the solution of a
common UI use case would be an invocation of modal behavior.

Recursive editing is noobie unfriendly.  It is not particularly
discoverable.  It requires maintaining a more complex mental model of the
editor's state while performing the rename.  And it requires more key
strokes than direct support.

/john

[-- Attachment #2: Type: text/html, Size: 1506 bytes --]

Re: A better UI than perform-replace

[John Wiegley]

>>>>> John Yates <john@yates-sheets.org> writes:

> I am not a power users. It would never occurr to me that the solution of a
> common UI use case would be an invocation of modal behavior.

> Recursive editing is noobie unfriendly. It is not particularly discoverable.
> It requires maintaining a more complex mental model of the editor's state
> while performing the rename. And it requires more key strokes than direct
> support.

Wanting to use Emacs for other editing, while in the middle of a replace
operation, is fairly expert also. We don't use a separate window with "Find
Next" and "Replace" buttons, as graphical editors do: we use a modal prompt in
the mini-buffer. Most Emacs users expect this to mean that they must complete
their operation before Emacs is fully available again.

I agree that recursive editing is not well known, but the right answer may be
education, rather than building new functionality for this one case, just so
people don't have to learn a feature of Emacs that's (a) been available for a
very long time, and (b) is handy in far more circumstances than merely find
and replace.

John

Re: A better UI than perform-replace

[John Yates]

[-- Attachment #1: Type: text/plain, Size: 1970 bytes --]

On Thu, Nov 19, 2015 at 11:31 AM, John Wiegley <jwiegley@gmail.com> wrote:

> Wanting to use Emacs for other editing, while in the middle of a replace
> operation, is fairly expert also. We don't use a separate window with "Find
> Next" and "Replace" buttons, as graphical editors do: we use a modal
> prompt in
> the mini-buffer. Most Emacs users expect this to mean that they must
> complete
> their operation before Emacs is fully available again.
>

Disclaimer: my model for the proposed interaction is query-replace.
If we are actually contemplating something quite different then
perhaps my comments are irrelevant.  That said...

The usage scenario is not uncommon and is not an instance of wanting
to perform arbitrary editing in the middle of a replace operation.
Rather it is a question of wanting (or needing) to clean-up alignment
_after_ replacing a token with a token of a different length.  A very
common instance is ensuring that end of line comments start at a
specified column.

IIUC when prompted as to whether I wish to replace the current token
the recursive edit approach requires that I:

  1) type C-r to enter a recursive edit
  2) adjust whitespace (thereby corrupting the visible alignment
     around the as yet un-replaced token) hoping that I properly
     anticipate the whitespace needs that will obtain once the
     replace actually occurs
  3) type C-M-c to exit recursive edit
  4) respond to the query-replace prompt

Following 4) token replacement occurs and the display updates to
the next match.  I get no visual feedback as to whether my attempt
to adjust alignment was actually correct.  This is the antithesis
of a satisfactory WYSIWYG experience.  I might as well be editing
in TECO (which _I_ actually did at the beginning of my career :-)

If I am working in an environment that bans hard tabs I just may
get it right.  Allow hard tabs and I doubt that there is a person
on this list who would defend the experience.

/john

[-- Attachment #2: Type: text/html, Size: 2513 bytes --]

Re: A better UI than perform-replace

[John Wiegley]

>>>>> John Yates <john@yates-sheets.org> writes:

> Rather it is a question of wanting (or needing) to clean-up alignment
> _after_ replacing a token with a token of a different length. A very common
> instance is ensuring that end of line comments start at a specified column.

Ah, yes, I feel your pain now, most directly. I find myself in this same
circumstance quite often.

Personally, I think the answer is to offer a new mode targeted at "intelligent
replacements" -- beginning life in ELPA -- rather than making changes to our
current machinery. That way we can test out how it works before considering
upsetting a very old apple cart.

For example, swiper.el is a fantastic replacement to i-search in many ways,
providing not only excellent visuals, but also a nicer way to navigate hits.
However, changing the centuries-old behavior of C-s is not to be done lightly;
nor is there really any reason to, since users can always bind C-s to `swiper'
as they see fit.

John

Re: A better UI than perform-replace

[David Kastrup]

John Yates <john@yates-sheets.org> writes:

> On Thu, Nov 19, 2015 at 11:31 AM, John Wiegley <jwiegley@gmail.com> wrote:
>
>> Wanting to use Emacs for other editing, while in the middle of a replace
>> operation, is fairly expert also. We don't use a separate window with "Find
>> Next" and "Replace" buttons, as graphical editors do: we use a modal
>> prompt in
>> the mini-buffer. Most Emacs users expect this to mean that they must
>> complete
>> their operation before Emacs is fully available again.
>>
>
> Disclaimer: my model for the proposed interaction is query-replace.
> If we are actually contemplating something quite different then
> perhaps my comments are irrelevant.  That said...
>
> The usage scenario is not uncommon and is not an instance of wanting
> to perform arbitrary editing in the middle of a replace operation.
> Rather it is a question of wanting (or needing) to clean-up alignment
> _after_ replacing a token with a token of a different length.  A very
> common instance is ensuring that end of line comments start at a
> specified column.
>
> IIUC when prompted as to whether I wish to replace the current token
> the recursive edit approach requires that I:
>
>   1) type C-r to enter a recursive edit
>   2) adjust whitespace (thereby corrupting the visible alignment
>      around the as yet un-replaced token) hoping that I properly
>      anticipate the whitespace needs that will obtain once the
>      replace actually occurs
>   3) type C-M-c to exit recursive edit
>   4) respond to the query-replace prompt
>
> Following 4) token replacement occurs and the display updates to
> the next match.

Wrong response to the query-replace prompt then.

    Type Space or ‘y’ to replace one match, Delete or ‘n’ to skip to next,
    RET or ‘q’ to exit, Period to replace one match and exit,
    Comma to replace but not move point immediately,
    C-r to enter recursive edit (C-M-c to get out again),
    C-w to delete match and recursive edit,
    C-l to clear the screen, redisplay, and offer same replacement again,
    ! to replace all remaining matches in this buffer with no more questions,
    ^ to move point back to previous match,
    E to edit the replacement string.
    In multi-buffer replacements type ‘Y’ to replace all remaining
    matches in all remaining buffers with no more questions,
    ‘N’ to skip to the next buffer without replacing remaining matches
    in the current buffer.

So you should have typed a comma.

Or, once you consider yourself foiled, follow the "press ? for help"
prompt and discover "^ to move point back to previous match".

-- 
David Kastrup

RE: A better UI than perform-replace

[Drew Adams]

> >   4) respond to the query-replace prompt
> > Following 4) token replacement occurs and the display updates to
> > the next match.
> 
> Wrong response to the query-replace prompt then.

Exactly.

>     Type Space or ‘y’ to replace one match, Delete or ‘n’ to skip to next,
>     RET or ‘q’ to exit, Period to replace one match and exit,
>     Comma to replace but not move point immediately,
>     C-r to enter recursive edit (C-M-c to get out again),
>     C-w to delete match and recursive edit,
>     C-l to clear the screen, redisplay, and offer same replacement again,
>     ! to replace all remaining matches in this buffer with no more
>       questions,
>     ^ to move point back to previous match,
>     E to edit the replacement string.
>     In multi-buffer replacements type ‘Y’ to replace all remaining
>     matches in all remaining buffers with no more questions,
>     ‘N’ to skip to the next buffer without replacing remaining matches
>     in the current buffer.
> 
> So you should have typed a comma.
> 
> Or, once you consider yourself foiled, follow the "press ? for help"
> prompt and discover "^ to move point back to previous match".

Not to mention that the very first paragraph of `C-h k M-%' says:

Replace some occurrences of FROM-STRING with TO-STRING.
As each match is found, the user must type a character saying
what to do with it.  For directions, type C-h at that time.
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

And yes, newbies too (nay, newbies _especially_) should learn
first _how to ask Emacs_.  And having learned that, they should
_ask Emacs_ first.  It's really not complicated to ask Emacs
what `M-%' does: `C-h k M-%'.  And it should really not be hard
to see "For directions, type C-h" up front.  And if you do hit
`C-h' (or `?'), it should really not be hard to understand how
to replace without moving to the next search hit.

(But maybe the text should say "Comma (`,')..." instead of
"Comma...".  Oh, and each of the keys should be enclosed in
quotes, or else none of them should be, IMO.)

Every tutorial and intro to Emacs should start with a basic
lesson about how to _ask Emacs_.  (And every intermediate lesson
about Emacs should include deeper info about how to ask Emacs
about itself, including Elisp.)

I already agreed that an enhancement of `occur' along the
lines of what Oleh suggested should be a welcome addition.
But I disagree about it being difficult to understand or
discover the behavior of `query-replace' (`M-%') - assuming
that you've learned about `C-h k'.

And as David points out, even if you do not check `C-h k M-%',
the prompt itself tells you, over and over, that you can
"press `?' for help".  How someone could claim that the
behavior is difficult to discover is beyond me.  (But people
are different.)

Re: A better UI than perform-replace

[Juri Linkov]

>> I think we need to keep both the current multi-file query-replace and
>> multi-occur, and extend the latter with query-replace capabilities,
>> (thus a better mode name would be 'occur-query-replace-mode') because
>> the current multi-file query-replace is already used in several places
>> such as tags-query-replace.  Then for xref you could take any of them.
>
> Yes, we'll need a separate occur-??? query-replace function that takes
> a set of files as an argument.
>
> It would be cool if both query-replace functions had the same calling
> convention, and we could choose the user-preferred one based on
> a global variable.
>
>> PS: I noticed a FIXME in xref--query-replace-1, and I wonder why can't you
>> use `multi-query-replace-map' the same way as it's used in tags-query-replace?
>
> Last time I tried, it wasn't a drop-in replacement: with
> `multi-query-replace-map', `perform-replace' really expects to be called
> once per buffer, and it would've been really inconvenient in the initial
> implementation. Less so with the current one, though.

Then instead of tags-loop-continue/next-file currently used by etags.el,
we could have a much simpler function multi-query-replace with bufs arg.

Re: A better UI than perform-replace

[Juri Linkov]

> And as David points out, even if you do not check `C-h k M-%',
> the prompt itself tells you, over and over, that you can
> "press `?' for help".  How someone could claim that the
> behavior is difficult to discover is beyond me.  (But people
> are different.)

If "press `?' for help" is intended for newbies, why do
we display it all the time?  The same question is about
customization of "[F1] Help" in Isearch proposed by Artur
in http://www.emacswiki.org/emacs/Isearch-prompt-proposal
where we need to hide the reminder after its first use too.

Re: A better UI than perform-replace

[Adrian.B.Robert]

>> Rather it is a question of wanting (or needing) to clean-up alignment
>> _after_ replacing a token with a token of a different length. A very common
>> instance is ensuring that end of line comments start at a specified column.
>
> ...
>
> Personally, I think the answer is to offer a new mode targeted at "intelligent
> replacements" -- beginning life in ELPA -- rather than making changes to our
> current machinery.

I would love to see something like this.  And something as simple as
running indent-for-tab on the line +/- context would go a long way.
IntelliJ just started doing this in version 15 and it is a pleasure.

Re: A better UI than perform-replace

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Date: Fri, 20 Nov 2015 02:42:16 +0200
> Cc: emacs-devel <emacs-devel@gnu.org>, David Kastrup <dak@gnu.org>,
> 	Dmitry Gutov <dgutov@yandex.ru>, Oleh Krehel <ohwoeowho@gmail.com>,
> 	John Yates <john@yates-sheets.org>
> 
> > And as David points out, even if you do not check `C-h k M-%',
> > the prompt itself tells you, over and over, that you can
> > "press `?' for help".  How someone could claim that the
> > behavior is difficult to discover is beyond me.  (But people
> > are different.)
> 
> If "press `?' for help" is intended for newbies, why do
> we display it all the time?

FWIW, this "newbie" takes the advantage of '?' all the time, at least
with commands I don't use frequently enough to have their options
burnt into my muscle memory.

Emacs's self-documentation features are not just for newbies, they are
for everyone.

Re: A better UI than perform-replace

[Eric Ludlam]

On 11/16/2015 06:47 AM, Oleh Krehel wrote:
> Dmitry Gutov <dgutov@yandex.ru> writes:
>
>> 5. Feel hugely motivated, and write a better UI for all of that.
>
> I think a better UI for `perform-replace' is warranted.  The current
> thing is very basic:
>
> - No good way to see how many matches there are.
> - No good way to get an overview of matches per buffer.
> - No good way to pause the replacement procedure.
> - No good way to undo a replacement.
>
> An idea to improve this would be with a permanent *replace* buffer,
> similar to `dired' or *Buffer List*. This buffer would be visible during
> the replacement operation, together with the actual buffers that contain
> the candidates. Here are some ideas for key bindings and the general
> interface:

You may be interested in the way semantic-symref handles broad renames. 
  The basic workflow is:

Place cursor in a declaration you want to rename

M-x semantic-symref RET

Or use

M-x semantic-symref-symbol RET

for an arbitrary symbol.

You then look through the hits, expand the ones you think are valid, 
collapse the ones you think aren't valid.  Or expand/contract all, of 
course.

Press R to start a rename.  Type in the new name.  Everything is 
replaced.   If you don't like it, just undo whichever buffer you made a 
mistake with.

Eric ... Following up on interesting things slowly.

Re: A better UI than perform-replace

[Dmitry Gutov]

On 11/20/2015 02:40 AM, Juri Linkov wrote:

> Then instead of tags-loop-continue/next-file currently used by etags.el,
> we could have a much simpler function multi-query-replace with bufs arg.

I'm not sure if that will make things that easier for 
xref--query-replace-1: it doesn't have a list of files, or a list of 
buffers. It has a list of pairs of markers.