regexp repacement, how to present replacement to user?

regexp repacement, how to present replacement to user?

[Paul Pogonyshev]

Hi,

I basically have a loop of `re-search-forward' with optional `replace-match'
after that.  I'd like to ask user if the particular match should be replaced
or not.  However, I'd like to present user with end-result strings, i.e. not
regexp and replacement with \N strings, but the match string and what
replacement would be in this case.  The former is as easy as (match-string 0),
but the second is not obvious how to do.

I.e., to clarify, I'd like this:

	Replace "foo-bar" with "foo bar"?

instead of this:

	Replace "\([[:alpha:]]+\)-\([[:alpha:]]+\)" with "\1 \2"?

Paul

Re: regexp repacement, how to present replacement to user?

[Juri Linkov]

> I basically have a loop of `re-search-forward' with optional `replace-match'
> after that.  I'd like to ask user if the particular match should be replaced
> or not.  However, I'd like to present user with end-result strings, i.e. not
> regexp and replacement with \N strings, but the match string and what
> replacement would be in this case.  The former is as easy as (match-string 0),
> but the second is not obvious how to do.
>
> I.e., to clarify, I'd like this:
>
> 	Replace "foo-bar" with "foo bar"?
>
> instead of this:
>
> 	Replace "\([[:alpha:]]+\)-\([[:alpha:]]+\)" with "\1 \2"?

Something like this?

(let ((from "\\([[:alpha:]]+\\)-\\([[:alpha:]]+\\)")
      (to "\\1 \\2"))
  (while (re-search-forward from nil t)
    (if (y-or-n-p (format "Replace %s with %s "
                          (match-string 0)
                          (replace-regexp-in-string from to (match-string 0))))
        (replace-match to nil nil))))

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: regexp repacement, how to present replacement to user?

[Stefan Monnier]

> (let ((from "\\([[:alpha:]]+\\)-\\([[:alpha:]]+\\)")
>       (to "\\1 \\2"))
>   (while (re-search-forward from nil t)
>     (if (y-or-n-p (format "Replace %s with %s "
>                           (match-string 0)
>                           (replace-regexp-in-string from to (match-string 0))))
>         (replace-match to nil nil))))

This work 99% with the last 1% being due to failure in boundary conditions
(e.g. if the pattern starts with "\\>").


        Stefan

Re: regexp repacement, how to present replacement to user?

[David Kastrup]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> (let ((from "\\([[:alpha:]]+\\)-\\([[:alpha:]]+\\)")
>>       (to "\\1 \\2"))
>>   (while (re-search-forward from nil t)
>>     (if (y-or-n-p (format "Replace %s with %s "
>>                           (match-string 0)
>>                           (replace-regexp-in-string from to (match-string 0))))
>>         (replace-match to nil nil))))
>
> This work 99% with the last 1% being due to failure in boundary conditions
> (e.g. if the pattern starts with "\\>").

     (while (re-search-forward from nil t)
        (if y-or-n-p (format "Replace %s with %s "
                       (match-string 0)
                          (save-match-data
                            (replace-match to nil nil (match-string 0)
                             (prog1 nil
                               (set-match-data
                                 (mapcar
                                   (lambda(x)(if (numberp x)
                                                   (- x (match-beginning 0))
                                                   x)))
                                 (match-data t)))

[...]

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: regexp repacement, how to present replacement to user?

[Paul Pogonyshev]

David Kastrup wrote:
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
> 
> >> (let ((from "\\([[:alpha:]]+\\)-\\([[:alpha:]]+\\)")
> >>       (to "\\1 \\2"))
> >>   (while (re-search-forward from nil t)
> >>     (if (y-or-n-p (format "Replace %s with %s "
> >>                           (match-string 0)
> >>                           (replace-regexp-in-string from to (match-string 0))))
> >>         (replace-match to nil nil))))
> >
> > This work 99% with the last 1% being due to failure in boundary conditions
> > (e.g. if the pattern starts with "\\>").
> 
>      (while (re-search-forward from nil t)
>         (if y-or-n-p (format "Replace %s with %s "
>                        (match-string 0)
>                           (save-match-data
>                             (replace-match to nil nil (match-string 0)
>                              (prog1 nil
>                                (set-match-data
>                                  (mapcar
>                                    (lambda(x)(if (numberp x)
>                                                    (- x (match-beginning 0))
>                                                    x)))
>                                  (match-data t)))

Thanks, but shouldn't this be a function somewhere?  I mean, it's
not like random peope are going to easily write such code.  Besides,
what if Emacs regexps are slightly improved with more features, like
named groups or whatnot?

Paul

Re: regexp repacement, how to present replacement to user?

[Juri Linkov]

>>      (while (re-search-forward from nil t)
>>         (if y-or-n-p (format "Replace %s with %s "
>>                        (match-string 0)
>>                           (save-match-data
>>                             (replace-match to nil nil (match-string 0)
>>                              (prog1 nil
>>                                (set-match-data
>>                                  (mapcar
>>                                    (lambda(x)(if (numberp x)
>>                                                    (- x (match-beginning 0))
>>                                                    x)))
>>                                  (match-data t)))
>
> Thanks, but shouldn't this be a function somewhere?  I mean, it's
> not like random peope are going to easily write such code.

`query-replace-regexp' already highlights the text matched by
`from-regexp' in the source buffer, so the user can see the text to
replace.  And the result of the replacement is also displayed in the
prompt in the expanded form, so the user sees the resulting string
before replacing it in the buffer.

The only thing I don't understand why the result of evaluating \,
expressions is displayed in the prompt, but the result of back references
is not.  I mean why, for instance,

C-M-% \(a\|b\) RET \,\1  \1 RET

displays

Query replacing \(a\|b\) with a \1:

but not

Query replacing \(a\|b\) with a a:

?

I think replacing \1 with the value of the back reference in the prompt
will be more useful for the user to see the result of the future replacement.

> Besides, what if Emacs regexps are slightly improved with more features,
> like named groups or whatnot?

Recently Stefan improved Emacs regexps with numbered groups where a number
can be explicitly assigned to a regexp group.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: regexp repacement, how to present replacement to user?

[David Kastrup]

Juri Linkov <juri@jurta.org> writes:

> The only thing I don't understand why the result of evaluating \,
> expressions is displayed in the prompt, but the result of back references
> is not.  I mean why, for instance,
>
> C-M-% \(a\|b\) RET \,\1  \1 RET
>
> displays
>
> Query replacing \(a\|b\) with a \1:
>
> but not
>
> Query replacing \(a\|b\) with a a:
>
> ?

For one thing, \, expressions are turned into Lisp at a much earlier
stage.  They are no longer available at that point of time.  For
another, this variant is very handy: you can write something like

C-M-% <<<<<\(blubb\)=====\(blah\)>>>> RET \1\? RET

(for things like conflict markers) and then make the decision to edit
\1 into \2 at each replacement.

Strictly speaking, this affects the replacement editing and not the
prompt display itself so it is not a direct argument against your
proposal.  But for large replacements of this kind, the prompt might
get quite unwieldy.  And I suspect that \1/\2 quite often can be very
much multiline if they capture something delimited by some markers.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: regexp repacement, how to present replacement to user?

[Paul Pogonyshev]

Juri Linkov wrote:
> >>      (while (re-search-forward from nil t)
> >>         (if y-or-n-p (format "Replace %s with %s "
> >>                        (match-string 0)
> >>                           (save-match-data
> >>                             (replace-match to nil nil (match-string 0)
> >>                              (prog1 nil
> >>                                (set-match-data
> >>                                  (mapcar
> >>                                    (lambda(x)(if (numberp x)
> >>                                                    (- x (match-beginning 0))
> >>                                                    x)))
> >>                                  (match-data t)))
> >
> > Thanks, but shouldn't this be a function somewhere?  I mean, it's
> > not like random peope are going to easily write such code.
> 
> `query-replace-regexp' already highlights the text matched by
> `from-regexp' in the source buffer, so the user can see the text to
> replace.  And the result of the replacement is also displayed in the
> prompt in the expanded form, so the user sees the resulting string
> before replacing it in the buffer.
> 
> The only thing I don't understand why the result of evaluating \,
> expressions is displayed in the prompt, but the result of back references
> is not.  I mean why, for instance,
> 
> C-M-% \(a\|b\) RET \,\1  \1 RET
> 
> displays
> 
> Query replacing \(a\|b\) with a \1:
> 
> but not
> 
> Query replacing \(a\|b\) with a a:

That is exactly what I'm asking for.  How to present user with expanded
string, without those \N.

In my case I want to scan a buffer for like 20 predefined regexps with
as well predefined replacements, meant only to somehow "standardize"
text.  User is not expected to edit replacement at all, he is expected
only to decide whether to replace, or to skip this case completely (e.g.
because it is a false match, regexps cannot cover all cases in a real
human-language text).

So, I want something like this:

	Replace 12,345.67 with 12 345,67 (Russian number format)?

The user doesn't need to see regexp or its replacement with back-
references.  He should see the text and proposed replacement and just
decide if it is OK, or not.  He is not interested how this particular
occurence was found, with a regexp or not, much less with which
regexp in particular.

> > Besides, what if Emacs regexps are slightly improved with more features,
> > like named groups or whatnot?
> 
> Recently Stefan improved Emacs regexps with numbered groups where a number
> can be explicitly assigned to a regexp group.

That's why I think this (replacement expansion with back-references) must
be a function inside Emacs and not written in a package.  For instance,
code (outside Emacs) written before wouldn't know about this enhancement
by Stefan.  Code written now cannot know in advance about possible future
enhancement.  But if one uses a Emacs function, he can expect that if his
code runs in e.g. Emacs 30, standard function will handle whatever regexp
syntax might evolve to by Emacs 30.

Paul

[patch] Re: regexp repacement, how to present replacement to user?

[Paul Pogonyshev]

I wrote:
> That's why I think this (replacement expansion with back-references) must
> be a function inside Emacs and not written in a package.

Here is a patch.  It adds two (actually one, other is just sugar) function.
Thanks to David Kastrup for the code.


2007-10-20  Paul Pogonyshev  <pogonyshev@gmx.net>

	* subr.el (match-substitute-replacement)
	(match-substitute-replacement-no-properties): New
	functions (code mostly by David Kastrup).


*** lisp/subr.el.~1.564.~	2007-08-30 22:17:43.000000000 +0300
--- lisp/subr.el	2007-10-20 16:48:21.709573376 +0300
***************
*** 2710,2715 ****
--- 2710,2745 ----
  	(buffer-substring-no-properties (match-beginning num)
  					(match-end num)))))
  
+ 
+ (defun match-substitute-replacement (replacement &optional fixedcase subexp)
+   "Return REPLACEMENT as it will be inserted by `replace-match'.
+ In other words, all back-references in the form `\\&' and `\\N'
+ are substituted with actual strings matched by the last search.
+ Optional FIXEDCASE abd SUBEXP have the same meaning as for
+ `replace-match'."
+   (let ((match (match-string 0)))
+     (save-match-data
+       (set-match-data (mapcar (lambda (x)
+ 				(if (numberp x)
+ 				    (- x (match-beginning 0))
+ 				  x))
+ 			      (match-data t)))
+       (replace-match replacement fixedcase nil match subexp))))
+ 
+ (defun match-substitute-replacement-no-properties (replacement
+ 						   &optional fixedcase subexp)
+   "Return REPLACEMENT as it will be inserted by `replace-match', without text properties.
+ See `match-substitute-replacement' for details."
+   (let ((match (match-string-no-properties 0)))
+     (save-match-data
+       (set-match-data (mapcar (lambda (x)
+ 				(if (numberp x)
+ 				    (- x (match-beginning 0))
+ 				  x))
+ 			      (match-data t)))
+       (replace-match replacement fixedcase nil match subexp))))
+ 
+ 
  (defun looking-back (regexp &optional limit greedy)
    "Return non-nil if text before point matches regular expression REGEXP.
  Like `looking-at' except matches before point, and is slower.

Re: [patch] Re: regexp repacement, how to present replacement to user?

[Richard Stallman]

This code makes sense, but why do we want it?

    2007-10-20  Paul Pogonyshev  <pogonyshev@gmx.net>

	    * subr.el (match-substitute-replacement)
	    (match-substitute-replacement-no-properties): New
	    functions (code mostly by David Kastrup).

If David Kastrup wrote the code, please put his name in the header.

Before installing this code, if we want to install it, we would need a
NEWS entry and changes for the Lisp manual.

Re: [patch] Re: regexp repacement, how to present replacement to user?

[Paul Pogonyshev]

Richard Stallman wrote:
> This code makes sense, but why do we want it?

These functions basically perform part of what `replace-match' does.
The latter substitutes replacement group references (`\N' and `\&')
and replaces match in the buffer with the result text.  Proposed
functions only generate and return that text, without modyfing the
buffer.

Currently, Emacs doesn't use this or similar functionality to my
knowledge.  However, this can be used in UI to present the user.
Especially useful if the user doesn't write regular expression and
replacement himself, but instead invokes some command that uses a
predefined regexp written by someone else.  E.g., for instance, it
would be quite cryptic if a user saw

    Replace \\([[:digit:]]+\\)\\.\\([[:digit:]]+\\) with \1,\2 (decimal comma)?

since he doesn't really care how this works internally at all.
Instead, seeing

    Replace 12.345 with 12,345 (decimal comma)?

is more understandable.  (This is an example of partly hypotetical
Elisp package standardizing numbers to use locale-specific comma
separator.)

> If David Kastrup wrote the code, please put his name in the header.

Done.

> Before installing this code, if we want to install it, we would need a
> NEWS entry and changes for the Lisp manual.

Done.


lisp ChangeLog entry:
2007-10-21  David Kastrup  <dak@gnu.org>

	* subr.el (match-substitute-replacement)
	(match-substitute-replacement-no-properties): New functions.

doc/lispref ChangeLog entry:
2007-10-21  Paul Pogonyshev  <pogonyshev@gmx.net>

	* searching.texi (Replacing Match): Describe new
	`match-substitute-replacement' and
	`match-substitute-replacement-no-properties'.

etc/NEWS entry:
** Two new functions `match-substitute-replacement' and
`match-substitute-replacement-no-properties' return the result of
`replace-match' without actually using it in the buffer.


Index: doc/lispref/searching.texi
===================================================================
RCS file: /cvsroot/emacs/emacs/doc/lispref/searching.texi,v
retrieving revision 1.2
diff -u -r1.2 searching.texi
--- doc/lispref/searching.texi	6 Sep 2007 04:27:40 -0000	1.2
+++ doc/lispref/searching.texi	21 Oct 2007 18:27:50 -0000
@@ -1260,6 +1260,22 @@
 just the text that matched @samp{\(ba*r\)}.
 @end defun
 
+@defun match-substitute-replacement replacement &optional fixedcase subexp
+This function returns the text that would be inserted into the buffer
+by @code{replace-match}, but without modifying the buffer.  It is
+useful if you want to present the user with actual replacement result,
+with constructs like @samp{\@var{n}} or @samp{\&} substituted with
+matched groups.  Arguments @var{replacement} and optional
+@var{fixedcase} and @var{subexp} have the same meaning as for
+@code{replace-match}.
+@end defun
+
+@defun match-substitute-replacement-no-properties replacement &optional fixedcase subexp
+This is like @code{match-substitute-replacement}, except that it
+returns text without any properties, just the characters themselves.
+@xref{Text Properties}.
+@end defun
+
 @node Simple Match Data
 @subsection Simple Match Data Access
 
Index: lisp/subr.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/subr.el,v
retrieving revision 1.564
diff -u -r1.564 subr.el
--- lisp/subr.el	29 Aug 2007 05:28:07 -0000	1.564
+++ lisp/subr.el	21 Oct 2007 18:27:54 -0000
@@ -2710,6 +2710,36 @@
 	(buffer-substring-no-properties (match-beginning num)
 					(match-end num)))))
 
+
+(defun match-substitute-replacement (replacement &optional fixedcase subexp)
+  "Return REPLACEMENT as it will be inserted by `replace-match'.
+In other words, all back-references in the form `\\&' and `\\N'
+are substituted with actual strings matched by the last search.
+Optional FIXEDCASE abd SUBEXP have the same meaning as for
+`replace-match'."
+  (let ((match (match-string 0)))
+    (save-match-data
+      (set-match-data (mapcar (lambda (x)
+				(if (numberp x)
+				    (- x (match-beginning 0))
+				  x))
+			      (match-data t)))
+      (replace-match replacement fixedcase nil match subexp))))
+
+(defun match-substitute-replacement-no-properties (replacement
+						   &optional fixedcase subexp)
+  "Return REPLACEMENT as it will be inserted by `replace-match', without text properties.
+See `match-substitute-replacement' for details."
+  (let ((match (match-string-no-properties 0)))
+    (save-match-data
+      (set-match-data (mapcar (lambda (x)
+				(if (numberp x)
+				    (- x (match-beginning 0))
+				  x))
+			      (match-data t)))
+      (replace-match replacement fixedcase nil match subexp))))
+
+
 (defun looking-back (regexp &optional limit greedy)
   "Return non-nil if text before point matches regular expression REGEXP.
 Like `looking-at' except matches before point, and is slower.

Re: [patch] Re: regexp repacement, how to present replacement to user?

[Juri Linkov]

> These functions basically perform part of what `replace-match' does.
> The latter substitutes replacement group references (`\N' and `\&')
> and replaces match in the buffer with the result text.  Proposed
> functions only generate and return that text, without modyfing the
> buffer.

Since new functions are similar to `replace-match', I think they
should also have similar names, e.g. `replace-match-string' or
`replace-match-as-string' meaning that it returns the replacement
as a string.  Also I think it should have exactly the same arguments
as `replace-match'.  And one function should be enough - there is no need
for -no-properties function because properties can be easily stripped
on the returned string when needed.

> Currently, Emacs doesn't use this or similar functionality to my
> knowledge.  However, this can be used in UI to present the user.
> Especially useful if the user doesn't write regular expression and
> replacement himself, but instead invokes some command that uses a
> predefined regexp written by someone else.  E.g., for instance, it
> would be quite cryptic if a user saw
>
>     Replace \\([[:digit:]]+\\)\\.\\([[:digit:]]+\\) with \1,\2 (decimal comma)?
>
> since he doesn't really care how this works internally at all.
> Instead, seeing
>
>     Replace 12.345 with 12,345 (decimal comma)?
>
> is more understandable.

I think there should be an option that will enable such understandable
replacements in the prompt (but not for multi-line replacements).

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: [patch] Re: regexp repacement, how to present replacement to user?

[Paul Pogonyshev]

Juri Linkov wrote:
> > These functions basically perform part of what `replace-match' does.
> > The latter substitutes replacement group references (`\N' and `\&')
> > and replaces match in the buffer with the result text.  Proposed
> > functions only generate and return that text, without modyfing the
> > buffer.
> 
> Since new functions are similar to `replace-match', I think they
> should also have similar names, e.g. `replace-match-string' or
> `replace-match-as-string' meaning that it returns the replacement
> as a string.

Mm, but they don't really replace anything, while such a name would
suggest it otherwise.  Though I don't really care, names in the patches
are certainly not optimal.

> Also I think it should have exactly the same arguments 
> as `replace-match'.  And one function should be enough - there is no need
> for -no-properties function because properties can be easily stripped
> on the returned string when needed.

OK.  I just followed `buffer-substring' and something else pattern.
Second function can certainly be dropped.

> > Currently, Emacs doesn't use this or similar functionality to my
> > knowledge.  However, this can be used in UI to present the user.
> > Especially useful if the user doesn't write regular expression and
> > replacement himself, but instead invokes some command that uses a
> > predefined regexp written by someone else.  E.g., for instance, it
> > would be quite cryptic if a user saw
> >
> >     Replace \\([[:digit:]]+\\)\\.\\([[:digit:]]+\\) with \1,\2 (decimal comma)?
> >
> > since he doesn't really care how this works internally at all.
> > Instead, seeing
> >
> >     Replace 12.345 with 12,345 (decimal comma)?
> >
> > is more understandable.
> 
> I think there should be an option that will enable such understandable
> replacements in the prompt (but not for multi-line replacements).

I agree.  But having this patch applied is a pre-requisite :)

Paul

Re: [patch] Re: regexp repacement, how to present replacement to user?

[Stefan Monnier]

> These functions basically perform part of what `replace-match' does.
> The latter substitutes replacement group references (`\N' and `\&')
> and replaces match in the buffer with the result text.  Proposed
> functions only generate and return that text, without modyfing the
> buffer.

I think the main part of it is the offsetting and that may be used at
other places.  So it's the only part we want to provide.  Then the user can
combine it with replace-match.

I suggest something like

  (defun match-data-move-base (start)
    "Change the `match-data' such that it assumes START to be at position 0.
  It can be combined for example as in:

    (progn
      (match-data-move-base (match-beginning 0))
      (replace-match (match-string 0)))"

    (set-match-data (mapcar (lambda (x)
				(if (numberp x)
				    (- x start)
				  x))
			      (match-data t))))

It could even take an `end' argument and clip the match-data to fit
within start..end.


        Stefan

Re: [patch] Re: regexp repacement, how to present replacement to user?

[Richard Stallman]

    These functions basically perform part of what `replace-match' does.
    The latter substitutes replacement group references (`\N' and `\&')
    and replaces match in the buffer with the result text.  Proposed
    functions only generate and return that text, without modyfing the
    buffer.

Yes, I understand that.  I agree that these functions make logical sense.

The question is, is this operation sufficiently useful that we should
add functions to Emacs to do it, and document them?

Re: [patch] Re: regexp repacement, how to present replacement to user?

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

>     These functions basically perform part of what `replace-match'
>     does.  The latter substitutes replacement group references (`\N'
>     and `\&') and replaces match in the buffer with the result text.
>     Proposed functions only generate and return that text, without
>     modyfing the buffer.
>
> Yes, I understand that.  I agree that these functions make logical
> sense.
>
> The question is, is this operation sufficiently useful that we
> should add functions to Emacs to do it, and document them?

Well, these sort of operations are sufficiently obscure that one
should add functions for them, I think.  The exact format of the match
data is more or less an implementation detail.  The core manipulation
used here strongly depends on those internals.

So even if it is not used that often, it leaves a bad feeling if some
code outside of the core of Emacs is required for doing this sort of
manipulation.

-- 
David Kastrup

Re: [patch] Re: regexp repacement, how to present replacement to user?

[Richard Stallman]

    > The question is, is this operation sufficiently useful that we
    > should add functions to Emacs to do it, and document them?

    Well, these sort of operations are sufficiently obscure that one
    should add functions for them, I think.

There must be thousands of things that are meaningful and obscure, and
it would be a change for the worse to complicate Emacs by adding them
all.  We should only add the ones that are substantially useful.  The
question is whether this is one of them.

Re: [patch] Re: regexp repacement, how to present replacement to user?

[Lennart Borgman (gmail)]

Richard Stallman wrote:
>     > The question is, is this operation sufficiently useful that we
>     > should add functions to Emacs to do it, and document them?
> 
>     Well, these sort of operations are sufficiently obscure that one
>     should add functions for them, I think.
> 
> There must be thousands of things that are meaningful and obscure, and
> it would be a change for the worse to complicate Emacs by adding them
> all.  We should only add the ones that are substantially useful.  The
> question is whether this is one of them.


Is not this something that can affect new users? It looks so to me from 
the situation where it can show up.

The lack of this feature might make Emacs look even more difficult than 
it is for a new user. It might also actually makes it more difficult for 
them by drawing their attention to something that is not worth their 
attention.

Re: [patch] Re: regexp repacement, how to present replacement to user?

[Richard Stallman]

    Is not this something that can affect new users? It looks so to me from 
    the situation where it can show up.

Most new users don't do a lot of Emacs Lisp programming, so no this
won't affect them.

Re: [patch] Re: regexp repacement, how to present replacement to user?

[Richard Stallman]

    I think the main part of it is the offsetting and that may be used at
    other places.  So it's the only part we want to provide.

The originally proposed function was much cleaner and easier to use
than this offsetting function.  If this facility is useful enough to
justify adding someting to Emacs, better to add the originally
proposed function.

The question is whether this is useful enough to be a reason to add
anything.

Re: [patch] Re: regexp repacement, how to present replacement to user?

[Paul Pogonyshev]

Richard Stallman wrote:
>     I think the main part of it is the offsetting and that may be used at
>     other places.  So it's the only part we want to provide.
> 
> The originally proposed function was much cleaner and easier to use
> than this offsetting function.  If this facility is useful enough to
> justify adding someting to Emacs, better to add the originally
> proposed function.
> 
> The question is whether this is useful enough to be a reason to add
> anything.

Sorry for long delay.

How about this addition using new function right inside Emacs.  In
`query-replace-regexp', hitting `t' (or anything else, doesn't matter)
will toggle between

	Query replacing REGEXP with REPLACEMENT: (C-h for help)

and

	Replace REGEXP with SUBSTITUTED-REPLACEMENT? (C-h for help)

This way you can preview replacement result, which can be useful if
your replacement string is complicated enough.

E.g. you will be able to toggle between

	Query replacing \([a-z]+\) \([a-z]+\) with \1 \2 \1: (C-h for help)

and

	Replace \([a-z]+\) \([a-z]+\) with foo bar foo? (C-h for help)

Optionally we can also toggle to `Replace MATCH...' instead of
`Replace REGEXP...', but that is probably not very useful given that
the match is highlighted in the buffer anyway.

Paul

Re: [patch] Re: regexp repacement, how to present replacement to user?

[Richard Stallman]

	    Replace \([a-z]+\) \([a-z]+\) with foo bar foo? (C-h for help)

This seems like a good reason to install the proposed facility and
make query-replace-regexp use it.  Would someone please install it?

But I don't think we need a feature to toggle between the two modes.
I think that showing the actual replacement string is simply superior.

Re: [patch] Re: regexp repacement, how to present replacement to user?

[Juri Linkov]

> 	    Replace \([a-z]+\) \([a-z]+\) with foo bar foo? (C-h for help)
>
> This seems like a good reason to install the proposed facility and
> make query-replace-regexp use it.  Would someone please install it?
>
> But I don't think we need a feature to toggle between the two modes.
> I think that showing the actual replacement string is simply superior.

I agree that a key to toggle between the two modes is not necessary,
but we should have an option to turn this feature off for users who
like the old behavior.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: [patch] Re: regexp repacement, how to present replacement to user?

[Richard Stallman]

    I agree that a key to toggle between the two modes is not necessary,
    but we should have an option to turn this feature off for users who
    like the old behavior.

I have no objection to that option.

Re: [patch] Re: regexp repacement, how to present replacement to user?

[Paul Pogonyshev]

Richard Stallman wrote:
>     I agree that a key to toggle between the two modes is not necessary,
>     but we should have an option to turn this feature off for users who
>     like the old behavior.
> 
> I have no objection to that option.

OK, here is a stab at making a patch.  I'm ommiting doc changes, since
this is most likely not going to be final version anyway.

I changed the new function to accept all arguments `replace-match'
does and removed the `-no-properties' variant, as Juri Linkov suggested.

It probably makes sense to make the replacement text in the
`query-regexp-replace' prompt stand out, by using quotes or a text
property.  However, I didn't implement this as current implementation
doesn't.

Paul

2007-10-31  Paul Pogonyshev  <pogonyshev@gmx.net>

	* replace.el (query-replace-substitute-replacement): New
	defcustom.
	(perform-replace): Use `match-substitute-replacement' if
	`query-replace-substitute-replacement' is non-nil.

2007-10-31  David Kastrup  <dak@gnu.org>

	* subr.el (match-substitute-replacement): New functions.


Index: lisp/replace.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/replace.el,v
retrieving revision 1.259
diff -u -r1.259 replace.el
--- lisp/replace.el	26 Jul 2007 05:26:32 -0000	1.259
+++ lisp/replace.el	31 Oct 2007 21:17:37 -0000
@@ -69,6 +69,12 @@
   :group 'matching
   :version "22.1")
 
+(defcustom query-replace-substitute-replacement t
+  "*Non-nil means to show what actual replacement text will be."
+  :type 'boolean
+  :group 'matching
+  :version "23.1")
+
 (defcustom query-replace-highlight t
   "*Non-nil means to highlight matches during query replacement."
   :type 'boolean
@@ -1570,10 +1576,17 @@
 		   (or delimited-flag regexp-flag) case-fold-search)
 		  ;; Bind message-log-max so we don't fill up the message log
 		  ;; with a bunch of identical messages.
-		  (let ((message-log-max nil))
+		  (let ((message-log-max nil)
+			(replacement-presentation
+			 (if query-replace-substitute-replacement
+			     (save-match-data
+			       (set-match-data real-match-data)
+			       (match-substitute-replacement next-replacement
+							     nocasify literal))
+			   next-replacement)))
 		    (message message
                              (query-replace-descr from-string)
-                             (query-replace-descr next-replacement)))
+                             (query-replace-descr replacement-presentation)))
 		  (setq key (read-event))
 		  ;; Necessary in case something happens during read-event
 		  ;; that clobbers the match data.
Index: lisp/subr.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/subr.el,v
retrieving revision 1.565
diff -u -r1.565 subr.el
--- lisp/subr.el	27 Oct 2007 09:07:15 -0000	1.565
+++ lisp/subr.el	31 Oct 2007 21:17:37 -0000
@@ -2709,6 +2709,24 @@
 	(buffer-substring-no-properties (match-beginning num)
 					(match-end num)))))
 
+
+(defun match-substitute-replacement (replacement
+				     &optional fixedcase literal string subexp)
+  "Return REPLACEMENT as it will be inserted by `replace-match'.
+In other words, all back-references in the form `\\&' and `\\N'
+are substituted with actual strings matched by the last search.
+Optional FIXEDCASE, LITERAL, STRING and SUBEXP have the same
+meaning as for `replace-match'."
+  (let ((match (match-string 0 string)))
+    (save-match-data
+      (set-match-data (mapcar (lambda (x)
+				(if (numberp x)
+				    (- x (match-beginning 0))
+				  x))
+			      (match-data t)))
+      (replace-match replacement fixedcase literal match subexp))))
+
+
 (defun looking-back (regexp &optional limit greedy)
   "Return non-nil if text before point matches regular expression REGEXP.
 Like `looking-at' except matches before point, and is slower.

Re: [patch] Re: regexp repacement, how to present replacement to user?

[Richard Stallman]

	    * replace.el (query-replace-substitute-replacement): New
	    defcustom.

That name is rather long, so please call it
`query-replace-show-replacement'.  (That is also a clearer
description of what the feature does.)

Other than that, it looks good to me.  But please write the 
changes for NEWS and the manuals before installing this.

Re: [patch] Re: regexp repacement, how to present replacement to user?

[Paul Pogonyshev]

Richard Stallman wrote:
> 	    * replace.el (query-replace-substitute-replacement): New
> 	    defcustom.
> 
> That name is rather long, so please call it
> `query-replace-show-replacement'.  (That is also a clearer
> description of what the feature does.)
> 
> Other than that, it looks good to me.  But please write the 
> changes for NEWS and the manuals before installing this.

Here is a patch.  I don't have CVS write access, so I'd be grateful
if someone installs this.

Paul


/etc/NEWS:
** C-M-% now shows replacement as it would look in the buffer, with
`\N' and `\&' substituted according to the match.  Old behavior can be
restored by customizing `query-replace-show-replacement'.

** New function `match-substitute-replacement' returns the result of
`replace-match' without actually using it in the buffer.

doc/lispref/ChangeLog:
2007-11-02  Paul Pogonyshev  <pogonyshev@gmx.net>

	* searching.texi (Replacing Match): Describe new
	`match-substitute-replacement' and
	`match-substitute-replacement-no-properties'.

doc/emacs/ChangeLog:
2007-11-02  Paul Pogonyshev  <pogonyshev@gmx.net>

	* search.texi (Query Replace): Mention
	`query-replace-show-replacement'.

lisp/ChangeLog:
2007-11-02  Paul Pogonyshev  <pogonyshev@gmx.net>

	* replace.el (query-replace-show-replacement): New defcustom.
	(perform-replace): Use `match-substitute-replacement' if
	`query-replace-show-replacement' is non-nil.

2007-11-02  David Kastrup  <dak@gnu.org>

	* subr.el (match-substitute-replacement): New functions.


Index: doc/lispref/searching.texi
===================================================================
RCS file: /cvsroot/emacs/emacs/doc/lispref/searching.texi,v
retrieving revision 1.2
diff -u -r1.2 searching.texi
--- doc/lispref/searching.texi	6 Sep 2007 04:27:40 -0000	1.2
+++ doc/lispref/searching.texi	2 Nov 2007 20:32:36 -0000
@@ -1260,6 +1260,16 @@
 just the text that matched @samp{\(ba*r\)}.
 @end defun
 
+@defun match-substitute-replacement replacement &optional fixedcase literal string subexp
+This function returns the text that would be inserted into the buffer
+by @code{replace-match}, but without modifying the buffer.  It is
+useful if you want to present the user with actual replacement result,
+with constructs like @samp{\@var{n}} or @samp{\&} substituted with
+matched groups.  Arguments @var{replacement} and optional
+@var{fixedcase}, @var{literal}, @var{string} and @var{subexp} have the
+same meaning as for @code{replace-match}.
+@end defun
+
 @node Simple Match Data
 @subsection Simple Match Data Access
 
Index: doc/emacs/search.texi
===================================================================
RCS file: /cvsroot/emacs/emacs/doc/emacs/search.texi,v
retrieving revision 1.1
diff -u -r1.1 search.texi
--- doc/emacs/search.texi	6 Sep 2007 04:48:32 -0000	1.1
+++ doc/emacs/search.texi	2 Nov 2007 20:32:36 -0000
@@ -1172,7 +1172,11 @@
   These commands highlight the current match using the face
 @code{query-replace}.  They highlight other matches using
 @code{lazy-highlight} just like incremental search (@pxref{Incremental
-Search}).
+Search}).  By default, @code{query-replace-regexp} will show
+substituted replacement string for the current match in the
+minibuffer.  If you want to keep special sequences @samp{\&} and
+@samp{\@var{n}} unexpanded, customize
+@code{query-replace-show-replacement} variable.
 
   The characters you can type when you are shown a match for the string
 or regexp are:
Index: lisp/replace.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/replace.el,v
retrieving revision 1.259
diff -u -r1.259 replace.el
--- lisp/replace.el	26 Jul 2007 05:26:32 -0000	1.259
+++ lisp/replace.el	2 Nov 2007 20:32:36 -0000
@@ -69,6 +69,12 @@
   :group 'matching
   :version "22.1")
 
+(defcustom query-replace-show-replacement t
+  "*Non-nil means to show what actual replacement text will be."
+  :type 'boolean
+  :group 'matching
+  :version "23.1")
+
 (defcustom query-replace-highlight t
   "*Non-nil means to highlight matches during query replacement."
   :type 'boolean
@@ -1570,10 +1576,17 @@
 		   (or delimited-flag regexp-flag) case-fold-search)
 		  ;; Bind message-log-max so we don't fill up the message log
 		  ;; with a bunch of identical messages.
-		  (let ((message-log-max nil))
+		  (let ((message-log-max nil)
+			(replacement-presentation
+			 (if query-replace-show-replacement
+			     (save-match-data
+			       (set-match-data real-match-data)
+			       (match-substitute-replacement next-replacement
+							     nocasify literal))
+			   next-replacement)))
 		    (message message
                              (query-replace-descr from-string)
-                             (query-replace-descr next-replacement)))
+                             (query-replace-descr replacement-presentation)))
 		  (setq key (read-event))
 		  ;; Necessary in case something happens during read-event
 		  ;; that clobbers the match data.
Index: lisp/subr.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/subr.el,v
retrieving revision 1.565
diff -u -r1.565 subr.el
--- lisp/subr.el	27 Oct 2007 09:07:15 -0000	1.565
+++ lisp/subr.el	2 Nov 2007 20:32:39 -0000
@@ -2709,6 +2709,24 @@
 	(buffer-substring-no-properties (match-beginning num)
 					(match-end num)))))
 
+
+(defun match-substitute-replacement (replacement
+				     &optional fixedcase literal string subexp)
+  "Return REPLACEMENT as it will be inserted by `replace-match'.
+In other words, all back-references in the form `\\&' and `\\N'
+are substituted with actual strings matched by the last search.
+Optional FIXEDCASE, LITERAL, STRING and SUBEXP have the same
+meaning as for `replace-match'."
+  (let ((match (match-string 0 string)))
+    (save-match-data
+      (set-match-data (mapcar (lambda (x)
+				(if (numberp x)
+				    (- x (match-beginning 0))
+				  x))
+			      (match-data t)))
+      (replace-match replacement fixedcase literal match subexp))))
+
+
 (defun looking-back (regexp &optional limit greedy)
   "Return non-nil if text before point matches regular expression REGEXP.
 Like `looking-at' except matches before point, and is slower.

Re: [patch] Re: regexp repacement, how to present replacement to user?

[Paul Pogonyshev]

Can anyone pleas install it?

http://lists.gnu.org/archive/html/emacs-devel/2007-11/msg00085.html

Paul

Re: [patch] Re: regexp repacement, how to present replacement to user?

[Juri Linkov]

>> Other than that, it looks good to me.  But please write the 
>> changes for NEWS and the manuals before installing this.
>
> Here is a patch.  I don't have CVS write access, so I'd be grateful
> if someone installs this.

Installed.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: [patch] Re: regexp repacement, how to present replacement to user?

[Richard Stallman]

That change has been installed, hasn't it?  I saw the diffs from
Savannah.