find-file-noselect needs save-match-data

find-file-noselect needs save-match-data

[Herbert Euler]

If `find-file-noselect' is without `save-match-data', there will be a
bug when `type-break-mode' is turned on.  To reproduce the bug, start
emacs with the -Q option and execute the following expressions:

(type-break-mode 1)
(setq s "This is a sentence")
(string-match "sen" s)
(substring s (match-beginning 0) (match-end 0))

The following patch helps understanding where is wrong:

*** files.el.~1.666.2.82.~      2007-06-04 23:06:41.000000000 -0400
--- files.el    2007-06-04 23:07:29.000000000 -0400
***************
*** 1422,1427 ****
--- 1422,1428 ----
  and visit all the matching files.  When wildcards are actually
  used and expanded, return a list of buffers that are visiting
  the various files."
+   (save-match-data
    (setq filename
        (abbreviate-file-name
         (expand-file-name filename)))
***************
*** 1593,1599 ****
          (setq buf (create-file-buffer filename))
          ;; find-file-noselect-1 may use a different buffer.
          (find-file-noselect-1 buf filename nowarn
!                               rawfile truename number))))))

  (defun find-file-noselect-1 (buf filename nowarn rawfile truename number)
    (let (error)
--- 1594,1600 ----
          (setq buf (create-file-buffer filename))
          ;; find-file-noselect-1 may use a different buffer.
          (find-file-noselect-1 buf filename nowarn
!                               rawfile truename number)))))))

  (defun find-file-noselect-1 (buf filename nowarn rawfile truename number)
    (let (error)

To make the patch short, many lines that should be re-indented are
left as they were.

Regards,
Guanpeng Xu

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

Re: find-file-noselect needs save-match-data

[martin rudalics]

While you're there, could you look at some other functions in
files.el as well?  At least `convert-standard-filename',
`file-truename', `file-name-sans-versions', `insert-directory',
could benefit from `save-match-data' too.

Re: find-file-noselect needs save-match-data

[Herbert Euler]

>While you're there, could you look at some other functions in
>files.el as well?  At least `convert-standard-filename',
>`file-truename', `file-name-sans-versions', `insert-directory',
>could benefit from `save-match-data' too.

I just got a possibly wrong idea today morning.  The error happens in
the case that some command, which changes the match data, is added
into a hook.  When the hook is executed, the match data is also
changed.  Now, if we could assume that _hooks shall not change the
match data_, we can save and restore the match data automatically when
we run hooks.  I am not sure about this assumption, though.

Adding `save-match-data' to functions one by one works, but it is
difficult to check all of the functions, and it is not the overall
solution.

How do you think?

Regards,
Guanpeng Xu

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

Re: find-file-noselect needs save-match-data

[Richard Stallman]

Please install this patch in Emacs 22.

Re: find-file-noselect needs save-match-data

[martin rudalics]

 > I just got a possibly wrong idea today morning.  The error happens in
 > the case that some command, which changes the match data, is added
 > into a hook.  When the hook is executed, the match data is also
 > changed.  Now, if we could assume that _hooks shall not change the
 > match data_, we can save and restore the match data automatically when
 > we run hooks.  I am not sure about this assumption, though.

`save-match-data' is not cheap.  I suppose it's not advisable to
automatically wrap every hook into it.

 > Adding `save-match-data' to functions one by one works, but it is
 > difficult to check all of the functions, and it is not the overall
 > solution.

I think that `find-file-noselect' and `file-truename' should not change
match-data regardless of whether they run in a hook or not (though a
number of hooks should probably not run `find-file-noselect' in the
first place).

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    `save-match-data' is not cheap.  I suppose it's not advisable to
    automatically wrap every hook into it.

Not every hook.  On the other hand, find-file is slow, so a little
extra slowness in find-file is not a problem.

    I think that `find-file-noselect' and `file-truename' should not change
    match-data regardless of whether they run in a hook or not

I agree, expecially for `file-truename'.
Functions that just manipulate file names ought to be made easy
to call without worries.

Re: find-file-noselect needs save-match-data

[Herbert Euler]

[-- Attachment #1: Type: text/plain, Size: 1562 bytes --]

> > I just got a possibly wrong idea today morning.  The error happens in
> > the case that some command, which changes the match data, is added
> > into a hook.  When the hook is executed, the match data is also
> > changed.  Now, if we could assume that _hooks shall not change the
> > match data_, we can save and restore the match data automatically when
> > we run hooks.  I am not sure about this assumption, though.
>
>`save-match-data' is not cheap.  I suppose it's not advisable to
>automatically wrap every hook into it.
>
> > Adding `save-match-data' to functions one by one works, but it is
> > difficult to check all of the functions, and it is not the overall
> > solution.
>
>I think that `find-file-noselect' and `file-truename' should not change
>match-data regardless of whether they run in a hook or not (though a
>number of hooks should probably not run `find-file-noselect' in the
>first place).

Ok.  So I wrote a script to find out all such defuns.  I found many
defuns are possible needing `save-match-data' with it.  The script is
attached, and to start finding, first load the file and then execute
`list-possible-wrong-match-defuns-in-dir'.  An exception in the
executing is for some files a question about whether to apply local
variables list.

Should we check all these defuns and add `save-match-data' to some of
them?

Regards,
Guanpeng Xu

_________________________________________________________________
FREE pop-up blocking with the new MSN Toolbar - get it now! 
http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/

[-- Attachment #2: ss.el --]
[-- Type: text/plain, Size: 3665 bytes --]

(defvar change-match-data-calling
  (regexp-opt '("(search-forward "
		"(search-backward "
		"(word-search-forward "
		"(word-search-backward "
		"(re-search-forward "
		"(re-search-backward "
		"(string-match "
		"(looking-at "
		"(looking-back "
		"(search-spaces-regexp "
		"(posix-search-forward "
		"(posix-search-backward "
		"(posix-looking-at "
		"(posix-string-match "
		"(set-match-data ")))

(defvar defun-form
  (regexp-opt '("(defun "
		"(define-minor-mode "
		"(define-derived-mode ")))

(defun find-possible-wrong-match-defuns-in-buffer (buffer)
  (let (possible-wrong-functions)
    (catch 'skip-this-file
      (with-current-buffer buffer
	(goto-char (point-min))
	;; Find a calling.
	(while (re-search-forward change-match-data-calling nil t)
	  (let ((valid t)
		func-name)
	    (catch 'not-a-defun
	      (condition-case nil
		  (progn
		    (let ((last-pos (point)))
		      (mark-defun)
		      (unless (looking-at (concat "\n" defun-form))
			(goto-char last-pos)
			(throw 'not-a-defun t)))
		    (setq func-name (save-excursion
				      (down-list)
				      (forward-sexp)
				      (skip-syntax-forward " ")
				      (buffer-substring-no-properties (point)
								      (save-excursion
									(forward-sexp)
									(point)))))
		    (save-restriction
		      (narrow-to-region (region-beginning) (region-end))
		      ;; Check each calling.
		      (while (and valid (re-search-forward change-match-data-calling nil 
t))
			(save-excursion
			  (backward-up-list)
			  (catch 'found-save-match-data
			    (condition-case nil
				(while t
				  (backward-up-list)
				  (if (looking-at "(save-match-data")
				      (throw 'found-save-match-data t)))
			      (error (setq valid nil))))))
		      (unless valid
			(setq possible-wrong-functions (cons func-name
							     possible-wrong-functions))
			(while (re-search-forward change-match-data-calling nil t)
			  nil))))
		(error (throw 'skip-this-file t))))))))
    possible-wrong-functions))

(defun find-possible-wrong-match-defuns-in-file (file)
  (when (file-exists-p file)
    (let* ((buffer (find-file-noselect file))
	   (possible-wrong-functions (find-possible-wrong-match-defuns-in-buffer 
buffer)))
      (kill-buffer buffer)
      (cons file possible-wrong-functions))))

(defun find-possible-wrong-match-defuns-in-dir (dir)
  (when (file-exists-p dir)
    (let (functions-list)
      (dolist (file (directory-files dir))
	(when (not (or (string= file ".")
		       (string= file "..")
		       (string-match "elc$" file)))
	  (let* ((fullname (concat dir (if (not (string-match "/$" dir))
					   "/"
					 "") file))
		 (isdir (car (file-attributes fullname)))
		 functions
		 sub-functions-list)
	    (if isdir
		(progn (setq sub-functions-list
			     (find-possible-wrong-match-defuns-in-dir fullname))
		       (when sub-functions-list
			 (setq functions-list (append sub-functions-list functions-list))))
	      (setq functions (find-possible-wrong-match-defuns-in-file fullname))
	      (if (> (length functions) 1)
		  (setq functions-list (cons functions functions-list)))))))
      functions-list)))

(defun list-possible-wrong-match-defuns-in-dir (dir)
  (interactive "DDirectory: ")
  (let ((functions-list (find-possible-wrong-match-defuns-in-dir dir))
	(buffer (get-buffer-create "*possible wrong match defuns*")))
    (with-current-buffer buffer
      (dolist (functions functions-list)
	(insert "In " (car functions) ":\n")
	(dolist (function (cdr functions))
	  (insert "\t" function "\n"))
	(insert "\n")))
    (delete-other-windows)
    (split-window-vertically)
    (other-window 1)
    (switch-to-buffer buffer)
    (goto-char (point-min))))

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: find-file-noselect needs save-match-data

[Stefan Monnier]

> If `find-file-noselect' is without `save-match-data', there will be a
> bug when `type-break-mode' is turned on.  To reproduce the bug, start
> emacs with the -Q option and execute the following expressions:

I think this patch is wrong-headed: the bug is in type-break rather than in
files.el.  Maybe there's also a documentation bug somewhere that explains
this type-break bug.


        Stefan

Re: find-file-noselect needs save-match-data

[Stefan Monnier]

> While you're there, could you look at some other functions in
> files.el as well?  At least `convert-standard-filename',
> `file-truename', `file-name-sans-versions', `insert-directory',
> could benefit from `save-match-data' too.

No.  Most functions should not need to save-match-data (which is an
expensive operation very rarely needed).  The place to save-match-data is
typically on functions that use regexps and are placed on some sensitive
hooks (or at the pbvious place when a function needs to call
regexp-manipulating code between a regexp-match and its use of the
match-data).


        Stefan

Re: find-file-noselect needs save-match-data

[Herbert Euler]

> > While you're there, could you look at some other functions in
> > files.el as well?  At least `convert-standard-filename',
> > `file-truename', `file-name-sans-versions', `insert-directory',
> > could benefit from `save-match-data' too.
>
>No.  Most functions should not need to save-match-data (which is an
>expensive operation very rarely needed).  The place to save-match-data is
>typically on functions that use regexps and are placed on some sensitive
>hooks (or at the pbvious place when a function needs to call
>regexp-manipulating code between a regexp-match and its use of the
>match-data).

I don't understand.  In this case, `find-file-noselect' uses regexps,
and is placed on `post-command-hook' indirectly (I don't know whether
`post-command-hook' is sensitive).

Following your opinion, I conclude that it's type-break's
responsibility to know `find-file-noselect' uses regexps and so it
should use `save-match-data' when placing it to `post-command-hook'.
While the name `find-file-noselect' does not indicate it uses regexps,
and so does its document, it is more or less not appropriate of this
conclusion to me.

Regards,
Guanpeng Xu

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

Re: find-file-noselect needs save-match-data

[martin rudalics]

 > No.  Most functions should not need to save-match-data (which is an
 > expensive operation very rarely needed).  The place to save-match-data is
 > typically on functions that use regexps and are placed on some sensitive
 > hooks

... like when you put a function calling `find-file-noselect' or
`file-truename' on a sensitive hook.  You may have to look at the code
of each function on the entire call tree to detect whether one of them
might change match-data.  Since hardly anyone ever does that we'll have
to live with spurious, very hard-to-detect bugs clobbering match-data.
 From the caller's point of view it's also hard to conceive why, for
example, the simpler `file-chase-links' does save match-data while the
more complicated `file-truename' doesn't.

The real problem is that most calls of `string-match' (and `looking-at')
don't make use of match-data anyway.  `file-truename', for example, has

	((and (string= (substring filename 0 1) "~")
	      (string-match "~[^/]*/?" filename))

and

	  (unless (string-match "[[*?]" filename)
	    ;; If filename exists, use the long name
	    (setq filename (or (w32-long-file-name filename) filename))))

Both throw away match-data immediately.  If match-data is expensive why
build it in the first place when it's not needed?  Why not make a thing
like `fast-string-match' available in Lisp?

Re: find-file-noselect needs save-match-data

[martin rudalics]

 > I don't understand.  In this case, `find-file-noselect' uses regexps,
 > and is placed on `post-command-hook' indirectly (I don't know whether
 > `post-command-hook' is sensitive).

Every hook is sensitive.

 > Following your opinion, I conclude that it's type-break's
 > responsibility to know `find-file-noselect' uses regexps and so it
 > should use `save-match-data' when placing it to `post-command-hook'.
 > While the name `find-file-noselect' does not indicate it uses regexps,
 > and so does its document, it is more or less not appropriate of this
 > conclusion to me.

In any case we fill the stack with unneeded match-data.

Re: find-file-noselect needs save-match-data

[David Kastrup]

martin rudalics <rudalics@gmx.at> writes:

>> I don't understand.  In this case, `find-file-noselect' uses regexps,
>> and is placed on `post-command-hook' indirectly (I don't know whether
>> `post-command-hook' is sensitive).
>
> Every hook is sensitive.
>
>> Following your opinion, I conclude that it's type-break's
>> responsibility to know `find-file-noselect' uses regexps and so it
>> should use `save-match-data' when placing it to `post-command-hook'.
>> While the name `find-file-noselect' does not indicate it uses regexps,
>> and so does its document, it is more or less not appropriate of this
>> conclusion to me.
>
> In any case we fill the stack with unneeded match-data.

Stupid question: why does save-match-data not use (match-data t) and
thus avoid the creation of markers?

Match-data usually is _not_ adjusted for insertion/deletion in the
buffer, is it?  So it would seem rather self-defeating that

(save-match-data
  (manipulate the buffer))

will adjust the match data for insertions/deletions in the buffer
while

(manipulate the buffer)

in itself won't (assuming that it does not do any matching itself, of
course).

Try the following:

(with-temp-buffer
  (list (progn
          (insert "woozle")
          (goto-char 1)
          (search-forward "z")
          (insert-before-markers "x")
          (match-beginning 0))
        (match-end 0)
        (progn
           (goto-char 1)
           (search-forward "z")
           (save-match-data
             (insert-before-markers "x"))
           (match-beginning 0))
        (match-end 0)))

It will output (4 5 4 6).  Namely, the _expensive_ operation of
save-match-data _changes_ the outcome.

I think this is a mistake.

Here is the code:

(defmacro save-match-data (&rest body)
  "Execute the BODY forms, restoring the global value of the match data.
The value returned is the value of the last form in BODY."
  ;; It is better not to use backquote here,
  ;; because that makes a bootstrapping problem
  ;; if you need to recompile all the Lisp files using interpreted code.
  (declare (indent 0) (debug t))
  (list 'let
	'((save-match-data-internal (match-data)))
	(list 'unwind-protect
	      (cons 'progn body)
	      ;; It is safe to free (evaporate) markers immediately here,
	      ;; as Lisp programs should not copy from save-match-data-internal.
	      '(set-match-data save-match-data-internal 'evaporate))))

If we call (match-data t) here, we don't need the 'evaporate argument.
There is one potential problem: if some filter function uses
save-match-data for the _undocumented side effect_ of adjusting the
match data upon insertion, then this change will cause a change in
behavior.

So it is probably not something we want to try in the EMACS_22_BASE
branch.  But I definitely think we want it in the trunk.  It seems way
more sane, in addition to being more efficient.

-- 
David Kastrup

Re: find-file-noselect needs save-match-data

[Stefan Monnier]

> Following your opinion, I conclude that it's type-break's
> responsibility to know `find-file-noselect' uses regexps and so it
> should use `save-match-data' when placing it to `post-command-hook'.
> While the name `find-file-noselect' does not indicate it uses regexps,
> and so does its document, it is more or less not appropriate of this
> conclusion to me.

99% of non-trivial functions use regexps.  So you cannot assume a function
doesn't use regexps just because it doesn't say that it uses regexps.
It's rather the opposite: a function can only be assumed not to use regexps
if it is really trivial/primitive (e.g. `car' or `expand-file-name').

Otherwise we'll end up having to either add save-match-data around most
functions, or add a note in their docstring that they may modify the
match-data.


        Stefan

Re: find-file-noselect needs save-match-data

[Stefan Monnier]

> ... like when you put a function calling `find-file-noselect' or
> `file-truename' on a sensitive hook.  You may have to look at the code
> of each function on the entire call tree to detect whether one of them
> might change match-data.  Since hardly anyone ever does that we'll have
> to live with spurious, very hard-to-detect bugs clobbering match-data.
> From the caller's point of view it's also hard to conceive why, for
> example, the simpler `file-chase-links' does save match-data while the
> more complicated `file-truename' doesn't.

I'd say it's just an unexpected feature of file-chase-links that it saves
match-data.  It shouldn't need to.

> Both throw away match-data immediately.  If match-data is expensive why
> build it in the first place when it's not needed?  Why not make a thing
> like `fast-string-match' available in Lisp?

It's not the match data that's expensive.  It's the save-match-data.
The match-data is built cheaply during the call to string-match in the form
of a C data structure.  It's only when `match-data' is called that it ends
up building a Lisp lisp (sometimes with markers to boot).


        Stefan

Re: find-file-noselect needs save-match-data

[Lennart Borgman (gmail)]

Stefan Monnier wrote:

> It's not the match data that's expensive.  It's the save-match-data.
> The match-data is built cheaply during the call to string-match in the form
> of a C data structure.  It's only when `match-data' is called that it ends
> up building a Lisp lisp (sometimes with markers to boot).


Would it be possible to rewrite save-match-data in C then?

Re: find-file-noselect needs save-match-data

[martin rudalics]

 >>Both throw away match-data immediately.  If match-data is expensive why
 >>build it in the first place when it's not needed?  Why not make a thing
 >>like `fast-string-match' available in Lisp?
 >
 >
 > It's not the match data that's expensive.  It's the save-match-data.
 > The match-data is built cheaply during the call to string-match in the form
 > of a C data structure.  It's only when `match-data' is called that it ends
 > up building a Lisp lisp (sometimes with markers to boot).

match-data is expensive because it consumes (1) brain resources when
people have to think about saving it before they write another match
operation, and (2) processing resources when they decide to save it.

Re: find-file-noselect needs save-match-data

[Richard Stallman]

     > I don't understand.  In this case, `find-file-noselect' uses regexps,
     > and is placed on `post-command-hook' indirectly (I don't know whether
     > `post-command-hook' is sensitive).

    Every hook is sensitive.

I don't think so.  Many hooks are called from places
where it just doesn't matter.  For instance, mode hooks.
A major mode can do just about anything, and callers
had better not expect one to preserve the match data.
Therefore, there is no reason to take care about the functions
called by the mode hook, either.

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    99% of non-trivial functions use regexps.  So you cannot assume a function
    doesn't use regexps just because it doesn't say that it uses regexps.
    It's rather the opposite: a function can only be assumed not to use regexps
    if it is really trivial/primitive (e.g. `car' or `expand-file-name').

I almost agree with you.  There are some functions that aren't
literally primitive, but seem like things that should be safe to call.
file-truename is an example.  It would be good to make file-truename
safe regarding regexps.

find-file-noselect doesn't have to be regexp-safe, but why not make it
regexp-safe, since it is so slow.

However, we shouldn't go around trying to make all Emacs functions
regexp-safe.

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    > Both throw away match-data immediately.  If match-data is expensive why
    > build it in the first place when it's not needed?  Why not make a thing
    > like `fast-string-match' available in Lisp?

    It's not the match data that's expensive.  It's the save-match-data.
    The match-data is built cheaply during the call to string-match in the form
    of a C data structure.  It's only when `match-data' is called that it ends
    up building a Lisp lisp (sometimes with markers to boot).

That is true, but still it might be good to add a variant of
string-match that doesn't alter the match-data.

This function won't be faster than string-match, but it could be a lot
faster than (save-match-data (string-match...)), which is the way
to do this now.

Perhaps the name should be `safe-string-match'.

Re: find-file-noselect needs save-match-data

[Stefan Monnier]

> That is true, but still it might be good to add a variant of
> string-match that doesn't alter the match-data.

I don't see a strong need for it.  And why only string-match?  Wouldn't we
want to do the same for looking-at, and re-search-forward, and ...?

> This function won't be faster than string-match, but it could be a lot
> faster than (save-match-data (string-match...)), which is the way
> to do this now.

> Perhaps the name should be `safe-string-match'.

I don't find it much safer, so maybe `string-match-nodata' would be better?

Then again, maybe adding an `inhibit-match-data' variable would
be preferable.


        Stefan

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    > Perhaps the name should be `safe-string-match'.

    I don't find it much safer, so maybe `string-match-nodata' would be better?

"safe" seems good to me.  "nodata" doesn't seem to express a clear meaning.

    Then again, maybe adding an `inhibit-match-data' variable would
    be preferable.

That idea could be useful.  I'd prefer the name `preserve-match-data',
or `set-match-data' (which you could bind to nil).

Re: find-file-noselect needs save-match-data

[Kim F. Storm]

Richard Stallman <rms@gnu.org> writes:

>     > Perhaps the name should be `safe-string-match'.
>
>     I don't find it much safer, so maybe `string-match-nodata' would be better?
>
> "safe" seems good to me.  "nodata" doesn't seem to express a clear meaning.

It's quite the opposite IMO.  "safe" can mean many things, whereas
"nodata" clearly indicates that it doesn't set (or modify) match-data.



>     Then again, maybe adding an `inhibit-match-data' variable would
>     be preferable.
>
> That idea could be useful.  I'd prefer the name `preserve-match-data',
> or `set-match-data' (which you could bind to nil).


Again, I like the inhibit- name, as other similar variables use the same
prefix, so it will be easier to remember.

But in practice, we should have a macro to encapsulate calls:

    (preserve-match-data (string-match ...))


-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: find-file-noselect needs save-match-data

[martin rudalics]

 > It's quite the opposite IMO.  "safe" can mean many things, whereas
 > "nodata" clearly indicates that it doesn't set (or modify) match-data.

"-nodata" is good.

 > But in practice, we should have a macro to encapsulate calls:
 >
 >     (preserve-match-data (string-match ...))

This would violate referential transparency.  Consider

(defvar foobar "foobar")

(defun foo ()
   (string-match "foo" foobar)
   (match-string 0 foobar))

(defun bar ()
   (string-match "bar" foobar)
   (preserve-match-data
    (foo)))

Re: find-file-noselect needs save-match-data

[Kim F. Storm]

martin rudalics <rudalics@gmx.at> writes:

>> It's quite the opposite IMO.  "safe" can mean many things, whereas
>> "nodata" clearly indicates that it doesn't set (or modify) match-data.
>
> "-nodata" is good.
>
>> But in practice, we should have a macro to encapsulate calls:
>>
>>     (preserve-match-data (string-match ...))
>
> This would violate referential transparency.  Consider
>
> (defvar foobar "foobar")
>
> (defun foo ()
>   (string-match "foo" foobar)
>   (match-string 0 foobar))
>
> (defun bar ()
>   (string-match "bar" foobar)
>   (preserve-match-data
>    (foo)))


Well, I specifically intended the preserve-match-data to be used
in "closed contexts" like

   (preserve-match-data (looking-at ...))
   (preserve-match-data (string-match ...))

etc.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    > This would violate referential transparency.  Consider
    >
    > (defvar foobar "foobar")
    >
    > (defun foo ()
    >   (string-match "foo" foobar)
    >   (match-string 0 foobar))
    >
    > (defun bar ()
    >   (string-match "bar" foobar)
    >   (preserve-match-data
    >    (foo)))

If referential transparency is something which dynamic variables
"violate", then it isn't relevant to Emacs.

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    It's quite the opposite IMO.  "safe" can mean many things, whereas
    "nodata" clearly indicates that it doesn't set (or modify) match-data.

Just "data" doesn't clearly refer to the match data.

What this function would do is avoid side effects.
What other names do people use to mean "without side effects"?

    Again, I like the inhibit- name, as other similar variables use the same

The name `inhibit-match-data' doesn't fit with a feature that inhibit
_setting_ the match data.  Other variables called `inhibit-...'
followed by a the name of some data inhibit the data from being obeyed
or acted on.  It doesn't seem right to use a similar name to prevent
setting the data.

Re: find-file-noselect needs save-match-data

[Juri Linkov]

>     It's quite the opposite IMO.  "safe" can mean many things, whereas
>     "nodata" clearly indicates that it doesn't set (or modify) match-data.
>
> Just "data" doesn't clearly refer to the match data.
>
> What this function would do is avoid side effects.
> What other names do people use to mean "without side effects"?

"Pure".  Though `pure-string-match' doesn't refer to the match data
at all.  It can refer to many different things.

I see no better name than `string-match-no-data' because it clearly
says that it doesn't set match-data according to three words in its name:
"no", "match", "data".

We already have a very similar name `match-string-no-properties'
which returns a string without text properties.  By analogy
`string-match-no-data' would return the match index without setting
the match data.

>     Again, I like the inhibit- name, as other similar variables use the same
>
> The name `inhibit-match-data' doesn't fit with a feature that inhibit
> _setting_ the match data.  Other variables called `inhibit-...'
> followed by a the name of some data inhibit the data from being obeyed
> or acted on.  It doesn't seem right to use a similar name to prevent
> setting the data.

What about `inhibit-set-match-data'?

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: find-file-noselect needs save-match-data

[David House]

On 09/06/07, Juri Linkov <juri@jurta.org> wrote:
> I see no better name than `string-match-no-data' because it clearly
> says that it doesn't set match-data according to three words in its name:
> "no", "match", "data".

How about string-match-no-match-data? "no-data" is a bit vague, but
this should be clear.

-- 
-David House, dmhouse@gmail.com

Re: find-file-noselect needs save-match-data

[Juri Linkov]

>> I see no better name than `string-match-no-data' because it clearly
>> says that it doesn't set match-data according to three words in its name:
>> "no", "match", "data".
>
> How about string-match-no-match-data? "no-data" is a bit vague, but
> this should be clear.

IMHO, the substring "match-no-match" is too confusing.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: find-file-noselect needs save-match-data

[Jason Rumney]

David House wrote:
> How about string-match-no-match-data? "no-data" is a bit vague, but
> this should be clear.

It's no more clear than any other suggestion so far. The problem here is
what does "no-match-data" mean? It seems to mean
"without-setting-match-data", but that isn't what I would expect from
that name.

Re: find-file-noselect needs save-match-data

[Miles Bader]

"David House" <dmhouse@gmail.com> writes:
> How about string-match-no-match-data? "no-data" is a bit vague, but
> this should be clear.

string-match-it-was-a-dark-and-stormy-night?

FWIW, I don't really have strong feelings either way, but I think
"string-match-no-data" is "clear enough" -- one can't expect function
names to essentially provide documentation.  It's enough that they be
easily recognizable to someone who's used them before, and provide a
good hint to those who haven't.

"string-match-no-data" satisfies those criteria well I think, and
"string-match-no-match-data" is just a little too long and
redundant-seeming.

-miles

-- 
If you can't beat them, arrange to have them beaten.  [George Carlin]

RE: find-file-noselect needs save-match-data

[Drew Adams]

> string-match-it-was-a-dark-and-stormy-night?

string-match-it-was-a-dark-and-stormy-night-with-no-match

Re: find-file-noselect needs save-match-data

[Herbert Euler]

> > It's quite the opposite IMO.  "safe" can mean many things, whereas
> > "nodata" clearly indicates that it doesn't set (or modify) match-data.
>
>"-nodata" is good.
>
> > But in practice, we should have a macro to encapsulate calls:
> >
> >     (preserve-match-data (string-match ...))
>
>This would violate referential transparency.  Consider
>
>(defvar foobar "foobar")
>
>(defun foo ()
>   (string-match "foo" foobar)
>   (match-string 0 foobar))
>
>(defun bar ()
>   (string-match "bar" foobar)
>   (preserve-match-data
>    (foo)))

Since most applications do not use the match data in practice, I think
if an application use the match data, it should require it explicitly.
I.e., please take a look at the following:

    ;; Normal applications (no use of match data).
    (if (string-match "foo" bar)
        ;; True branch.
      ;; False branch.)
    (foo-bar)

    ;; Applications that use match-data.
    (if (string-match "foo" bar)
        (with-match-data (match-data)
          ;; True branch.)
      ;; No need to use `with-match-data' here, since there is no
      ;; match data.
      ;; False branch.)
    (foo-bar)

IMO, if we can adapt to this model easily, we can make least changes
to existing code.

Regards,
Guanpeng Xu

_________________________________________________________________
FREE pop-up blocking with the new MSN Toolbar - get it now! 
http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/

Re: find-file-noselect needs save-match-data

[Herbert Euler]

>Since most applications do not use the match data in practice, I think
>if an application use the match data, it should require it explicitly.
>I.e., please take a look at the following:
>
>    ;; Normal applications (no use of match data).
>    (if (string-match "foo" bar)
>        ;; True branch.
>      ;; False branch.)
>    (foo-bar)
>
>    ;; Applications that use match-data.
>    (if (string-match "foo" bar)
>        (with-match-data (match-data)
>          ;; True branch.)
>      ;; No need to use `with-match-data' here, since there is no
>      ;; match data.
>      ;; False branch.)
>    (foo-bar)

And of course functions like `string-match' do not set match data
by default, as if `inhibit-match-data' is turned on by default.

Regards,
Guanpeng Xu

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    I see no better name than `string-match-no-data' because it clearly
    says that it doesn't set match-data according to three words in its name:
    "no", "match", "data".

It doesn't express that message.  You see the message which you
know is intended.

    What about `inhibit-set-match-data'?

That name is suitable, but I hope to find a three-word name
that is also suitable.

    How about string-match-no-match-data? "no-data" is a bit vague, but
    this should be clear.

That name is clear, but I'd rather find a shorter one.

Re: find-file-noselect needs save-match-data

[Bob Rogers]

   From: Richard Stallman <rms@gnu.org>
   Date: Sun, 10 Jun 2007 09:19:37 -0400

   . . .

       How about string-match-no-match-data? "no-data" is a bit vague, but
       this should be clear.

   That name is clear, but I'd rather find a shorter one.

The only possible use for a "string-match" that doesn't set the match
data is the boolean return value, right?  So how about "string-match-p"?
[And my apologies if this suggestion is not to the point; I'm having
trouble keeping up with emacs-devel these days.]

					-- Bob Rogers
					   http://rgrjr.dyndns.org/

Re: find-file-noselect needs save-match-data

[Miles Bader]

Bob Rogers <rogers-emacs@rgrjr.dyndns.org> writes:
> The only possible use for a "string-match" that doesn't set the match
> data is the boolean return value, right?  So how about "string-match-p"?

Er, no.  string-match returns the index of the match (or nil for no match),
which is actually quite useful.

-miles
-- 
((lambda (x) (list x x)) (lambda (x) (list x x)))

Re: find-file-noselect needs save-match-data

[Juri Linkov]

> Bob Rogers <rogers-emacs@rgrjr.dyndns.org> writes:
>> The only possible use for a "string-match" that doesn't set the match
>> data is the boolean return value, right?  So how about "string-match-p"?
>
> Er, no.  string-match returns the index of the match (or nil for no match),
> which is actually quite useful.

Since it returns the value of (match-beginning 0), how about the name
`string-match-0'?  It is short, and zero has additional mnemonic that
no data is set by this function.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: find-file-noselect needs save-match-data

[Andreas Schwab]

Juri Linkov <juri@jurta.org> writes:

>> Bob Rogers <rogers-emacs@rgrjr.dyndns.org> writes:
>>> The only possible use for a "string-match" that doesn't set the match
>>> data is the boolean return value, right?  So how about "string-match-p"?
>>
>> Er, no.  string-match returns the index of the match (or nil for no match),
>> which is actually quite useful.
>
> Since it returns the value of (match-beginning 0), how about the name
> `string-match-0'?  It is short, and zero has additional mnemonic that
> no data is set by this function.

How about string-match-no-record?

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: find-file-noselect needs save-match-data

[David Kastrup]

Andreas Schwab <schwab@suse.de> writes:

> Juri Linkov <juri@jurta.org> writes:
>
>>> Bob Rogers <rogers-emacs@rgrjr.dyndns.org> writes:
>>>> The only possible use for a "string-match" that doesn't set the match
>>>> data is the boolean return value, right?  So how about "string-match-p"?
>>>
>>> Er, no.  string-match returns the index of the match (or nil for no match),
>>> which is actually quite useful.
>>
>> Since it returns the value of (match-beginning 0), how about the name
>> `string-match-0'?  It is short, and zero has additional mnemonic that
>> no data is set by this function.
>
> How about string-match-no-record?

What about string-match-beginning?  Or string-match-save?  Or
stealthy-string-match?

The possibilities are endless.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: find-file-noselect needs save-match-data

[Miles Bader]

David Kastrup <dak@gnu.org> writes:
>> How about string-match-no-record?
>
> What about string-match-beginning?  Or string-match-save?  Or
> stealthy-string-match?

I think all of those are misleading.

"string-match-reentrant"?

-miles

-- 
"Most attacks seem to take place at night, during a rainstorm, uphill,
 where four map sheets join."   -- Anon. British Officer in WW I

Re: find-file-noselect needs save-match-data

[David Kastrup]

Miles Bader <miles@gnu.org> writes:

> David Kastrup <dak@gnu.org> writes:
>>> How about string-match-no-record?
>>
>> What about string-match-beginning?  Or string-match-save?  Or
>> stealthy-string-match?
>
> I think all of those are misleading.
>
> "string-match-reentrant"?

What use is reentrancy in a routine that does not call Lisp and can't
be called in interrupts?

Anyway, such a function should likely _return_ the corresponding
match-data (rather than setting a global variable to it), and things
like match-beginning should optionally take an argument for this
particular match-data.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: find-file-noselect needs save-match-data

[Richard Stallman]

I like string-match-p as a way of indicating that all it does is return
a value.  It is ok for a predicate to return true values other than t.

string-match-reentrant is also good, but reentrant-string-match
is better.

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    > I think all of those are misleading.
    >
    > "string-match-reentrant"?

    What use is reentrancy in a routine that does not call Lisp and can't
    be called in interrupts?

The question need not be answered.  We want a meaningful and clear way
to distinguish this function from `string-match'.  The fact that it is
reentrant is a good way to distinguish.  The name looks good.

    Anyway, such a function should likely _return_ the corresponding
    match-data (rather than setting a global variable to it), and things
    like match-beginning should optionally take an argument for this
    particular match-data.

That is a good idea.  If there is no match, it returns nil; if there
is a match, it returns the match data.

Re: find-file-noselect needs save-match-data

[Stefan Monnier]

>     Anyway, such a function should likely _return_ the corresponding
>     match-data (rather than setting a global variable to it), and things
>     like match-beginning should optionally take an argument for this
>     particular match-data.

> That is a good idea.  If there is no match, it returns nil; if there
> is a match, it returns the match data.

Then what's the advantage compared to using save-match-data?  This whole
discussion started because people want to add save-match-data everywhere and
I pointed out that it's very rarely necessary, and it can be costly (where
the "it" is generating the Lisp-level match data).
The suggestion to have a specialized string-match operation that preserves
match-data was for performance reasons, AFAICT, like fast_c_string_match.


        Stefan

Re: find-file-noselect needs save-match-data

[David Kastrup]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>     Anyway, such a function should likely _return_ the corresponding
>>     match-data (rather than setting a global variable to it), and things
>>     like match-beginning should optionally take an argument for this
>>     particular match-data.
>
>> That is a good idea.  If there is no match, it returns nil; if there
>> is a match, it returns the match data.
>
> Then what's the advantage compared to using save-match-data?

That it's reentrant...  And that it does not touch the match data.
The idea was to have a function that does not touch global data, yet
delivers the full information.

Now one problem here is that we don't want the delivery of information
to be expensive, so consing together a list is probably not really the
hottest idea.

> This whole discussion started because people want to add
> save-match-data everywhere and I pointed out that it's very rarely
> necessary, and it can be costly (where the "it" is generating the
> Lisp-level match data).  The suggestion to have a specialized
> string-match operation that preserves match-data was for performance
> reasons, AFAICT, like fast_c_string_match.

The main problem I currently see is that the current semantics and
usage would want to have save-match-data used in places where actually
nobody changes match-data.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: find-file-noselect needs save-match-data

[Stefan Monnier]

> The main problem I currently see is that the current semantics and
> usage would want to have save-match-data used in places where actually
> nobody changes match-data.

I'm not sure I understand.
AFAIK save-match-data is only needed in 2 circumstances:

1 - when implementing a function that needs to leave the
    match-data unaffected.  This is typically only the case for functions
    that are placed on hooks.  In most cases such save-match-data are
    necessary because the function you define does touch the match-data
    (otherwise you wouldn't have used save-match-data) and the function that
    runs the hook will sooner or later need the match-data to
    stay unaffected.

2 - around code that needs to be run after string-match but before match-end
    and of which we do not know that it leaves the match-data unaffected.
    This may be unnecessary because the code may in fact preserve the
    match-data even though you don't know it.

So although number 1 may be an unnecessary use of save-match-data, I find
it's usually useful/necessary.  Number 2 OTOH can often turn out to be
unnecessary.  Luckily number 2 is very rarely needed because in most cases
it's easier to restructure your code such that save-match-data is not needed
any more.

So I don't see current usage as problematic w.r.t efficiency.
The only real problem with it is that people tend to not know/understand it
and are thus tempted to complain that most functions need to be wrapped in
save-match-data, which then leads to long discussions such as this one.
Maybe we should make the convention more prominent by adding it to the
docstring of save-match-data?


        Stefan

Re: find-file-noselect needs save-match-data

[Miles Bader]

Richard Stallman <rms@gnu.org> writes:
>     Anyway, such a function should likely _return_ the corresponding
>     match-data (rather than setting a global variable to it), and things
>     like match-beginning should optionally take an argument for this
>     particular match-data.
>
> That is a good idea.  If there is no match, it returns nil; if there
> is a match, it returns the match data.

One alternative to _returning_ the match data would be accepting an
argument -- a vector or list -- in which to _place_ the match data
(presumably this would be optional, and after the "normal" args).

This would have the advantage that it avoids consing the match-data
unless the user actually wants it, and yet remains convenient if the
user _does_ want it.

E.g:

   (string-match-reentrant REGEXP STRING &optional START MATCH-DATA-VEC)

with usage like:

   (let ((mdata (vector 10)))
     (if (string-match-reentrant regexp string nil mdata)
         (message "blargh [%s] [%s]" (aref mdata 1) (aref mdata 2))))

[I'm not sure exactly the format of mdata; there'd have to be some sort
of accessor functions]

The disadvantage of this is that it would mean allocating the
return-vector every time the caller wants match-data, even if there ends
up being no match -- though often I guess this overhead could be reduced
when a piece of code can use the same vector for many different calls to
string-match-reentrant.

Which alternative better depends on the expected frequency of various
cases, but I think being able to easily avoid the allocation in the
extremely common "just match no data" case is valuable, so I'd tend to
think my above solution is preferable.

-Miles

-- 
`Suppose Korea goes to the World Cup final against Japan and wins,' Moon said.
`All the past could be forgiven.'   [NYT]

Re: find-file-noselect needs save-match-data

[Stefan Monnier]

More seriously: I feel like we're wasting our time here trying to solve
a non-problem.


        Stefan

Re: find-file-noselect needs save-match-data

[Herbert Euler]

>So I don't see current usage as problematic w.r.t efficiency.
>The only real problem with it is that people tend to not know/understand it
>and are thus tempted to complain that most functions need to be wrapped in
>save-match-data, which then leads to long discussions such as this one.
>Maybe we should make the convention more prominent by adding it to the
>docstring of save-match-data?

This is probably true.  I found this strange behavior when I studied
the previous SKIP_SYNTAX_BACKWARD patch.  I grep'ed to get all the
places that invoke SKIP_SYNTAX_BACKWARD, and looked at them one by
one.  Several are in regex.c.  I wrote many expressions in the
*scratch* buffer and evaluated them with C-x C-e sequentially to make
sure all the places in regex.c get called.  When I evaluated the three
expressions in the beginning of this thread, I got wrong results and
started not to understand it.  Starting Emacs with -Q, I found the
result is correct.  So I knew it's something wrong with my
configuration.  Finally, I knew it's in type-break, and then I knew
the problem is caused by `find-file-noselect' which is put on
`post-command-hook' and changes the match data.

I missed something, at least the efficiency.  And the operations above
is truely rare: I don't think people would do that often too.  If in
most cases such a wrong result will not happen, it is then not a
problem for common usage.  Unless some mode makes the wrong cases
common, it is safe to ignore it.

However, this is surely a shortcoming of Emacs.

Regards,
Guanpeng Xu

_________________________________________________________________
Don't just search. Find. Check out the new MSN Search! 
http://search.msn.click-url.com/go/onm00200636ave/direct/01/

Re: find-file-noselect needs save-match-data

[martin rudalics]

 > Then what's the advantage compared to using save-match-data?  This whole
 > discussion started because people want to add save-match-data everywhere and
 > I pointed out that it's very rarely necessary, and it can be costly (where
 > the "it" is generating the Lisp-level match data).

Yes, I proposed to add `save-match-data' to a variety of file handling
functions at the outermost level.  The main reason for my proposal was
to introduce some sort of consistency among functions in files.el with
respect to saving match-data.  If you proposed to remove _all_ instances
of `save-match-data's from files.el instead, you get my 100% agreement.
ISTR also that I was the first in this thread to mention that saving
`match-data' is expensive.  If we really cared about this we could start
right in files.el.  The following instance in `abbreviate-file-name'

			 (save-match-data
			   (string-match "^[a-zA-`]:/$" filename)))))

should be replaced by binding the outcome of (match-string 1) of the
previous string-match around it.  Though using

			 (fast-string-match "^[a-zA-`]:/$" filename))))

instead would be clearer and make such binding unnecessary ;-)

 > The suggestion to have a specialized string-match operation that preserves
 > match-data was for performance reasons, AFAICT, like fast_c_string_match.

It was my suggestion to introduce - on the Lisp level - somthing like
fast_string_match.  My motivation was completely different though: Code
using that function should reassure the reader that there would be no
need to save match-data around it.  It goes without saying that a "fast"
string matching function should return the same value as `string-match'
and _not_ `match-data'.

I still believe that people spend more valuable time _thinking_ about
whether match-data should be saved in a specific circumstance than when
it is actually saved on their computer.  Look at info.el: In
`Info-dir-remove-duplicates' we have

	    (while (re-search-forward "^\\(\\w.*\\)\n\\*" limit t)
	      (when (if re (save-match-data (string-match re (match-string 1)))
		      (equal name (match-string 1)))
		
and in `Info-fontify-node'

			(cond ((save-match-data (looking-back "\\<see"))
			       "")
			      ((save-match-data (looking-back "\\<in"))
			       "")
			      ((memq (char-before) '(nil ?\. ?! ??))
                                "See ")
			      ((save-match-data
				 (save-excursion
				   (search-forward "\n\n" start t)))
			       "See ")
			      (t "see "))))

both deeply nested in loops.  Did I mention that Info's fontification
can be a pain?

Re: find-file-noselect needs save-match-data

[martin rudalics]

 > AFAIK save-match-data is only needed in 2 circumstances:
 >
 > 1 - when implementing a function that needs to leave the
 >     match-data unaffected.  This is typically only the case for functions
 >     that are placed on hooks.  In most cases such save-match-data are
 >     necessary because the function you define does touch the match-data
 >     (otherwise you wouldn't have used save-match-data) and the function that
 >     runs the hook will sooner or later need the match-data to
 >     stay unaffected.

Exact.

 > 2 - around code that needs to be run after string-match but before match-end

I fail to understand this.

 >     and of which we do not know that it leaves the match-data unaffected.
 >     This may be unnecessary because the code may in fact preserve the
 >     match-data even though you don't know it.
 >
 > So although number 1 may be an unnecessary use of save-match-data, I find
 > it's usually useful/necessary.  Number 2 OTOH can often turn out to be
 > unnecessary.  Luckily number 2 is very rarely needed because in most cases
 > it's easier to restructure your code such that save-match-data is not needed
 > any more.

Saving match-data is needed if the index for accessing match-data is
known only after performing a second match operation like in

(let (index)
   (when (re-search-forward ...)
     (save-match-data
       (if (string-match ...)
	  (setq index ...)))
     (match-beginning index)))

I think we agree that this is rarely used.

 > So I don't see current usage as problematic w.r.t efficiency.

The info.el examples in my earlier mail seem to contradict this.

 > The only real problem with it is that people tend to not know/understand it
 > and are thus tempted to complain that most functions need to be wrapped in
 > save-match-data, which then leads to long discussions such as this one.
 > Maybe we should make the convention more prominent by adding it to the
 > docstring of save-match-data?

We probably should remove bad uses of `save-match-data' from Emacs code
in the first place.

Re: find-file-noselect needs save-match-data

[martin rudalics]

 > More seriously: I feel like we're wasting our time here trying to solve
 > a non-problem.

If the question is whether `fast-string-match' should return match-data
or the same value as `string-match' we're not only wasting our time here
but in the future as well.  `fast-string-match' (or whatever you want to
call it) has to behave exactly like `string-match' but for leaving the
internal value of match data untouched.

Re: find-file-noselect needs save-match-data

[Stefan Monnier]

>> 2 - around code that needs to be run after string-match but before match-end

> I fail to understand this.

>> and of which we do not know that it leaves the match-data unaffected.
>> This may be unnecessary because the code may in fact preserve the
>> match-data even though you don't know it.
>> 
>> So although number 1 may be an unnecessary use of save-match-data, I find
>> it's usually useful/necessary.  Number 2 OTOH can often turn out to be
>> unnecessary.  Luckily number 2 is very rarely needed because in most cases
>> it's easier to restructure your code such that save-match-data is not needed
>> any more.

> Saving match-data is needed if the index for accessing match-data is
> known only after performing a second match operation like in

Yes, this is the number 2 I'm thinking of.

>> So I don't see current usage as problematic w.r.t efficiency.
> The info.el examples in my earlier mail seem to contradict this.

I'm not saying it can't be misused.  But using it right isn't that hard.
The info.el example should be fixed.

>> The only real problem with it is that people tend to not know/understand it
>> and are thus tempted to complain that most functions need to be wrapped in
>> save-match-data, which then leads to long discussions such as this one.
>> Maybe we should make the convention more prominent by adding it to the
>> docstring of save-match-data?

> We probably should remove bad uses of `save-match-data' from Emacs code
> in the first place.

Agreed.
As for removing most uses of save-match-data from files.el, I could agree.
I probably have argued against most of the ones I've seen added there, but
usually the other side wins because "well, it can't hurt to add it just
here".  So little by little it creeps in.


        Stefan

Re: find-file-noselect needs save-match-data

[Herbert Euler]

Excuse me but just another question: as Stefan said, 99% of
non-trivial functions use regexps, does it mean that these 99%
non-trivial functions shall not be put on the hooks that may be run
between two regexp-sensitive functions/commands unless the putter
saves match-data?  Or, does it mean that this type of hooks need
`save-match-data'?  `pre-command-hook', `post-command-hook',
`before-change-functions' and `after-change-functions' seem to be some
of the hooks.  I notice that `c-before-change' and `c-after-change'
use `save-match-data'.  Are these applications of `save-match-data'
trivial, or should other modes save match-data when putting functions
on the hooks?

Regards,
Guanpeng Xu

Btw, `post-command-hook' is the source of the original problem.

_________________________________________________________________
FREE pop-up blocking with the new MSN Toolbar - get it now! 
http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/

Re: find-file-noselect needs save-match-data

[Stefan Monnier]

>>>>> "Herbert" == Herbert Euler <herberteuler@hotmail.com> writes:

> If `find-file-noselect' is without `save-match-data', there will be a
> bug when `type-break-mode' is turned on.  To reproduce the bug, start
> emacs with the -Q option and execute the following expressions:

> (type-break-mode 1)
> (setq s "This is a sentence")
> (string-match "sen" s)
> (substring s (match-beginning 0) (match-end 0))

How do you execute it?

I tried M-x type-break-mode RET
and then M-: (progn (setq s "This is a sentence")
                    (string-match "sen" s)
                    (substring s (match-beginning 0) (match-end 0))) RET

and it worked just fine.  I guess what you did is to use a separate M-: (or
C-x C-e or M-C-x) for each line.  Then the result is completely expected:
there's a lot of code executed between each interactive command, so of
course the match data can't be assumed to be preserved.


        Stefan

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    >     Anyway, such a function should likely _return_ the corresponding
    >     match-data (rather than setting a global variable to it), and things
    >     like match-beginning should optionally take an argument for this
    >     particular match-data.

    Then what's the advantage compared to using save-match-data?

This function would be simpler, and probably noticeably faster, than

    (save-match-data
       (string-match X Y)
       (match-data))

You have a valid point that this function would be somewhat
slower than a similar function which returned only the position
of the start of the match.  I am not sure whether this speed
difference is significant.  Would someone like to try
implementing those two functions, and see how they compare
in speed to the expression above?

Re: find-file-noselect needs save-match-data

[Richard Stallman]

       (let ((mdata (vector 10)))
	 (if (string-match-reentrant regexp string nil mdata)
	     (message "blargh [%s] [%s]" (aref mdata 1) (aref mdata 2))))

That is complicated enough that it would be easier to use
save-match-data.  Therefore, we may as well not add this
extra complexity.

We have seen two possible simple specs for this function's return
value: return just the location of the start of the match, and return
the entire match data.  Stefan argued against the latter on grounds
of efficiency, so I suggested trying both approaches and measuring
the efficiency.

Re: find-file-noselect needs save-match-data

[Stefan Monnier]

> Excuse me but just another question: as Stefan said, 99% of
> non-trivial functions use regexps, does it mean that these 99%
> non-trivial functions shall not be put on the hooks that may be run
> between two regexp-sensitive functions/commands unless the putter
> saves match-data?

Normally, none of these functions will ever be put on a hook.
OTOH they may be called from other functions which are put on hooks.
It's the responsibility of *those* functions to use save-match-data.

Just like in the OP: find-file-noselect is not put on any hook.  It's some
other function from type-break which is put on a hook, so if there is a bug
it's in this other function, not in find-file-noselect.

> Btw, `post-command-hook' is the source of the original problem.

Indeed.  If the problem you posted is considered as a bug (I wouldn't
consider it as such), then a possible fix is the patch below:


        Stefan


--- orig/lisp/type-break.el
+++ mod/lisp/type-break.el
@@ -740,7 +740,7 @@
 \f
 (defun type-break-run-tb-post-command-hook ()
   (and type-break-mode
-       (run-hooks 'type-break-post-command-hook)))
+       (save-match-data (run-hooks 'type-break-post-command-hook))))
 
 (defun type-break-check ()
   "Ask to take a typing break if appropriate.
@@ -1274,5 +1274,5 @@
 (if type-break-mode
     (type-break-mode 1))
 
-;;; arch-tag: 943a2eb3-07e6-420b-993f-96e4796f5fd0
+;; arch-tag: 943a2eb3-07e6-420b-993f-96e4796f5fd0
 ;;; type-break.el ends here

Re: find-file-noselect needs save-match-data

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

>     >     Anyway, such a function should likely _return_ the corresponding
>     >     match-data (rather than setting a global variable to it), and things
>     >     like match-beginning should optionally take an argument for this
>     >     particular match-data.
>
>     Then what's the advantage compared to using save-match-data?
>
> This function would be simpler, and probably noticeably faster, than
>
>     (save-match-data
>        (string-match X Y)
>        (match-data))
>
> You have a valid point that this function would be somewhat
> slower than a similar function which returned only the position
> of the start of the match.  I am not sure whether this speed
> difference is significant.  Would someone like to try
> implementing those two functions, and see how they compare
> in speed to the expression above?

I think we have the problem that
a) save-match-data is a Lisp macro using (match-data), and
b) match-data constructs a very expensive data structure: it takes
quite a bit of consing, the default is to get markers, and accessing
any piece of match data is O(n) instead of O(1).

The basic global match-data structure is an array.  Maybe we should
make save-match-data a cheap C operation.  Another possibility for an
(incompatible) API change would be that save-match-data _clears_ the
match-data.  That way, nested calls of save-match-data become cheap
since the inner levels don't need to allocate anything.

Once save-match-data becomes cheap (and nested calls very cheap), one
can propose that, as a rule, non-interactive functions should leave
the match-data alone.

All of the proposals of when to call and when not to call
save-match-data revolve around the problem that save-match-data is an
expensive operation. If we get rid of that problem, we don't need to
weigh advantages and disadvantages that much.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: find-file-noselect needs save-match-data

[Stefan Monnier]

> the entire match data.  Stefan argued against the latter on grounds
> of efficiency, so I suggested trying both approaches and measuring
> the efficiency.

Actually, I'd even argue against any such new function, on the grounds that
it's solving a non-problem, i.e. it will just add new functions, new code,
new complexity for no real benefit.


        Stefan

Re: find-file-noselect needs save-match-data

[Herbert Euler]

> > Excuse me but just another question: as Stefan said, 99% of
> > non-trivial functions use regexps, does it mean that these 99%
> > non-trivial functions shall not be put on the hooks that may be run
> > between two regexp-sensitive functions/commands unless the putter
> > saves match-data?
>
>Normally, none of these functions will ever be put on a hook.
>OTOH they may be called from other functions which are put on hooks.
>It's the responsibility of *those* functions to use save-match-data.
>
>Just like in the OP: find-file-noselect is not put on any hook.  It's some
>other function from type-break which is put on a hook, so if there is a bug
>it's in this other function, not in find-file-noselect.

Yes, this is the reason that I got wrong result (I mean some function
is put on a hook and it invokes functions like `find-file-noselect').
And the functions (like `find-file-noselect') are indeed not proper to
be put on a hook.

> > Btw, `post-command-hook' is the source of the original problem.
>
>Indeed.  If the problem you posted is considered as a bug (I wouldn't
>consider it as such), then a possible fix is the patch below:

I did execute them as several single expressions, with several single
interactive commands.  As you said, "there's a lot of code executed
between each interactive command, so of course the match data can't be
assumed to be preserved."  This also convinces me that the problem I
posted is not a bug, although it may cause a surprise.  (I'm not sure
whether it's worthy to make it more clear to users, to avoid such a
surprising result.  Or I was too stupid not to understand the
behavior.)

Saving interesting match data immediately after invoking matching
functions, is that a good usage to avoid error caused by match data
changing?  If so, it seems to be an overall solution.

Thank you.

Regards,
Guanpeng Xu

_________________________________________________________________
FREE pop-up blocking with the new MSN Toolbar - get it now! 
http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/

Re: find-file-noselect needs save-match-data

[martin rudalics]

 > Actually, I'd even argue against any such new function,

How about adding another optional argument then?


string-match is a built-in function in `C source code'.
(string-match regexp string &optional start preserve-match-data))

Return index of start of first match for REGEXP in STRING, or nil.
Matching ignores case if `case-fold-search' is non-nil.
If third arg START is non-nil, start search at that index in string.

For index of first char beyond the match, do (match-end 0).
`match-end' and `match-beginning' also give indices of substrings
matched by parenthesis constructs in the pattern.
You can use the function `match-string' to extract the substrings
matched by the parenthesis constructions in REGEXP.

Fourth argument PRESERVE-MATCH-DATA non-nil means do not modify match
data.  This means you cannot use `match-beginning', `match-end', or
`match-string' to extract further details of the current match.
On the other hand, you don't have to use `save-match-data' in order
to preserve data of a previous match.


 > on the grounds that
 > it's solving a non-problem, i.e. it will just add new functions, new code,
 > new complexity for no real benefit.

Consider `abbreviate-file-name'.  It has the following construct:

       (if (and (string-match abbreviated-home-dir filename)
	       ;; If the home dir is just /, don't change it.
	       (not (and (= (match-end 0) 1)
			 (= (aref filename 0) ?/)))
	       ;; MS-DOS root directories can come with a drive letter;
	       ;; Novell Netware allows drive letters beyond `Z:'.
	       (not (and (or (eq system-type 'ms-dos)
			     (eq system-type 'cygwin)
			     (eq system-type 'windows-nt))
			 (save-match-data
			   (string-match "^[a-zA-`]:/$" filename)))))
	  (setq filename
		(concat "~"
			(match-string 1 filename)
			(substring filename (match-end 0)))))


How would you eliminate the call to `save-match-data' without
introducing new variables and complexity?  I could simply write:

       (if (and (string-match abbreviated-home-dir filename)
	       ;; If the home dir is just /, don't change it.
	       (not (and (= (match-end 0) 1)
			 (= (aref filename 0) ?/)))
	       ;; MS-DOS root directories can come with a drive letter;
	       ;; Novell Netware allows drive letters beyond `Z:'.
	       (not (and (or (eq system-type 'ms-dos)
			     (eq system-type 'cygwin)
			     (eq system-type 'windows-nt))
			 (string-match "^[a-zA-`]:/$" filename nil t))))
	  (setq filename
		(concat "~"
			(match-string 1 filename)
			(substring filename (match-end 0)))))

Re: find-file-noselect needs save-match-data

[Richard Stallman]

      Finally, I knew it's in type-break, and then I knew
    the problem is caused by `find-file-noselect' which is put on
    `post-command-hook' and changes the match data.

I think it is a bug to expect the match data to be preserved across
`post-command-hook'.  Which code expects that?  We should fix that
code to protect itself.

However, I think `find-file-noselect' may as well call `save-match-data'
for the sake of other places it may be used.

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    Yes, I proposed to add `save-match-data' to a variety of file handling
    functions at the outermost level.  The main reason for my proposal was
    to introduce some sort of consistency among functions in files.el with
    respect to saving match-data.

I think that the file-manipulation functions that seem clean and
abstract ought to call `save-match-data'.

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    Excuse me but just another question: as Stefan said, 99% of
    non-trivial functions use regexps, does it mean that these 99%
    non-trivial functions shall not be put on the hooks that may be run
    between two regexp-sensitive functions/commands unless the putter
    saves match-data?  Or, does it mean that this type of hooks need
    `save-match-data'?

It depends on how the hook is used.

			`pre-command-hook', `post-command-hook',
    `before-change-functions' and `after-change-functions' seem to be some
    of the hooks.

`pre-command-hook' and `post-command-hook' are not called from many
places.  The principal one is the main command loop, which does not
care if the match data is clobbered by the hook.  Therefore, anyplace
else that runs these hooks should make sure not to care either.

`before-change-functions' and `after-change-functions' can be called
from lots of primitives, including `insert'.  It would be a horrible
pain to have to worry that all these primitives might clobber the
match data.  Therefore, any hook functions put on these hooks must
preserve the match data.

That should be documented in the doc strings of those hooks
and in the Lisp Ref Manual.  Would someone like to do that?

Re: find-file-noselect needs save-match-data

[Richard Stallman]

Please install your type-break.el patch.

However, this doesn't mean we shouldn't also add a few calls
to save-match-data in functions that seem as if you could trust
them not to change the match data.

Re: find-file-noselect needs save-match-data

[Stefan Monnier]

>> Actually, I'd even argue against any such new function,
> How about adding another optional argument then?

Same deal: you're trying to solve a non-problem.

>> on the grounds that it's solving a non-problem, i.e. it will just add new
>> functions, new code, new complexity for no real benefit.

> Consider `abbreviate-file-name'.  It has the following construct:

>       (if (and (string-match abbreviated-home-dir filename)
> 	       ;; If the home dir is just /, don't change it.
> 	       (not (and (= (match-end 0) 1)
> 			 (= (aref filename 0) ?/)))
> 	       ;; MS-DOS root directories can come with a drive letter;
> 	       ;; Novell Netware allows drive letters beyond `Z:'.
> 	       (not (and (or (eq system-type 'ms-dos)
> 			     (eq system-type 'cygwin)
> 			     (eq system-type 'windows-nt))
> 			 (save-match-data
> 			   (string-match "^[a-zA-`]:/$" filename)))))
> 	  (setq filename
> 		(concat "~"
> 			(match-string 1 filename)
> 			(substring filename (match-end 0)))))

Looks fine to me.  I don't find it particularly unreadable and its
efficiency seems to be perfectly adequate.


        Stefan


PS: By the way, the above code looks odd.  The comment seems to imply that
the MS-DOS check just reproduces the preceding / check, but it's not
the case: the / check makes sure that if HOME is / then /a/b will not be
changed to ~/a/b, whereas the MS-DOS check will only prevent changing A:/ to
~ but it will not prevent changing A:/b/c to ~/b/c.  I noticed this when
I considered replacing those two checks with

	       ;; If the home dir is just /, don't change it.
               ;; Same thing for A:/ under FreeDOS.
               (let ((home (directory-file-name (match-string 0))))
                  (equal home (file-name-directory home)))

Re: find-file-noselect needs save-match-data

[Stefan Monnier]

> However, I think `find-file-noselect' may as well call `save-match-data'
> for the sake of other places it may be used.

This illustrates what I was saying yesterday: in the end I lose the
argument, and little by little the save-match-data creep in.


        Stefan

Re: find-file-noselect needs save-match-data

[Stefan Monnier]

>     Yes, I proposed to add `save-match-data' to a variety of file handling
>     functions at the outermost level.  The main reason for my proposal was
>     to introduce some sort of consistency among functions in files.el with
>     respect to saving match-data.

> I think that the file-manipulation functions that seem clean and
> abstract ought to call `save-match-data'.

find-file-noselect is a huge function which runs major modes, looks for
file-local variables, looks for other buffers already visiting the file,
checks the buffer if it is still uptodate w.r.t the file, ...

It's nowhere near "clean and abstract".
Anybody who expects such a large function to preserve the match-data will
basically expect *every* function to preserve the match-data.


        Stefan

Re: find-file-noselect needs save-match-data

[Herbert Euler]

>PS: By the way, the above code looks odd.  The comment seems to imply that
>the MS-DOS check just reproduces the preceding / check, but it's not
>the case: the / check makes sure that if HOME is / then /a/b will not be
>changed to ~/a/b, whereas the MS-DOS check will only prevent changing A:/ 
>to
>~ but it will not prevent changing A:/b/c to ~/b/c.  I noticed this when
>I considered replacing those two checks with
>
>	       ;; If the home dir is just /, don't change it.
>                ;; Same thing for A:/ under FreeDOS.
>                (let ((home (directory-file-name (match-string 0))))
>                   (equal home (file-name-directory home)))

But the behavior is correct for another reason.
`abbreviate-file-name' appends a slash at the end of abbreviated home
directory:

      ;; Compute and save the abbreviated homedir name.
      ;; We defer computing this until the first time it's needed, to
      ;; give time for directory-abbrev-alist to be set properly.
      ;; We include a slash at the end, to avoid spurious matches
      ;; such as `/usr/foobar' when the home dir is `/usr/foo'.
      (or abbreviated-home-dir
	  (setq abbreviated-home-dir
		(let ((abbreviated-home-dir "$foo"))
		  (concat "^" (abbreviate-file-name (expand-file-name "~"))
			  "\\(/\\|\\'\\)"))))

So on Windows with root directories e.g. "A:/" as home directory, the
value of `abbreviated-home-dir' is "^A:/\\(/\\|\\'\\)".  So file names
such as "A:/b/c" even can't satisfy the first condition.  Although the
outcome is that "A:/b/c" is not changed to "~", it's not good to get
this outcome in the current way.

Regards,
Guanpeng Xu

_________________________________________________________________
FREE pop-up blocking with the new MSN Toolbar - get it now! 
http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/

Re: find-file-noselect needs save-match-data

[Herbert Euler]

>       Finally, I knew it's in type-break, and then I knew
>     the problem is caused by `find-file-noselect' which is put on
>     `post-command-hook' and changes the match data.
>
>I think it is a bug to expect the match data to be preserved across
>`post-command-hook'.  Which code expects that?  We should fix that
>code to protect itself.

Then currently nothing needs to be fixed.  The code that expects it is
me, when I studied the Emacs source.  And I (with Stefan's and your
help) have fixed this code.

Thank you.

Regards,
Guanpeng Xu

_________________________________________________________________
Don't just search. Find. Check out the new MSN Search! 
http://search.msn.com/

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    The basic global match-data structure is an array.  Maybe we should
    make save-match-data a cheap C operation.

That can't hurt.  Would someone like to write it?

					       Another possibility for an
    (incompatible) API change would be that save-match-data _clears_ the
    match-data.  That way, nested calls of save-match-data become cheap
    since the inner levels don't need to allocate anything.

This particular incompatible change would probably be a disaster,
but maybe a new C-level implementation could optimize the usual case
and get the same speedup.

    All of the proposals of when to call and when not to call
    save-match-data revolve around the problem that save-match-data is an
    expensive operation.

Any slowdown is just part of it.  It is also something you need to
remember to do.  Whether it is fast or slow, we need to set up
guidelines for when to do it.

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    Actually, I'd even argue against any such new function, on the grounds that
    it's solving a non-problem, i.e. it will just add new functions, new code,
    new complexity for no real benefit.

This function is good because it will enable programmers to DTRT in a
simpler way in many cases.

Re: find-file-noselect needs save-match-data

[Richard Stallman]

I'd rather have a new function than add an argument to string-match.
It will be easier to use and easier to remember.

Re: find-file-noselect needs save-match-data

[Stefan Monnier]

>     All of the proposals of when to call and when not to call
>     save-match-data revolve around the problem that save-match-data is an
>     expensive operation.

Not only that: also that it's a pervasive problem that we don't want to
solve by adding save-match-data everywhere.


        Stefan

Re: find-file-noselect needs save-match-data

[Stefan Monnier]

>     Actually, I'd even argue against any such new function, on the grounds
>     that it's solving a non-problem, i.e. it will just add new functions,
>     new code, new complexity for no real benefit.

> This function is good because it will enable programmers to DTRT in a
> simpler way in many cases.

I fail to see the simplicity.  All the difficulty comes from having to think
about when to be careful about the match-data.

After that, there is no real difference whether the problem is solved by
save-match-data or fast-string-match or some extra argument to string-match
or by let-binding inihibit-match-data.


        Stefan

Re: find-file-noselect needs save-match-data

[David Kastrup]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>     Actually, I'd even argue against any such new function, on the grounds
>>     that it's solving a non-problem, i.e. it will just add new functions,
>>     new code, new complexity for no real benefit.
>
>> This function is good because it will enable programmers to DTRT in a
>> simpler way in many cases.
>
> I fail to see the simplicity.  All the difficulty comes from having to think
> about when to be careful about the match-data.
>
> After that, there is no real difference whether the problem is solved by
> save-match-data or fast-string-match or some extra argument to string-match
> or by let-binding inihibit-match-data.

Apropos: it appears to have been lost in the noise: save-match-data in
its current implementation is _not_ a noop around code not touching
match-data, since it converts integer data into markers (which move
their positions under buffer changes) and then back again.  Is there a
compelling rationale for that?  If not: now would be the time to
change this.  We have quite a lot of time to figure out whether this
change causes unexpected problems for someone.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    > I think that the file-manipulation functions that seem clean and
    > abstract ought to call `save-match-data'.

    find-file-noselect is a huge function which runs major modes, looks for
    file-local variables, looks for other buffers already visiting the file,
    checks the buffer if it is still uptodate w.r.t the file, ...

So what?

    It's nowhere near "clean and abstract".

Viewed from outside, it might seem clean and abstract:
a function you can call to do a very clear job with a simple
description (even though its implementation is complex).

Re: find-file-noselect needs save-match-data

[Herbert Euler]

>I fail to see the simplicity.  All the difficulty comes from having to 
>think
>about when to be careful about the match-data.

What we currently have is a model that each call to the matching
functions such as `string-match', `looking-at' etc changes the
match-data.  Furthermore, almost any function calls one of these
functions.  So with this model thinking about preserving match-data is
extravagant, since after each call to almost any function, the
match-data is possibly changed.  This means that you can't expect the
match-data is preserved even if there is only one simple invocation to
some function between the positions where the matching functions are
called and where the match-data is used, unless the invocation is
inside a `save-match-data' form.

If, for example, we add some function such as `fast-string-match', or
add some variable such as `inhibit-match-data', this actually doesn't
help solve the problem.  Since changing match-data is possible by
calling matching functions (although achieving it is a bit difficult
compared with the usage now), match-data is not guaranteed not to be
changed, just as before.

So with the current model, the only way to preserve match-data is to
use the `save-match-data' form everywhere, which is impratical.

But we can't make match-data unchanged, so there seems no solution to
this problem, and just as Stefan said, this is a non-problem.

Regards,
Guanpeng Xu

_________________________________________________________________
Don't just search. Find. Check out the new MSN Search! 
http://search.msn.com/

Re: find-file-noselect needs save-match-data

[martin rudalics]

 > I'd rather have a new function than add an argument to string-match.
 > It will be easier to use and easier to remember.

The argument could be 'match-string instead of t so it would be easier
to see what an actual call does.  And the argument could be for example
'match-beginning so `string-match' would return the begin of the match.
`looking-at', `re-search-forward', etc. could be handled the same way.

Well, I really don't know.  Stefan's arguments didn't convince me
entirely but usually he's right.  Sometimes he fails to understand more
simple-minded programmers' concerns ...

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    > This function is good because it will enable programmers to DTRT in a
    > simpler way in many cases.

    I fail to see the simplicity.  All the difficulty comes from having to think
    about when to be careful about the match-data.

Yes, but if people get in the habit of using a new function
whenever they don't actually want to use the match data,
which I expect is most of the time, that need to think carefully
will be much less frequent.

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    Apropos: it appears to have been lost in the noise: save-match-data in
    its current implementation is _not_ a noop around code not touching
    match-data, since it converts integer data into markers (which move
    their positions under buffer changes) and then back again.  Is there a
    compelling rationale for that?

I am sure it was done to fix a bug, and I can imagine what the bug
looked like.

Re: find-file-noselect needs save-match-data

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

>     Apropos: it appears to have been lost in the noise:
>     save-match-data in its current implementation is _not_ a noop
>     around code not touching match-data, since it converts integer
>     data into markers (which move their positions under buffer
>     changes) and then back again.  Is there a compelling rationale
>     for that?
>
> I am sure it was done to fix a bug, and I can imagine what the bug
> looked like.

Does anybody have actual evidence for that?  I should think that if
there was a particular bug to be handled in this manner, fixing it by
introducing an unexpected and undocumented side-effect into
save-match-data might have been an unfortunate way of tackling the
problem.  After all, this side-effect (letting the match-data track
buffer insertions and deletions by turning it into markers and back
again) is quite expensive in addition to being undocumented.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: find-file-noselect needs save-match-data

[Miles Bader]

Richard Stallman <rms@gnu.org> writes:
> Yes, but if people get in the habit of using a new function
> whenever they don't actually want to use the match data,
> which I expect is most of the time, that need to think carefully
> will be much less frequent.

I think the name `string-match-p' probably aids that end more than the
name I suggested.  A guideline like "use (string-match-p) if you just
need to do a test, and (save-match-data (string-match ...)) if you need
the match data" seems fairly natural.

-miles

-- 
Everywhere is walking distance if you have the time.  -- Steven Wright

Re: find-file-noselect needs save-match-data

[Richard Stallman]

     > I'd rather have a new function than add an argument to string-match.
     > It will be easier to use and easier to remember.

    The argument could be 'match-string instead of t so it would be easier
    to see what an actual call does.  And the argument could be for example
    'match-beginning so `string-match' would return the begin of the match.
    `looking-at', `re-search-forward', etc. could be handled the same way.

This adds too much complexity, I think.

Re: find-file-noselect needs save-match-data

[Herbert Euler]

>     > This function is good because it will enable programmers to DTRT in 
>a
>     > simpler way in many cases.
>
>     I fail to see the simplicity.  All the difficulty comes from having to 
>think
>     about when to be careful about the match-data.
>
>Yes, but if people get in the habit of using a new function
>whenever they don't actually want to use the match data,
>which I expect is most of the time, that need to think carefully
>will be much less frequent.

No, they would think carefully as frequently as now.  I mean, when
they would think carefully now they would think carefully after adding
such a function; when they don't need to think carefully now they
wouldn't think carefully after adding such a function.  Please take a
look at the following case, in `c-electric-brace':

      ;; cc-cmds.el, line 749
      ;; `}': compact to a one-liner defun?
      (save-match-data
	(when
	    (and (eq last-command-char ?\})
		 (memq 'one-liner-defun c-cleanup-list)
		 (c-intersect-lists '(defun-close) syntax)
		 (c-try-one-liner))
	  (setq here (- (point-max) pos))))

`save-match-data' is used here, but nothing implies directly that
match-data will be changed in its body.  Some functions change it, but
is called _indirectly_.  One would need to check the source of each
function in its body to see where the match-data is changed.  This is
the only case that the problems caused by unexpected match-data
changes happen.  If one writes the following code:

      ;; dired.el, line 2207, in function `dired-build-subdir-alist'
      (while (re-search-forward dired-subdir-regexp nil t)
	;; Avoid taking a file name ending in a colon
	;; as a subdir name.
	(unless (save-excursion
		  (goto-char (match-beginning 0))
		  (beginning-of-line)
		  (forward-char 2)
		  (save-match-data (looking-at dired-re-perms)))

one will know that `looking-at' changes match-data, and so uses
`save-match-data' properly to avoid the problem.

And if match-data needs to be preserved, the programmer would pay
attention to it, no matter whether there are some functions that won't
change match-data exist.  When they think about which function to use,
they've already thought about whether the match-data should be
preserved.

Regards,
Guanpeng Xu

_________________________________________________________________
Don't just search. Find. Check out the new MSN Search! 
http://search.msn.com/

Re: find-file-noselect needs save-match-data

[Herbert Euler]

>      ;; cc-cmds.el, line 749
>      ;; `}': compact to a one-liner defun?
>      (save-match-data
>	(when
>	    (and (eq last-command-char ?\})
>		 (memq 'one-liner-defun c-cleanup-list)
>		 (c-intersect-lists '(defun-close) syntax)
>		 (c-try-one-liner))
>	  (setq here (- (point-max) pos))))
>
>`save-match-data' is used here, but nothing implies directly that
>match-data will be changed in its body.  Some functions change it, but
>is called _indirectly_.  One would need to check the source of each
>function in its body to see where the match-data is changed.  This is
>the only case that the problems caused by unexpected match-data
>changes happen.

And adding a new function does not eliminate this case.  The function
itself still needs to be checked, either its source or its docstring.

Regards,
Guanpeng Xu

_________________________________________________________________
FREE pop-up blocking with the new MSN Toolbar - get it now! 
http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    I think the name `string-match-p' probably aids that end more than the
    name I suggested.

Would you like to implement it?

We should also have `looking-at-p'.

Re: find-file-noselect needs save-match-data

[Stefan Monnier]

> functions.  So with this model thinking about preserving match-data is
> extravagant, since after each call to almost any function, the
> match-data is possibly changed.

It's not extravagant, it just forces you to only think about it where it
actually matters (i.e. where you need to call a non-trivial function before
using some match-data) rather than "where it may conceivably matter".

> this problem, and just as Stefan said, this is a non-problem.

Can't argue with this ;-)


        Stefan

Re: find-file-noselect needs save-match-data

[Stefan Monnier]

>> This function is good because it will enable programmers to DTRT in a
>> simpler way in many cases.

>     I fail to see the simplicity.  All the difficulty comes from having to
>     think about when to be careful about the match-data.

> Yes, but if people get in the habit of using a new function
> whenever they don't actually want to use the match data,
> which I expect is most of the time, that need to think carefully
> will be much less frequent.

I think this is a delusion.  Maybe 100 years from now, it won't be the case
any more but in the foreseeable future most functions will still potentially
(i.e. unless documented otherwise) change the match-data and it'll still be
the only safe assumption to make.


        Stefan

Re: find-file-noselect needs save-match-data

[Juri Linkov]

> and in `Info-fontify-node'
>
> 			(cond ((save-match-data (looking-back "\\<see"))
> 			       "")
> 			      ((save-match-data (looking-back "\\<in"))
> 			       "")
> 			      ((memq (char-before) '(nil ?\. ?! ??))
>                                "See ")
> 			      ((save-match-data
> 				 (save-excursion
> 				   (search-forward "\n\n" start t)))
> 			       "See ")
> 			      (t "see "))))
>
> both deeply nested in loops.  Did I mention that Info's fontification
> can be a pain?

Funny, but neither of the proposed new functions (`string-match-p' and
`looking-at-p') will help improving the code in info.el above (which uses
`looking-back' and `search-forward').

It seems that instead of adding a prefix/suffix to names of all functions
that change the match-data, it's better to add a new variable and/or a macro
like `save-match-data' with different internals (that will set a global
variable telling match functions not to set the match data).

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: find-file-noselect needs save-match-data

[Lennart Borgman (gmail)]

Juri Linkov wrote:
> It seems that instead of adding a prefix/suffix to names of all functions
> that change the match-data, it's better to add a new variable and/or a macro
> like `save-match-data' with different internals (that will set a global
> variable telling match functions not to set the match data).

Or, if it is possible a new fast version of save-match-data, probably 
with at least parts written in C.

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    > I am sure it was done to fix a bug, and I can imagine what the bug
    > looked like.

    Does anybody have actual evidence for that?

You can look at the emacs-devel archives for a few weeks before the
relevant date if you want to see what the bug was.

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    No, they would think carefully as frequently as now.  I mean, when
    they would think carefully now they would think carefully after adding
    such a function; when they don't need to think carefully now they
    wouldn't think carefully after adding such a function.  Please take a
    look at the following case, in `c-electric-brace':

You're right about cases like that one; but there are plenty of functions
that only call primitives, plus other functions in the same file which
the same programmer will know about.

Re: find-file-noselect needs save-match-data

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

>     > I am sure it was done to fix a bug, and I can imagine what the bug
>     > looked like.
>
>     Does anybody have actual evidence for that?
>
> You can look at the emacs-devel archives for a few weeks before the
> relevant date if you want to see what the bug was.

What is the relevant date for a function that never has been
different?  It has been introduced in 1992 according to the ChangeLog
files.  There is no record of the behavior ever being changed in the
ChangeLog files, and it certainly has not been changed in that respect
as long as Emacs has been in CVS according to the logs.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    > Yes, but if people get in the habit of using a new function
    > whenever they don't actually want to use the match data,
    > which I expect is most of the time, that need to think carefully
    > will be much less frequent.

    I think this is a delusion.

We will have to agree to disagree.

Would someone please implement string-match-p and looking-at-p?
(A variable would be more general, but not as simple to use.
Its only advantage over save-match-data would be a small speedup.)

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    What is the relevant date for a function that never has been
    different?

I thought you said it had been changed, and complained that the change
could cause a bug.  I took your word for it.

If `save-match-data' was never changed, then there was no point at which
a change could have broken anything, thus no issue.

Re: find-file-noselect needs save-match-data

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

>     What is the relevant date for a function that never has been
>     different?
>
> I thought you said it had been changed, and complained that the
> change could cause a bug.  I took your word for it.

Oh please.  You took your mistaken recollection of my word for it.  I
certainly _never_ claimed that it had been changed.  I claimed that
the current implementation is arguably wrong in addition to being
unnecesssarily inefficient.

> If `save-match-data' was never changed, then there was no point at
> which a change could have broken anything, thus no issue.

I already gave an explicit Elisp example where save-match-data around
a passage not accessing the match data leads to different results from
not using save-match-data.

So save-match-data is _not_ a noop in the absence of match-string and
similar, and I asked what the rationale was for that (since it
actually is _expensive_ to use markers here).

I still am waiting for any rationale better than "there must be some
reason".

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    > If `save-match-data' was never changed, then there was no point at
    > which a change could have broken anything, thus no issue.

    I already gave an explicit Elisp example where save-match-data around
    a passage not accessing the match data leads to different results from
    not using save-match-data.

Using markers helps avoid strange results when the code inside the
construct edits the buffer.  I think that is a good reason for this
feature.

Re: find-file-noselect needs save-match-data

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

>     > If `save-match-data' was never changed, then there was no point at
>     > which a change could have broken anything, thus no issue.
>
>     I already gave an explicit Elisp example where save-match-data around
>     a passage not accessing the match data leads to different results from
>     not using save-match-data.
>
> Using markers helps avoid strange results when the code inside the
> construct edits the buffer.  I think that is a good reason for this
> feature.

It is neither documented in the Elisp manual nor in the DOC string and
is not the functionality the function is intended to provide.  It is
also inconsistent with save_search_regs in search.c which means that
the documentation in

(info "(elisp) Filter functions")

       In earlier Emacs versions, every filter function that did regular
    expression searching or matching had to explicitly save and restore the
    match data.  Now Emacs does this automatically for filter functions;
    they never need to do it explicitly.  *Note Match Data::.

is factually incorrect since the effect is different from
save-match-data.

Also, in
(info "(elisp) Saving Match Data")
we have

       Emacs automatically saves and restores the match data when it runs
    process filter functions (*note Filter Functions::) and process
    sentinels (*note Sentinels::).

Again, no mention of the different behavior of save-match-data and the
filter functions and sentinels.

And in
(info "(elisp) Sentinels")


       In earlier Emacs versions, every sentinel that did regular expression
    searching or matching had to explicitly save and restore the match data.
    Now Emacs does this automatically for sentinels; they never need to do
    it explicitly.  *Note Match Data::.

Again, no mention that save-match-data would use markers and track
insertions while sentinels don't.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: find-file-noselect needs save-match-data

[Herbert Euler]

>     > Yes, but if people get in the habit of using a new function
>     > whenever they don't actually want to use the match data,
>     > which I expect is most of the time, that need to think carefully
>     > will be much less frequent.
>
>     I think this is a delusion.
>
>We will have to agree to disagree.
>
>Would someone please implement string-match-p and looking-at-p?
>(A variable would be more general, but not as simple to use.
>Its only advantage over save-match-data would be a small speedup.)

Is there somebody working on it?  If not, I would like to start it.

Regards,
Guanpeng Xu

_________________________________________________________________
Don't just search. Find. Check out the new MSN Search! 
http://search.msn.click-url.com/go/onm00200636ave/direct/01/

Re: find-file-noselect needs save-match-data

[Juri Linkov]

>>     > Yes, but if people get in the habit of using a new function
>>     > whenever they don't actually want to use the match data,
>>     > which I expect is most of the time, that need to think carefully
>>     > will be much less frequent.
>>
>>     I think this is a delusion.
>>
>>We will have to agree to disagree.
>>
>>Would someone please implement string-match-p and looking-at-p?
>>(A variable would be more general, but not as simple to use.
>>Its only advantage over save-match-data would be a small speedup.)
>
> Is there somebody working on it?  If not, I would like to start it.

Do you want to work on a new variable?  Since adding `-p' to only two
match functions doesn't make sense.  And adding `-p' to all functions
makes even less sense.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: find-file-noselect needs save-match-data

[Herbert Euler]

> >>     > Yes, but if people get in the habit of using a new function
> >>     > whenever they don't actually want to use the match data,
> >>     > which I expect is most of the time, that need to think carefully
> >>     > will be much less frequent.
> >>
> >>     I think this is a delusion.
> >>
> >>We will have to agree to disagree.
> >>
> >>Would someone please implement string-match-p and looking-at-p?
> >>(A variable would be more general, but not as simple to use.
> >>Its only advantage over save-match-data would be a small speedup.)
> >
> > Is there somebody working on it?  If not, I would like to start it.
>
>Do you want to work on a new variable?  Since adding `-p' to only two
>match functions doesn't make sense.  And adding `-p' to all functions
>makes even less sense.

I want if this is the final decision.  But what is it?  I prefer adding a
new variable too, but Richard seems to prefer `-p' over it.

Regards,
Guanpeng Xu

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    >Would someone please implement string-match-p and looking-at-p?
    >(A variable would be more general, but not as simple to use.
    >Its only advantage over save-match-data would be a small speedup.)

    Is there somebody working on it?  If not, I would like to start it.

Please do it.

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    >>Would someone please implement string-match-p and looking-at-p?
    >>(A variable would be more general, but not as simple to use.
    >>Its only advantage over save-match-data would be a small speedup.)
    >
    > Is there somebody working on it?  If not, I would like to start it.

    Do you want to work on a new variable?  Since adding `-p' to only two
    match functions doesn't make sense.  And adding `-p' to all functions
    makes even less sense.

For the reasons explained in my words quoted above, I've
decided I want the two new functions and NOT the variable.

Re: find-file-noselect needs save-match-data

[Herbert Euler]

>     >>Would someone please implement string-match-p and looking-at-p?
>     >>(A variable would be more general, but not as simple to use.
>     >>Its only advantage over save-match-data would be a small speedup.)
>     >
>     > Is there somebody working on it?  If not, I would like to start it.
>
>     Do you want to work on a new variable?  Since adding `-p' to only two
>     match functions doesn't make sense.  And adding `-p' to all functions
>     makes even less sense.
>
>For the reasons explained in my words quoted above, I've
>decided I want the two new functions and NOT the variable.

Last confirming before working: can we implement a new variable at the
C level and implement the two new functions (possibly as macros) at
the Lisp level?  This is an eclectic proposal and takes advantages
from both sides.

Regards,
Guanpeng Xu

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

Re: find-file-noselect needs save-match-data

[David Kastrup]

"Herbert Euler" <herberteuler@hotmail.com> writes:

>>     >>Would someone please implement string-match-p and looking-at-p?
>>     >>(A variable would be more general, but not as simple to use.
>>     >>Its only advantage over save-match-data would be a small speedup.)
>>     >
>>     > Is there somebody working on it?  If not, I would like to start it.
>>
>>     Do you want to work on a new variable?  Since adding `-p' to only two
>>     match functions doesn't make sense.  And adding `-p' to all functions
>>     makes even less sense.
>>
>>For the reasons explained in my words quoted above, I've
>>decided I want the two new functions and NOT the variable.
>
> Last confirming before working: can we implement a new variable at the
> C level and implement the two new functions (possibly as macros) at
> the Lisp level?  This is an eclectic proposal and takes advantages
> from both sides.

Just as a note aside: regular expressions like \(.\)\1 are possible,
so the match data _has_ to be stored somewhere, presumably.  This
might impact the attractiveness of the "C variable" implementation.

-- 
David Kastrup

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    Last confirming before working: can we implement a new variable at the
    C level and implement the two new functions (possibly as macros) at
    the Lisp level?  This is an eclectic proposal and takes advantages
    from both sides.

That sounds ok.  The new functions could be written as defsubsts.

Re: find-file-noselect needs save-match-data

[Richard Stallman]

    > Using markers helps avoid strange results when the code inside the
    > construct edits the buffer.  I think that is a good reason for this
    > feature.

    It is neither documented in the Elisp manual nor in the DOC string

That is true.  Details like this don't always have to be
documented, but maybe this one should be.

								       and
    is not the functionality the function is intended to provide.

Isn't it?

      It is
    also inconsistent with save_search_regs in search.c which means that
    the documentation in

You are right.  save_search_regs was written in a way that was
inconsistent with the already-existing save-match-data.

Is this a real problem?
Do we need to change save_search_regs?
I hope not, because it would cause a slowdown.
But no so much of one, since it only does anything
if there is a real search inside the sentinel or filter.

One annoying thing is that the use of save-match-data by the sentinel
or filter won't prevent save_search_regs from also doing its thing.
If save-match-data were implemented in C, it could do so.

Re: find-file-noselect needs save-match-data

[Herbert Euler]

>Just as a note aside: regular expressions like \(.\)\1 are possible,
>so the match data _has_ to be stored somewhere, presumably.  This
>might impact the attractiveness of the "C variable" implementation.

Fortunately we can solve this with providing another re_registers
variable.  I've almost finished it, but I got segmentation fault after
adding a DEFVAR_LISP in syms_of_search.  The segfault is triggered
by GC.  Any hint on how to solve it?  Thanks.

I'll post my change later.

Regards,
Guanpeng Xu

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

Re: find-file-noselect needs save-match-data

[Herbert Euler]

>>Just as a note aside: regular expressions like \(.\)\1 are possible,
>>so the match data _has_ to be stored somewhere, presumably.  This
>>might impact the attractiveness of the "C variable" implementation.
>
>Fortunately we can solve this with providing another re_registers
>variable.  I've almost finished it, but I got segmentation fault after
>adding a DEFVAR_LISP in syms_of_search.  The segfault is triggered
>by GC.  Any hint on how to solve it?  Thanks.

The crash goes away after I re-checked out the source from CVS.  Below
is the patches and the change-log entries I did for the two functions.
All patches are based on files in unicode 2 branch.  Please check
them.  If they are OK, I'll write document for them.  Thanks.

Regards,
Guanpeng Xu




Index: lisp/subr.el
===================================================================
RCS file: /sources/emacs/emacs/lisp/subr.el,v
retrieving revision 1.360.2.68
diff -c -F '^[_a-zA-Z0-9$]+ *(' -r1.360.2.68 subr.el
*** lisp/subr.el	16 Jun 2007 22:31:38 -0000	1.360.2.68
--- lisp/subr.el	29 Jun 2007 14:26:21 -0000
*************** t (mix it with ordinary output), or a fi
*** 2667,2672 ****
--- 2667,2684 ----
  	    (looking-at (concat "\\(?:"  regexp "\\)\\'")))))
      (not (null pos))))

+ (defsubst looking-at-p (regexp)
+   "\
+ Same as `looking-at' except this function does not change the match data."
+   (let ((inhibit-changing-match-data t))
+     (looking-at regexp)))
+
+ (defsubst string-match-p (regexp string &optional start)
+   "\
+ Same as `string-match' except this function does not change the match 
data."
+   (let ((inhibit-changing-match-data t))
+     (string-match regexp string start)))
+
  (defun subregexp-context-p (regexp pos &optional start)
    "Return non-nil if POS is in a normal subregexp context in REGEXP.
  A subregexp context is one where a sub-regexp can appear.
Index: src/search.c
===================================================================
RCS file: /sources/emacs/emacs/src/search.c,v
retrieving revision 1.174.2.38
diff -c -F '^[_a-zA-Z0-9$]+ *(' -r1.174.2.38 search.c
*** src/search.c	11 Jun 2007 00:57:33 -0000	1.174.2.38
--- src/search.c	29 Jun 2007 14:26:27 -0000
***************
*** 60,73 ****
  struct regexp_cache *searchbuf_head;


! /* Every call to re_match, etc., must pass &search_regs as the regs
!    argument unless you can show it is unnecessary (i.e., if re_match
!    is certainly going to be called again before region-around-match
!    can be called).

     Since the registers are now dynamically allocated, we need to make
     sure not to refer to the Nth register before checking that it has
!    been allocated by checking search_regs.num_regs.

     The regex code keeps track of whether it has allocated the search
     buffer using bits in the re_pattern_buffer.  This means that whenever
--- 60,74 ----
  struct regexp_cache *searchbuf_head;


! /* Every call to re_match, etc., must pass &search_regs_nochange or
!    &search_regs as the regs argument unless you can show it is
!    unnecessary (i.e., if re_match is certainly going to be called
!    again before region-around-match can be called).

     Since the registers are now dynamically allocated, we need to make
     sure not to refer to the Nth register before checking that it has
!    been allocated by checking search_regs_nochange.num_regs, or
!    search_regs.num_regs.

     The regex code keeps track of whether it has allocated the search
     buffer using bits in the re_pattern_buffer.  This means that whenever
***************
*** 76,82 ****
     time you call a searching or matching function.  Therefore, we need
     to call re_set_registers after compiling a new pattern or after
     setting the match registers, so that the regex functions will be
!    able to free or re-allocate it properly.  */
  static struct re_registers search_regs;

  /* The buffer in which the last search was performed, or
--- 77,90 ----
     time you call a searching or matching function.  Therefore, we need
     to call re_set_registers after compiling a new pattern or after
     setting the match registers, so that the regex functions will be
!    able to free or re-allocate it properly.
!
!    We now have two families of searching or matching functions.
!    Functions in one family won't change the match data at Lisp level,
!    while functions in the other family will change it.  The variable
!    search_regs_nochange is used for the former family, and the
!    variable search_regs is used for the latter one.  */
! static struct re_registers search_regs_nochange;
  static struct re_registers search_regs;

  /* The buffer in which the last search was performed, or
***************
*** 93,98 ****
--- 101,115 ----

  Lisp_Object Vsearch_spaces_regexp;

+ /* If non-nil, the match data will not be changed during call to
+    searching or matching functions.  This variable is for internal use
+    only.  */
+ Lisp_Object Vinhibit_changing_match_data;
+
+ #define SEARCH_REGS_PTR (NILP (Vinhibit_changing_match_data)	\
+ 			 ? &search_regs				\
+ 			 : &search_regs_nochange)
+
  static void set_search_regs ();
  static void save_search_regs ();
  static int simple_search ();
*************** looking_at_1 (string, posix)
*** 280,285 ****
--- 297,303 ----
    int s1, s2;
    register int i;
    struct re_pattern_buffer *bufp;
+   struct re_registers *regs = SEARCH_REGS_PTR;

    if (running_asynch_code)
      save_search_regs ();
*************** looking_at_1 (string, posix)
*** 289,295 ****
      = current_buffer->case_eqv_table;

    CHECK_STRING (string);
!   bufp = compile_pattern (string, &search_regs,
  			  (!NILP (current_buffer->case_fold_search)
  			   ? current_buffer->case_canon_table : Qnil),
  			  posix,
--- 307,313 ----
      = current_buffer->case_eqv_table;

    CHECK_STRING (string);
!   bufp = compile_pattern (string, regs,
  			  (!NILP (current_buffer->case_fold_search)
  			   ? current_buffer->case_canon_table : Qnil),
  			  posix,
*************** looking_at_1 (string, posix)
*** 320,326 ****
    re_match_object = Qnil;

    i = re_match_2 (bufp, (char *) p1, s1, (char *) p2, s2,
! 		  PT_BYTE - BEGV_BYTE, &search_regs,
  		  ZV_BYTE - BEGV_BYTE);
    immediate_quit = 0;

--- 338,344 ----
    re_match_object = Qnil;

    i = re_match_2 (bufp, (char *) p1, s1, (char *) p2, s2,
! 		  PT_BYTE - BEGV_BYTE, regs,
  		  ZV_BYTE - BEGV_BYTE);
    immediate_quit = 0;

*************** looking_at_1 (string, posix)
*** 329,341 ****

    val = (0 <= i ? Qt : Qnil);
    if (i >= 0)
!     for (i = 0; i < search_regs.num_regs; i++)
!       if (search_regs.start[i] >= 0)
  	{
! 	  search_regs.start[i]
! 	    = BYTE_TO_CHAR (search_regs.start[i] + BEGV_BYTE);
! 	  search_regs.end[i]
! 	    = BYTE_TO_CHAR (search_regs.end[i] + BEGV_BYTE);
  	}
    XSETBUFFER (last_thing_searched, current_buffer);
    return val;
--- 347,357 ----

    val = (0 <= i ? Qt : Qnil);
    if (i >= 0)
!     for (i = 0; i < regs->num_regs; i++)
!       if (regs->start[i] >= 0)
  	{
! 	  regs->start[i] = BYTE_TO_CHAR (regs->start[i] + BEGV_BYTE);
! 	  regs->end[i] = BYTE_TO_CHAR (regs->end[i] + BEGV_BYTE);
  	}
    XSETBUFFER (last_thing_searched, current_buffer);
    return val;
*************** string_match_1 (regexp, string, start, p
*** 373,378 ****
--- 389,395 ----
    struct re_pattern_buffer *bufp;
    int pos, pos_byte;
    int i;
+   struct re_registers *regs = SEARCH_REGS_PTR;

    if (running_asynch_code)
      save_search_regs ();
*************** string_match_1 (regexp, string, start, p
*** 399,405 ****
    XCHAR_TABLE (current_buffer->case_canon_table)->extras[2]
      = current_buffer->case_eqv_table;

!   bufp = compile_pattern (regexp, &search_regs,
  			  (!NILP (current_buffer->case_fold_search)
  			   ? current_buffer->case_canon_table : Qnil),
  			  posix,
--- 416,422 ----
    XCHAR_TABLE (current_buffer->case_canon_table)->extras[2]
      = current_buffer->case_eqv_table;

!   bufp = compile_pattern (regexp, regs,
  			  (!NILP (current_buffer->case_fold_search)
  			   ? current_buffer->case_canon_table : Qnil),
  			  posix,
*************** string_match_1 (regexp, string, start, p
*** 410,429 ****
    val = re_search (bufp, (char *) SDATA (string),
  		   SBYTES (string), pos_byte,
  		   SBYTES (string) - pos_byte,
! 		   &search_regs);
    immediate_quit = 0;
    last_thing_searched = Qt;
    if (val == -2)
      matcher_overflow ();
    if (val < 0) return Qnil;

!   for (i = 0; i < search_regs.num_regs; i++)
!     if (search_regs.start[i] >= 0)
        {
! 	search_regs.start[i]
! 	  = string_byte_to_char (string, search_regs.start[i]);
! 	search_regs.end[i]
! 	  = string_byte_to_char (string, search_regs.end[i]);
        }

    return make_number (string_byte_to_char (string, val));
--- 427,444 ----
    val = re_search (bufp, (char *) SDATA (string),
  		   SBYTES (string), pos_byte,
  		   SBYTES (string) - pos_byte,
! 		   regs);
    immediate_quit = 0;
    last_thing_searched = Qt;
    if (val == -2)
      matcher_overflow ();
    if (val < 0) return Qnil;

!   for (i = 0; i < regs->num_regs; i++)
!     if (regs->start[i] >= 0)
        {
! 	regs->start[i] = string_byte_to_char (string, regs->start[i]);
! 	regs->end[i] = string_byte_to_char (string, regs->end[i]);
        }

    return make_number (string_byte_to_char (string, val));
*************** search_buffer (string, pos, pos_byte, li
*** 1059,1064 ****
--- 1074,1080 ----
    int len = SCHARS (string);
    int len_byte = SBYTES (string);
    register int i;
+   struct re_registers *regs = SEARCH_REGS_PTR;

    if (running_asynch_code)
      save_search_regs ();
*************** search_buffer (string, pos, pos_byte, li
*** 1077,1083 ****
        int s1, s2;
        struct re_pattern_buffer *bufp;

!       bufp = compile_pattern (string, &search_regs, trt, posix,
  			      !NILP (current_buffer->enable_multibyte_characters));

        immediate_quit = 1;	/* Quit immediately if user types ^G,
--- 1093,1099 ----
        int s1, s2;
        struct re_pattern_buffer *bufp;

!       bufp = compile_pattern (string, regs, trt, posix,
  			      !NILP (current_buffer->enable_multibyte_characters));

        immediate_quit = 1;	/* Quit immediately if user types ^G,
*************** search_buffer (string, pos, pos_byte, li
*** 1110,1116 ****
  	  int val;
  	  val = re_search_2 (bufp, (char *) p1, s1, (char *) p2, s2,
  			     pos_byte - BEGV_BYTE, lim_byte - pos_byte,
! 			     &search_regs,
  			     /* Don't allow match past current point */
  			     pos_byte - BEGV_BYTE);
  	  if (val == -2)
--- 1126,1132 ----
  	  int val;
  	  val = re_search_2 (bufp, (char *) p1, s1, (char *) p2, s2,
  			     pos_byte - BEGV_BYTE, lim_byte - pos_byte,
! 			     regs,
  			     /* Don't allow match past current point */
  			     pos_byte - BEGV_BYTE);
  	  if (val == -2)
*************** search_buffer (string, pos, pos_byte, li
*** 1119,1136 ****
  	    }
  	  if (val >= 0)
  	    {
! 	      pos_byte = search_regs.start[0] + BEGV_BYTE;
! 	      for (i = 0; i < search_regs.num_regs; i++)
! 		if (search_regs.start[i] >= 0)
  		  {
! 		    search_regs.start[i]
! 		      = BYTE_TO_CHAR (search_regs.start[i] + BEGV_BYTE);
! 		    search_regs.end[i]
! 		      = BYTE_TO_CHAR (search_regs.end[i] + BEGV_BYTE);
  		  }
  	      XSETBUFFER (last_thing_searched, current_buffer);
  	      /* Set pos to the new position. */
! 	      pos = search_regs.start[0];
  	    }
  	  else
  	    {
--- 1135,1150 ----
  	    }
  	  if (val >= 0)
  	    {
! 	      pos_byte = regs->start[0] + BEGV_BYTE;
! 	      for (i = 0; i < regs->num_regs; i++)
! 		if (regs->start[i] >= 0)
  		  {
! 		    regs->start[i] = BYTE_TO_CHAR (regs->start[i] + BEGV_BYTE);
! 		    regs->end[i] = BYTE_TO_CHAR (regs->end[i] + BEGV_BYTE);
  		  }
  	      XSETBUFFER (last_thing_searched, current_buffer);
  	      /* Set pos to the new position. */
! 	      pos = regs->start[0];
  	    }
  	  else
  	    {
*************** search_buffer (string, pos, pos_byte, li
*** 1144,1150 ****
  	  int val;
  	  val = re_search_2 (bufp, (char *) p1, s1, (char *) p2, s2,
  			     pos_byte - BEGV_BYTE, lim_byte - pos_byte,
! 			     &search_regs,
  			     lim_byte - BEGV_BYTE);
  	  if (val == -2)
  	    {
--- 1158,1164 ----
  	  int val;
  	  val = re_search_2 (bufp, (char *) p1, s1, (char *) p2, s2,
  			     pos_byte - BEGV_BYTE, lim_byte - pos_byte,
! 			     regs,
  			     lim_byte - BEGV_BYTE);
  	  if (val == -2)
  	    {
*************** search_buffer (string, pos, pos_byte, li
*** 1152,1168 ****
  	    }
  	  if (val >= 0)
  	    {
! 	      pos_byte = search_regs.end[0] + BEGV_BYTE;
! 	      for (i = 0; i < search_regs.num_regs; i++)
! 		if (search_regs.start[i] >= 0)
  		  {
! 		    search_regs.start[i]
! 		      = BYTE_TO_CHAR (search_regs.start[i] + BEGV_BYTE);
! 		    search_regs.end[i]
! 		      = BYTE_TO_CHAR (search_regs.end[i] + BEGV_BYTE);
  		  }
  	      XSETBUFFER (last_thing_searched, current_buffer);
! 	      pos = search_regs.end[0];
  	    }
  	  else
  	    {
--- 1166,1180 ----
  	    }
  	  if (val >= 0)
  	    {
! 	      pos_byte = regs->end[0] + BEGV_BYTE;
! 	      for (i = 0; i < regs->num_regs; i++)
! 		if (regs->start[i] >= 0)
  		  {
! 		    regs->start[i] = BYTE_TO_CHAR (regs->start[i] + BEGV_BYTE);
! 		    regs->end[i] = BYTE_TO_CHAR (regs->end[i] + BEGV_BYTE);
  		  }
  	      XSETBUFFER (last_thing_searched, current_buffer);
! 	      pos = regs->end[0];
  	    }
  	  else
  	    {
*************** boyer_moore (n, base_pat, len, len_byte,
*** 1616,1621 ****
--- 1628,1634 ----
    register int i, j;
    unsigned char *pat, *pat_end;
    int multibyte = ! NILP (current_buffer->enable_multibyte_characters);
+   struct re_registers *regs = SEARCH_REGS_PTR;

    unsigned char simple_translate[0400];
    /* These are set to the preceding bytes of a byte to be translated
*************** boyer_moore (n, base_pat, len, len_byte,
*** 1919,1925 ****
  		    cursor += dirlen; /* to resume search */
  		  else
  		    return ((direction > 0)
! 			    ? search_regs.end[0] : search_regs.start[0]);
  		}
  	      else
  		cursor += stride_for_teases; /* <sigh> we lose -  */
--- 1932,1938 ----
  		    cursor += dirlen; /* to resume search */
  		  else
  		    return ((direction > 0)
! 			    ? regs->end[0] : regs->start[0]);
  		}
  	      else
  		cursor += stride_for_teases; /* <sigh> we lose -  */
*************** boyer_moore (n, base_pat, len, len_byte,
*** 1995,2001 ****
  		    pos_byte += dirlen; /* to resume search */
  		  else
  		    return ((direction > 0)
! 			    ? search_regs.end[0] : search_regs.start[0]);
  		}
  	      else
  		pos_byte += stride_for_teases;
--- 2008,2014 ----
  		    pos_byte += dirlen; /* to resume search */
  		  else
  		    return ((direction > 0)
! 			    ? regs->end[0] : regs->start[0]);
  		}
  	      else
  		pos_byte += stride_for_teases;
*************** set_search_regs (beg_byte, nbytes)
*** 2017,2041 ****
       int beg_byte, nbytes;
  {
    int i;

    /* Make sure we have registers in which to store
       the match position.  */
!   if (search_regs.num_regs == 0)
      {
!       search_regs.start = (regoff_t *) xmalloc (2 * sizeof (regoff_t));
!       search_regs.end = (regoff_t *) xmalloc (2 * sizeof (regoff_t));
!       search_regs.num_regs = 2;
      }

    /* Clear out the other registers.  */
!   for (i = 1; i < search_regs.num_regs; i++)
      {
!       search_regs.start[i] = -1;
!       search_regs.end[i] = -1;
      }

!   search_regs.start[0] = BYTE_TO_CHAR (beg_byte);
!   search_regs.end[0] = BYTE_TO_CHAR (beg_byte + nbytes);
    XSETBUFFER (last_thing_searched, current_buffer);
  }

--- 2030,2055 ----
       int beg_byte, nbytes;
  {
    int i;
+   struct re_registers *regs = SEARCH_REGS_PTR;

    /* Make sure we have registers in which to store
       the match position.  */
!   if (regs->num_regs == 0)
      {
!       regs->start = (regoff_t *) xmalloc (2 * sizeof (regoff_t));
!       regs->end = (regoff_t *) xmalloc (2 * sizeof (regoff_t));
!       regs->num_regs = 2;
      }

    /* Clear out the other registers.  */
!   for (i = 1; i < regs->num_regs; i++)
      {
!       regs->start[i] = -1;
!       regs->end[i] = -1;
      }

!   regs->start[0] = BYTE_TO_CHAR (beg_byte);
!   regs->end[0] = BYTE_TO_CHAR (beg_byte + nbytes);
    XSETBUFFER (last_thing_searched, current_buffer);
  }

*************** DEFUN ("set-match-data", Fset_match_data
*** 2998,3004 ****
  }

  /* If non-zero the match data have been saved in saved_search_regs
!    during the execution of a sentinel or filter. */
  static int search_regs_saved;
  static struct re_registers saved_search_regs;
  static Lisp_Object saved_last_thing_searched;
--- 3012,3022 ----
  }

  /* If non-zero the match data have been saved in saved_search_regs
!    during the execution of a sentinel or filter.  The value of
!    saved_search_regs is copied from and recovered to search_regs
!    rather than search_regs_nochange, since the value of
!    search_regs_nochange is intended to not be used and to be
!    discarded.  */
  static int search_regs_saved;
  static struct re_registers saved_search_regs;
  static Lisp_Object saved_last_thing_searched;
*************** syms_of_search ()
*** 3145,3150 ****
--- 3163,3175 ----
  A value of nil (which is the normal value) means treat spaces literally.  
*/);
    Vsearch_spaces_regexp = Qnil;

+   DEFVAR_LISP ("inhibit-changing-match-data", 
&Vinhibit_changing_match_data,
+       doc: /* Internal use only.
+ If non-nil, the match data will not be changed during call to searching or
+ matching functions, such as `looking-at', `string-match', 
`re-search-forward'
+ etc.  */);
+   Vinhibit_changing_match_data = Qnil;
+
    defsubr (&Slooking_at);
    defsubr (&Sposix_looking_at);
    defsubr (&Sstring_match);




2007-06-30  Guanpeng Xu  <herberteuler@hotmail.com>

	* subr.el (looking-at-p, string-match-p): New functions.

2007-06-30  Guanpeng Xu  <herberteuler@hotmail.com>

	* search.c (search_regs_nochange, Vinhibit_changing_match_data):
	New variables.
	(SEARCH_REGS_PTR): New macro.
	(looking_at_1): Don't chang match data at Lisp level if
	Vinhibit_changing_match_data is not nil.
	(string_match_1, search_buffer, boyer_moore, set_search_regs):
	Likewise.
	(syms_of_search): Add entry of Vinhibit_changing_match_data and
	set it to nil.

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

byte-compiling an elisp-file fails in Emacs 22.1 (reproducable)

[klaus.berndl]

Hi all,

1. using Emacs 22.1 for MS Windows.

2. starting Emacs with
     runemacs --quick
   so no private setup-stuff is loaded - just a plain and fresh Emacs

3. Opening my .emacs with C-x C-f

4. calling "Byte-compile This File" from the menu "Emacs-lisp"

   Get the error "Symbol's function definition is void: compilation-forget-errors"


When activating "Enter debugger on error" vefore step 4, i get the following backtrace:

Debugger entered--Lisp error: (void-function compilation-forget-errors)
  compilation-forget-errors()
  byte-compile-log-file()
  byte-compile-from-buffer(#<buffer  *Compiler Input*> "c:/home/.emacs")
  byte-compile-file("c:/home/.emacs")
  emacs-lisp-byte-compile()
  call-interactively(emacs-lisp-byte-compile)

Even explicitly loading compile.el (which defines this function) before byte-compiling
doesn't fix it....

Is this a known bug? 

Anyway it's an anoying bug which makes the new Emacs 22 quite unuseable for me
because i do a lot of elisp-programming and byte-compiling....

Is there a known work-around for that problem?

Thanks in advance!

Klaus

Re: byte-compiling an elisp-file fails in Emacs 22.1 (reproducable)

[Jason Rumney]

klaus.berndl@sdm.de wrote:
> Hi all,
>
> 1. using Emacs 22.1 for MS Windows.
>
> 2. starting Emacs with
>      runemacs --quick
>    so no private setup-stuff is loaded - just a plain and fresh Emacs
>
> 3. Opening my .emacs with C-x C-f
>
> 4. calling "Byte-compile This File" from the menu "Emacs-lisp"
>
>    Get the error "Symbol's function definition is void: compilation-forget-errors"
>
>   
I can't reproduce it. What does M-x list-load-path-shadows say?

Re: byte-compiling an elisp-file fails in Emacs 22.1 (reproducable)

[Richard Stallman]

    Debugger entered--Lisp error: (void-function compilation-forget-errors)
      compilation-forget-errors()
      byte-compile-log-file()
      byte-compile-from-buffer(#<buffer  *Compiler Input*> "c:/home/.emacs")
      byte-compile-file("c:/home/.emacs")
      emacs-lisp-byte-compile()
      call-interactively(emacs-lisp-byte-compile)

    Even explicitly loading compile.el (which defines this function) before byte-compiling
    doesn't fix it....

That is VERY strange.  Can you check (fboundp 'compilation-forget-errors)?

AW: byte-compiling an elisp-file fails in Emacs 22.1 (reproducable)

[klaus.berndl]

[-- Attachment #1.1: Type: text/plain, Size: 1106 bytes --]

Sorry guys, I must apologise for my own stupidity...

I had an old pre-version (some years old) of the new compile.el-implementation
(without compilation-forget-errors) in my site-lisp folder which "shadowed"
the shipped one....

So, sorry for the noise - all is working fine, it was my own fault.

Ciao,
Klaus


-----Ursprüngliche Nachricht-----
Von: Richard Stallman [mailto:rms@gnu.org]
Gesendet: Sa 07.07.2007 15:06
An: Berndl, Klaus
Cc: emacs-devel@gnu.org
Betreff: Re: byte-compiling an elisp-file fails in Emacs 22.1 (reproducable)
 
    Debugger entered--Lisp error: (void-function compilation-forget-errors)
      compilation-forget-errors()
      byte-compile-log-file()
      byte-compile-from-buffer(#<buffer  *Compiler Input*> "c:/home/.emacs")
      byte-compile-file("c:/home/.emacs")
      emacs-lisp-byte-compile()
      call-interactively(emacs-lisp-byte-compile)

    Even explicitly loading compile.el (which defines this function) before byte-compiling
    doesn't fix it....

That is VERY strange.  Can you check (fboundp 'compilation-forget-errors)?


[-- Attachment #1.2: Type: text/html, Size: 1917 bytes --]

[-- Attachment #2: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel