etags.el tags-search use global syntax table

etags.el tags-search use global syntax table

[isaac.to]

Hi,

When using AucTeX to edit some LaTeX files, I want to search for some
word globally.  My version is not new enough to support inter-file
searching directly, so I've used tags-search.  I'm searching for
something like "\bx\b".  It turned out that some of the matches are
missed, because $ is considered as a word (as in the fundamental
mode).  This is out of my expectation, since it is a LaTeX file and in
the LaTeX mode of AucTeX, $ is considered not a word.  Perhaps when
Emacs performs a tags-search it do it in a temp buffer which does not
have the correct mode loaded?  I consider this a bug in etags.

Regards,
Isaac

Re: etags.el tags-search use global syntax table

[Francesco Potorti`]

isaac.to@gmail.com:
>When using AucTeX to edit some LaTeX files, I want to search for some
>word globally.  My version is not new enough to support inter-file
>searching directly, so I've used tags-search.  I'm searching for
>something like "\bx\b".  It turned out that some of the matches are
>missed, because $ is considered as a word (as in the fundamental
>mode).  This is out of my expectation, since it is a LaTeX file and in
>the LaTeX mode of AucTeX, $ is considered not a word.  Perhaps when
>Emacs performs a tags-search it do it in a temp buffer which does not
>have the correct mode loaded?  I consider this a bug in etags.

This is indeed a bug in etags.el that was present in Emacs 21 and is
still there in Emacs 22.  It has nothing to do with Auctex, but pops up
whenever:
- you start a tags-search
- the file where you should see a match is not currently visited
- the search in Fundamental mode gives different results from the search
  in the proper mode of the buffer

The reason is that the next-file function in etags.el loads non-visited
files in a temporary buffer with insert-file-contents, rather than using
find-file, so the mode remains Fundamental.  I am not sure why it is so.
These are the relevant lines at the end of next-file:

      ;; Like find-file, but avoids random warning messages.
      (set-buffer (get-buffer-create " *next-file*"))
      (kill-all-local-variables)
      (erase-buffer)
      (setq new next)
      (insert-file-contents new nil))

Generally speaking, tags-search looks for all files in the TAGS file: if
they are visited, they are searched and when no match is found point is
restored.  If they are not visited, they are loaded into a temporary
buffer which is overwritten when no matches are found.  However, search
in this buffer is performed in Fundamental mode.

The cure is to use the syntax table relative to the natural mode of the
file when doing the search.  Is it really necessary to avoid using
find-file?  If yes, how can one guess the right syntax table and apply
it to the temporary buffer?

Re: etags.el tags-search use global syntax table

[Richard Stallman]

    The reason is that the next-file function in etags.el loads non-visited
    files in a temporary buffer with insert-file-contents, rather than using
    find-file, so the mode remains Fundamental.  I am not sure why it is so.

There are two reasons for this.  One is that some major modes can be
rather intrusive in what they do.  The other is that visiting a file
is much slower.

Of course, it was all based on the idea that it wouldn't really matter
for searching, and if it actually finds a match, then it visits the
file properly.  This bug shows that the major mode does matter in some
cases for searching.

I am not sure whether it is better to fix this bug or not.
If it is a big slowdown, we are better off not fixing it.

Re: etags.el tags-search use global syntax table

[John Dennis]

[-- Attachment #1: Type: text/plain, Size: 2645 bytes --]

>> The reason is that the next-file function in etags.el loads
>> non-visited files in a temporary buffer with insert-file-contents,
>> rather than using find-file, so the mode remains Fundamental.  I am
>> not sure why it is so.

> There are two reasons for this.  One is that some major modes can be 
> rather intrusive in what they do.  The other is that visiting a file 
> is much slower.

> Of course, it was all based on the idea that it wouldn't really
> matter for searching, and if it actually finds a match, then it
> visits the file properly.  This bug shows that the major mode does
> matter in some cases for searching.

The the major mode does matter for searching, in fact it's often 
critical for correct searching. There are two key things a major mode 
does which affects searching, it establishes a syntax and often defines 
the value of case-fold-search. If either of these is not done before the 
search proceeds you'll often get incorrect search results. This is 
especially critical with program source code, which almost by definition 
is the context when a tags search is performed.

I consider this to be a serious flaw in etags and have been fixing it in 
my local version of emacs for quite a while now. The novisit behavior 
has another down side to my thinking as well, the buffer is not 
available in the buffer list unless a match was found. This makes for an 
odd and arbitrary set of buffers. My preference is to have all buffers 
which participated in the search to be available. Yes, this does mean 
more memory is used and loading is slower. But virtual memory is a 
wonderful thing, universally supported, and I can't see on any modern 
system the performance difference. Plus, if one does repeated searches 
isn't it more efficient to have the buffer already loaded rather than 
repeatedly loading it and then throwing it away?

I've also been bothered by the interface to tags-search, unlike the 
other tags functions it does not default to the word surrounded by 
point. It's really nice to be able to put your point on what you want to 
search for and just hit a function key.

Attached is a patch that I believe fixes these issues and allows for 
customizing the behavior. I don't see an easy fix for establishing the 
syntax table and value of case-fold-search for the novisit case. I guess 
if a user want to continue to utilize novisit that's a liability you'll 
have to live with in order to retain that feature.

Also, as an aside, should we still be supporting the slow baud rate 
stuff? It adds complexity to the code which I suspect is no longer relevant.

-- 
John Dennis <jdennis@redhat.com>

[-- Attachment #2: etags.patch --]
[-- Type: text/x-patch, Size: 1526 bytes --]

--- etags.el.orig	2007-11-30 13:46:35.000000000 -0500
+++ etags.el	2007-11-30 15:12:28.000000000 -0500
@@ -102,6 +102,11 @@
   :group 'etags
   :type 'boolean)
 
+(defcustom tags-novisit t
+  "*Non-nil means use a temporary buffer to save time and avoid uninteresting warnings."
+  :type 'boolean
+  :group 'etags)
+
 (defvar tags-table-computed-list nil
   "List of tags tables to search, computed from `tags-table-list'.
 This includes tables implicitly included by other tables.  The list is not
@@ -1698,7 +1703,8 @@
 	 (with-current-buffer buffer
 	   (revert-buffer t)))
     (if (not (and new novisit))
-	(set-buffer (find-file-noselect next novisit))
+	(progn (set-buffer (find-file-noselect next t))
+	       (setq new nil))
       ;; Like find-file, but avoids random warning messages.
       (set-buffer (get-buffer-create " *next-file*"))
       (kill-all-local-variables)
@@ -1761,7 +1767,7 @@
 	      (goto-char original-point))
 
 	    (setq file-finished nil)
-	    (setq new (next-file first-time t))
+	    (setq new (next-file first-time tags-novisit))
 
 	    ;; If NEW is non-nil, we got a temp buffer,
 	    ;; and NEW is the file name.
@@ -1803,7 +1809,7 @@
 To continue searching for next match, use command \\[tags-loop-continue].
 
 See documentation of variable `tags-file-name'."
-  (interactive "sTags search (regexp): ")
+  (interactive (find-tag-interactive "Tags search (regexp): "))
   (if (and (equal regexp "")
 	   (eq (car tags-loop-scan) 're-search-forward)
 	   (null tags-loop-operate))

Re: etags.el tags-search use global syntax table

[Richard Stallman]

    The the major mode does matter for searching, in fact it's often 
    critical for correct searching. There are two key things a major mode 
    does which affects searching, it establishes a syntax and often defines 
    the value of case-fold-search.

I am concerned that making etags invoke all major modes
might lead to a substantial slowdown (as it did in the past).
We could try it and see.  If it does, the best thing to do
would be to indicate somehow which modes really need to be selected
during tags search.

    The novisit behavior 
    has another down side to my thinking as well, the buffer is not 
    available in the buffer list unless a match was found. This makes for an 
    odd and arbitrary set of buffers. My preference is to have all buffers 
    which participated in the search to be available.

I strongly disagree.