Strange behavior of word isearch

Strange behavior of word isearch

[Chong Yidong]

Hi Juri,

The isearch word commands that you checked in a couple of months ago
seem pretty buggy.  The search never succeeds.  For example, if I try to
search for the words "learn basic" in the Emacs welcome screen:

emacs -q       Enters the welcome screen

M-s w          Starts isearch-forward-word

l              Emacs beeps and the prompt says "Failing word I-search:"

earn           The prompt still says "Failing word I-search:"
               even though the word "learn" is now highlighted as an
               "other match".

SPC            The prompt still says "Failing word I-search:"

basic          The prompt still says "Failing word I-search:"


Could you try to fix this?

Re: Strange behavior of word isearch

[Chong Yidong]

Chong Yidong <cyd@stupidchicken.com> writes:

> The isearch word commands that you checked in a couple of months ago
> seem pretty buggy.  The search never succeeds.  For example, if I try to
> search for the words "learn basic" in the Emacs welcome screen:
>
> emacs -q       Enters the welcome screen
>
> M-s w          Starts isearch-forward-word
>
> l              Emacs beeps and the prompt says "Failing word I-search:"
>
> earn           The prompt still says "Failing word I-search:"
>                even though the word "learn" is now highlighted as an
>                "other match".
>
> SPC            The prompt still says "Failing word I-search:"
>
> basic          The prompt still says "Failing word I-search:"

The trouble here is that word-search-forward appends a \b regexp
condition to the end of the search string, so it does not treat "learn"
as a match for "l".

Is this behavior useful in practice?  If not, the following patch
removes the \b condition, which is an incompatible change to the
behavior of word-search-forward.

In other words, previously a word search for "foo" is a regexp search
for "\bfoo\b"; this patch changes the search to "\bfoo".

Or, it may be safer to either introduce a function that is similar to
word-search-forward but without the \b condition, or a variable that
turns that behavior off; and use that in isearch.

What do people think?


*** trunk/src/search.c.~1.234.~       2008-08-27    10:30:40.000000000 -0400
--- trunk/src/search.c        2008-09-24 23:35:07.000000000    -0400
***************
*** 2154,2160 ****
    if (!word_count)
      return empty_unibyte_string;
  
!   adjust = - punct_count + 5 * (word_count - 1) + 4;
    if (STRING_MULTIBYTE (string))
      val = make_uninit_multibyte_string (len + adjust,
                                              SBYTES (string)
--- 2154,2160 ----
    if (!word_count)
      return empty_unibyte_string;
  
!   adjust = - punct_count + 5 * (word_count - 1) + 2;
    if (STRING_MULTIBYTE (string))
      val = make_uninit_multibyte_string (len + adjust,
                                              SBYTES (string)
***************
*** 2192,2200 ****
        prev_c = c;
      }
  
-   *o++ = '\\';
-   *o++ = 'b';
- 
    return val;
  }
  
--- 2192,2197 ----

Re: Strange behavior of word isearch

[martin rudalics]

 > In other words, previously a word search for "foo" is a regexp search
 > for "\bfoo\b"; this patch changes the search to "\bfoo".

Couldn't this break modes like `ada-mode' or `flyspell-mode'?

martin

Re: Strange behavior of word isearch

[Chong Yidong]

martin rudalics <rudalics@gmx.at> writes:

>> In other words, previously a word search for "foo" is a regexp search
>> for "\bfoo\b"; this patch changes the search to "\bfoo".
>
> Couldn't this break modes like `ada-mode' or `flyspell-mode'?

Yeah, that might cause some unpredictable breakage.

So probably we need to introduce a separate function that performs an
"open-ended" version of word-search-forward.  Any suggestions for the
function name?

Re: Strange behavior of word isearch

[martin rudalics]

 > So probably we need to introduce a separate function that performs an
 > "open-ended" version of word-search-forward.  Any suggestions for the
 > function name?

We could use an optional "open-ended" argument.

martin

Re: Strange behavior of word isearch

[Chong Yidong]

martin rudalics <rudalics@gmx.at> writes:

>> So probably we need to introduce a separate function that performs an
>> "open-ended" version of word-search-forward.  Any suggestions for the
>> function name?
>
> We could use an optional "open-ended" argument.

Adding a new argument may not be so good.  Currently, the code has been
careful to keep the arguments of the different search functions
identical, and the isearch code makes use of this.

Re: Strange behavior of word isearch

[Chong Yidong]

I checked in the modified functions as word-search-forward-lax and
word-search-backward-lax, and made isearch use them.  If someone has a
better idea, let me know.

Re: Strange behavior of word isearch

[Juri Linkov]

> I checked in the modified functions as word-search-forward-lax
> and word-search-backward-lax, and made isearch use them.
> If someone has a better idea, let me know.

Currently word isearch is open-ended, so it is not possible to restrict
the search to the right word boundary, such as matching only singular
word forms and not plurals ("default" vs "defaults" etc.)  And typing
additional whitespace at the end is both inconvenient and not self-apparent
especially when there is only one word and not a list of words separated
by whitespace.  However, there exists a good solution.

In http://thread.gmane.org/gmane.emacs.devel/76174/focus=101795
Stefan writes:

    E.g. we could provide a key that says "here, I'm done entering the
    pattern".  That key could be C-s/C-r or RET, or something else.
    Entering more text should probably revert back to "the pattern is not
    done yet".

I now think this is the best way to implement the word search.
The following small patch does this.  It uses the lax versions of
word search functions recently implemented by Yidong, only when the user
incrementally adds or removes characters in the search string,
thus allowing the user to enter a complete word string without
failing in the middle.

In all other cases like using C-s/C-r to navigate between matches, it uses
the default strict versions of word search functions that allows to find
all exact word matches.  And lazy highlighting works correctly as well.
I believe this will provide the most useful behavior for the word search:

Index: lisp/isearch.el
===================================================================
RCS file: /sources/emacs/emacs/lisp/isearch.el,v
retrieving revision 1.331
diff -c -r1.331 isearch.el
*** lisp/isearch.el	19 Oct 2008 22:33:17 -0000	1.331
--- lisp/isearch.el	8 Nov 2008 22:54:14 -0000
***************
*** 2178,2184 ****
        (funcall isearch-search-fun-function)
      (cond
       (isearch-word
!       (if isearch-forward 'word-search-forward-lax 'word-search-backward-lax))
       (isearch-regexp
        (if isearch-forward 're-search-forward 're-search-backward))
       (t
--- 2182,2193 ----
        (funcall isearch-search-fun-function)
      (cond
       (isearch-word
!       ;; Use lax versions to not fail at the end of the word while the user
!       ;; adds and removes characters in the search string
!       (if (not (eq (length isearch-string)
! 		   (length (isearch-string-state (car isearch-cmds)))))
! 	  (if isearch-forward 'word-search-forward-lax 'word-search-backward-lax)
! 	(if isearch-forward 'word-search-forward 'word-search-backward)))
       (isearch-regexp
        (if isearch-forward 're-search-forward 're-search-backward))
       (t

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Strange behavior of word isearch

[Stefan Monnier]

> word forms and not plurals ("default" vs "defaults" etc.)  And typing
> additional whitespace at the end is both inconvenient and not self-apparent
> especially when there is only one word and not a list of words separated
> by whitespace.  However, there exists a good solution.

I disagree with SPC being inconvenient, but I agree that it's not
self-evident.

> Index: lisp/isearch.el
> ===================================================================
> RCS file: /sources/emacs/emacs/lisp/isearch.el,v
> retrieving revision 1.331
> diff -c -r1.331 isearch.el
> *** lisp/isearch.el	19 Oct 2008 22:33:17 -0000	1.331
> --- lisp/isearch.el	8 Nov 2008 22:54:14 -0000
> ***************
> *** 2178,2184 ****
>         (funcall isearch-search-fun-function)
>       (cond
>        (isearch-word
> !       (if isearch-forward 'word-search-forward-lax 'word-search-backward-lax))
>        (isearch-regexp
>         (if isearch-forward 're-search-forward 're-search-backward))
>        (t
> --- 2182,2193 ----
>         (funcall isearch-search-fun-function)
>       (cond
>        (isearch-word
> !       ;; Use lax versions to not fail at the end of the word while the user
> !       ;; adds and removes characters in the search string
> !       (if (not (eq (length isearch-string)
> ! 		   (length (isearch-string-state (car isearch-cmds)))))
> ! 	  (if isearch-forward 'word-search-forward-lax 'word-search-backward-lax)
> ! 	(if isearch-forward 'word-search-forward 'word-search-backward)))
>        (isearch-regexp
>         (if isearch-forward 're-search-forward 're-search-backward))
>        (t

Looks good.  Can people try it out and see if they are surprised?

I know a case where it won't work as nicely as just SPC for me:
I often use C-w to fill the search, but I often don't start at the right
position, so I type in the beginning of the word I want, then maybe hit
C-s a few times to jump to the right occurrence (when it's easier than
to keep on typing), then C-w.  But with your change, the C-s will skip
the right occurrence.


        Stefan

Re: Strange behavior of word isearch

[Juri Linkov]

> Looks good.  Can people try it out and see if they are surprised?
>
> I know a case where it won't work as nicely as just SPC for me:
> I often use C-w to fill the search, but I often don't start at the right
> position, so I type in the beginning of the word I want, then maybe hit
> C-s a few times to jump to the right occurrence (when it's easier than
> to keep on typing), then C-w.  But with your change, the C-s will skip
> the right occurrence.

For such a case it's more easy to start isearch with C-s, compose a
complete search string with moving point to the right occurrence using
C-w and C-s, and finally toggle the word search mode with `M-s w'.

Many applications have a search dialog box with a "Match Whole Word Only"
checkbox.  `M-s w' would be like selecting such an option to narrow the
search space to word boundaries.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Strange behavior of word isearch

[Stefan Monnier]

> The trouble here is that word-search-forward appends a \b regexp
> condition to the end of the search string, so it does not treat "learn"
> as a match for "l".

> Is this behavior useful in practice?  If not, the following patch
> removes the \b condition, which is an incompatible change to the
> behavior of word-search-forward.

> In other words, previously a word search for "foo" is a regexp search
> for "\bfoo\b"; this patch changes the search to "\bfoo".

> Or, it may be safer to either introduce a function that is similar to
> word-search-forward but without the \b condition, or a variable that
> turns that behavior off; and use that in isearch.

> What do people think?

I obviously agree, since I already pointed out this solution.
The important part is that "foo " should still be turned into "\bfoo\b".


        Stefan

Re: Strange behavior of word isearch

[Juri Linkov]

> Hi Juri,
>
> The isearch word commands that you checked in a couple of months ago
> seem pretty buggy.  The search never succeeds.  For example, if I try to
> search for the words "learn basic" in the Emacs welcome screen:
> [...]
> Could you try to fix this?

Sorry for the delay.  The development of this feature was ended
at the end of July due to the feature freeze.  But if the current
state is unacceptable for the next release, I could finish it now.
However, this requires making some major changes discussed in
http://thread.gmane.org/gmane.emacs.devel/76174/focus=101810

-- 
Juri Linkov
http://www.jurta.org/emacs/