search-backward returning matches which end after point

search-backward returning matches which end after point

[Spencer Baugh]

search-backward (and re-search-backward, and looking-back) don't return
matches which end after point.  Is there a version which does?

For example,
(with-temp-buffer
  (insert "foo")
  (save-excursion (insert "bar"))
  (search-backward "foobar"))
 
fails to find the match.

Is there a function which will find this match?

Specifically, I noticed this when trying to use
easy-mmode-define-navigation to define navigation commands for a major
mode based on a regex.

The -prev commands defined by easy-mmode-define-navigation use
re-search-backward.  So if the point is inside the thing which the regex
matches, then thing-prev command will not go to the start of the thing.
But if point is after the thing, then thing-prev will go to the start of
the thing.  This is because of this behavior of search-backward.

I'd like to instead define movement commands which will always go to the
start of the thing when point is inside the thing.  But to do that for
an arbitrary regex/string, I need a version of search-backward which
will return matches which end after point.

Re: search-backward returning matches which end after point

[Stefan Monnier via Users list for the GNU Emacs text editor]

> search-backward (and re-search-backward, and looking-back) don't return
> matches which end after point.  Is there a version which does?

No 🙁
You could write a `while + looking-at` loop to do that, but if you want
something faster, you'll need to change the C code.
I don't expect it would be a difficult change, tho (it should not require
any significant change in the regexp engine, for example).

> The -prev commands defined by easy-mmode-define-navigation use
> re-search-backward.  So if the point is inside the thing which the regex
> matches, then thing-prev command will not go to the start of the thing.

Of course, the same problem appears for search-forward when searching
from within the thing we're searching.  In both cases the usual
workaround is to start your search elsewhere (i.e. take "small" step
back before searching forward or vice-versa).


        Stefan

Re: search-backward returning matches which end after point

[Spencer Baugh]

Stefan Monnier via Users list for the GNU Emacs text editor
<help-gnu-emacs@gnu.org> writes:

>> search-backward (and re-search-backward, and looking-back) don't return
>> matches which end after point.  Is there a version which does?
>
> No 🙁
> You could write a `while + looking-at` loop to do that, but if you want
> something faster, you'll need to change the C code.
> I don't expect it would be a difficult change, tho (it should not require
> any significant change in the regexp engine, for example).

I'm happy to do that, if this is an addition which is likely to be
accepted into Emacs.

Should the API be a new flag in
search-backward/re-search-backward/looking-back?  Or should it maybe be
a new special variable?

>> The -prev commands defined by easy-mmode-define-navigation use
>> re-search-backward.  So if the point is inside the thing which the regex
>> matches, then thing-prev command will not go to the start of the thing.
>
> Of course, the same problem appears for search-forward when searching
> from within the thing we're searching.  In both cases the usual
> workaround is to start your search elsewhere (i.e. take "small" step
> back before searching forward or vice-versa).

Yes, that seems to be a common workaround.  Particularly common seems to
be going to pos-bol first, since the search is for something which can't
contain a newline.

In my case this is tricky though because the thing I'm searching for
*can* contain a newline, so it's not totally trivial to determin what
small step is correct to take...

Re: search-backward returning matches which end after point

[Stefan Monnier via Users list for the GNU Emacs text editor]

>> You could write a `while + looking-at` loop to do that, but if you want
>> something faster, you'll need to change the C code.
>> I don't expect it would be a difficult change, tho (it should not require
>> any significant change in the regexp engine, for example).
> I'm happy to do that, if this is an addition which is likely to be
> accepted into Emacs.

It's hard for me to judge.  OT1H it should be rather unintrusive, but
OTOH it's unsatisfying because doing the same in the other direction
(i.e. when searching forward) is much harder.

> Should the API be a new flag in
> search-backward/re-search-backward/looking-back?  Or should it maybe
> be a new special variable?

Or new functions?  It's a toss-up (I'm rather against special variables).

> In my case this is tricky though because the thing I'm searching for
> *can* contain a newline, so it's not totally trivial to determin what
> small step is correct to take...

A common other approach is to do a `search-forward` after the
`search-backward` to detect when `search-backward` went "too far".


        Stefan

Re: search-backward returning matches which end after point

[Spencer Baugh via Users list for the GNU Emacs text editor]

Stefan Monnier via Users list for the GNU Emacs text editor
<help-gnu-emacs@gnu.org> writes:

>>> You could write a `while + looking-at` loop to do that, but if you want
>>> something faster, you'll need to change the C code.
>>> I don't expect it would be a difficult change, tho (it should not require
>>> any significant change in the regexp engine, for example).
>> I'm happy to do that, if this is an addition which is likely to be
>> accepted into Emacs.
>
> It's hard for me to judge.  OT1H it should be rather unintrusive, but
> OTOH it's unsatisfying because doing the same in the other direction
> (i.e. when searching forward) is much harder.
>
>> Should the API be a new flag in
>> search-backward/re-search-backward/looking-back?  Or should it maybe
>> be a new special variable?
>
> Or new functions?  It's a toss-up (I'm rather against special variables).
>
>> In my case this is tricky though because the thing I'm searching for
>> *can* contain a newline, so it's not totally trivial to determin what
>> small step is correct to take...
>
> A common other approach is to do a `search-forward` after the
> `search-backward` to detect when `search-backward` went "too far".

Oh, clever, that would indeed work for me.

Could we update easy-mmode-define-navigation to use this technique, so
it reliably goes to the start of the current "thing"?  Again happy to
make that patch.  That would more directly satisfy my use case anyway,
which is just "getting reliable next/prev navigation commands".

Re: search-backward returning matches which end after point

[Stefan Monnier via Users list for the GNU Emacs text editor]

> Could we update easy-mmode-define-navigation to use this technique, so
> it reliably goes to the start of the current "thing"?  Again happy to
> make that patch.  That would more directly satisfy my use case anyway,
> which is just "getting reliable next/prev navigation commands".

I wouldn't oppose such a patch.
But note that sometimes you might prefer the current behavior, and also
some code (mostly `diff-mode`, IIRC) might use some of the generated
functions from ELisp where we may rely on the current behavior.
IOW, just a heads up that it could require adjustments elsewhere.


        Stefan

Re: search-backward returning matches which end after point

[Andreas Röhler]

When already inside, you could use the thingatpt utility,

for example

(defun mystring()
   (interactive)
   (message "%s" (word-at-point)))

Am 23.08.24 um 16:18 schrieb Spencer Baugh:
> search-backward (and re-search-backward, and looking-back) don't return
> matches which end after point.  Is there a version which does?
>
> For example,
> (with-temp-buffer
>    (insert "foo")
>    (save-excursion (insert "bar"))
>    (search-backward "foobar"))
>   
> fails to find the match.
>
> Is there a function which will find this match?
>
> Specifically, I noticed this when trying to use
> easy-mmode-define-navigation to define navigation commands for a major
> mode based on a regex.
>
> The -prev commands defined by easy-mmode-define-navigation use
> re-search-backward.  So if the point is inside the thing which the regex
> matches, then thing-prev command will not go to the start of the thing.
> But if point is after the thing, then thing-prev will go to the start of
> the thing.  This is because of this behavior of search-backward.
>
> I'd like to instead define movement commands which will always go to the
> start of the thing when point is inside the thing.  But to do that for
> an arbitrary regex/string, I need a version of search-backward which
> will return matches which end after point.
>

Re: search-backward returning matches which end after point

[Stefan Monnier via Users list for the GNU Emacs text editor]

> When already inside, you could use the thingatpt utility,

thingatpt is not magic.  It has to use the same underlying search
functions, with the same problem Spencer is facing.
IOW it's thingatpt which could use Spencer's solution rather than the reverse.


        Stefan