bug#8686: 24.0.50; `re-search-backward' does not respect `search-invisible'

bug#8686: 24.0.50; `re-search-backward' does not respect `search-invisible'

[Drew Adams]

My impression is that `re-search-backward' and `*-forward' do not
respect `search-invisible'.  This makes it difficult if not impossible
to use them in code to not find matches in hidden text.  Why would this
variable be respected only in the Isearch Lisp code?  What am I missing?
 
My use case at the moment is to make Imenu (conditionally) not count
definitions that are commented out, by temporarily hiding commented
code.  I bind `search-invisible' to nil, but that has no effect on
`re-search-backward', which is called by the Imenu code
(e.g. `imenu--generic-function').  The code finds commented out
definitions, in spite of `search-invisible'.
 

In GNU Emacs 24.0.50.1 (i386-mingw-nt5.1.2600)
 of 2011-05-16 on 3249CTO
Windowing system distributor `Microsoft Corp.', version 5.1.2600
configured using `configure --with-gcc (4.5) --no-opt --cflags
-Ic:/build/include'
 

bug#8686: 24.0.50; `re-search-backward' does not respect `search-invisible'

[Stefan Monnier]

> My impression is that `re-search-backward' and `*-forward' do not
> respect `search-invisible'.

These are low-level C functions, whereas search-invisible is implemented
on top in Elisp and is a Customization variable rather than a variable
for use by Elisp code.


        Stefan

bug#8686: 24.0.50; `re-search-backward' does not respect `search-invisible'

[Drew Adams]

> > My impression is that `re-search-backward' and `*-forward' do not
> > respect `search-invisible'.
> 
> These are low-level C functions, whereas search-invisible is 
> implemented on top in Elisp and is a Customization variable
> rather than a variable for use by Elisp code.

Well, yes.  In a sense that is precisely what this bug report is about.  There
should be a simple way in Lisp to direct the search functions to ignore or see
hidden text.  It should be enough to bind some variable for the relevant
context/duration.

And BTW, there is already Lisp code that binds `search-invisible' - in
isearch.el itself!  It is just as kosher for other Lisp code to act similarly.
If lazy highlighting can bind it for its own purposes, so can other code that
uses or adapts search functions.

Also BTW, the variable is named `search-invisible', not `isearch-invisible'
(unlike, e.g., `isearch-open-invisible').  Presumably it is not only for
incremental search, or even interactive search.

Question: What is the simplest way, from Lisp, to get the "low-level C" search
functions to optionally respect/ignore hidden text?  Should Lisp programmers be
required to reproduce the entire invisibility-respecting scaffolding created in
isearch.el (or equivalent)?  How about having a single, standard infrastructure
provided by Emacs?

bug#8686: 24.0.50; `re-search-backward' does not respect `search-invisible'

[Stefan Monnier]

> Question: What is the simplest way, from Lisp, to get the "low-level
> C" search functions to optionally respect/ignore hidden text?

Something like

  (while (and (re-search-forward blabla)
              (invisible-p (point)))
    ...)

The same idea can be used with syntax-ppss to skip over matches that are
inside comments/strings.


        Stefan

bug#8686: 24.0.50; `re-search-backward' does not respect `search-invisible'

[Lennart Borgman]

On Wed, May 18, 2011 at 2:51 AM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>> Question: What is the simplest way, from Lisp, to get the "low-level
>> C" search functions to optionally respect/ignore hidden text?
>
> Something like
>
>  (while (and (re-search-forward blabla)
>              (invisible-p (point)))
>    ...)
>
> The same idea can be used with syntax-ppss to skip over matches that are
> inside comments/strings.

Maybe implement this as minor modes for isearch, occur etc?

bug#8686: 24.0.50; `re-search-backward' does not respect `search-invisible'

[Drew Adams]

> > Question: What is the simplest way, from Lisp, to get the "low-level
> > C" search functions to optionally respect/ignore hidden text?
> 
> Something like
> 
>   (while (and (re-search-forward blabla)
>               (invisible-p (point)))
>     ...)
> 
> The same idea can be used with syntax-ppss to skip over 
> matches that are inside comments/strings.

Yes, but why not have this built into the search functions themselves, just as
is done for isearch?  That was the point/suggestion.

bug#8686: 24.0.50; `re-search-backward' does not respect `search-invisible'

[Lennart Borgman]

On Wed, May 18, 2011 at 4:18 AM, Drew Adams <drew.adams@oracle.com> wrote:
>> > Question: What is the simplest way, from Lisp, to get the "low-level
>> > C" search functions to optionally respect/ignore hidden text?
>>
>> Something like
>>
>>   (while (and (re-search-forward blabla)
>>               (invisible-p (point)))
>>     ...)
>>
>> The same idea can be used with syntax-ppss to skip over
>> matches that are inside comments/strings.
>
> Yes, but why not have this built into the search functions themselves, just as
> is done for isearch?  That was the point/suggestion.

Which also was the point of my suggestions...

bug#8686: 24.0.50; `re-search-backward' does not respect `search-invisible'

[Stefan Monnier]

>> > Question: What is the simplest way, from Lisp, to get the "low-level
>> > C" search functions to optionally respect/ignore hidden text?
>> 
>> Something like
>> 
>> (while (and (re-search-forward blabla)
>> (invisible-p (point)))
>> ...)
>> 
>> The same idea can be used with syntax-ppss to skip over 
>> matches that are inside comments/strings.

> Yes, but why not have this built into the search functions themselves,
> just as is done for isearch?

Because I'd rather that the low-level functions do one thing and one
thing well.


        Stefan

bug#8686: 24.0.50; `re-search-backward' does not respect `search-invisible'

[Lennart Borgman]

On Wed, May 18, 2011 at 2:13 PM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>>> > Question: What is the simplest way, from Lisp, to get the "low-level
>>> > C" search functions to optionally respect/ignore hidden text?
>>>
>>> Something like
>>>
>>> (while (and (re-search-forward blabla)
>>> (invisible-p (point)))
>>> ...)
>>>
>>> The same idea can be used with syntax-ppss to skip over
>>> matches that are inside comments/strings.
>
>> Yes, but why not have this built into the search functions themselves,
>> just as is done for isearch?
>
> Because I'd rather that the low-level functions do one thing and one
> thing well.

(Of course my suggestion with minor modes was not for the low level functions.)

bug#8686: 24.0.50; `re-search-backward' does not respect `search-invisible'

[Drew Adams]

> >> The same idea can be used with syntax-ppss to skip over 
> >> matches that are inside comments/strings.
> 
> > Yes, but why not have this built into the search functions 
> > themselves, just as is done for isearch?
> 
> Because I'd rather that the low-level functions do one thing and one
> thing well.

Sure.  Define `re-search-forward' & compagnie in Lisp.
Have them call C functions that each do one simple thing well.

That way, existing code that uses these long-standing functions can benefit from
this feature (or can ignore it) just by binding a variable.

bug#8686: 24.0.50; `re-search-backward' does not respect `search-invisible'

[Lars Ingebrigtsen]

"Drew Adams" <drew.adams@oracle.com> writes:

> My impression is that `re-search-backward' and `*-forward' do not
> respect `search-invisible'.  This makes it difficult if not impossible
> to use them in code to not find matches in hidden text.  Why would this
> variable be respected only in the Isearch Lisp code?  What am I missing?

Well, `search-invisible' is documented as only applying to interactive
searches:

----

search-invisible is a variable defined in ‘isearch.el’.
Its value is ‘open’

Documentation:
If t incremental search/query-replace can match hidden text.
A nil value means don’t match invisible text.

----

If you want code not to match invisible text, you have to make your code
check for the visibility.  Closing.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#8686: 24.0.50; `re-search-backward' does not respect `search-invisible'

[Drew Adams]

> > My impression is that `re-search-backward' and `*-forward' do not
> > respect `search-invisible'.  This makes it difficult if not impossible
> > to use them in code to not find matches in hidden text.  Why would this
> > variable be respected only in the Isearch Lisp code?  What am I missing?
> 
> Well, `search-invisible' is documented as only applying to interactive
> searches:
> 
> ----
> 
> search-invisible is a variable defined in 'isearch.el'.
> Its value is 'open'
> 
> Documentation:
> If t incremental search/query-replace can match hidden text.
> A nil value means don't match invisible text.
> 
> ----
> 
> If you want code not to match invisible text, you have to make your code
> check for the visibility.  Closing.

I don't think this is correct.  Did you check whether it
truly works only for interactive use?  It is a general
test, used in multiple places now (including `perform-replace').