search-invisible and friends

search-invisible and friends

[Lars Ingebrigtsen]

I have a mode where I plop in base64 image data, but hide it all under a
'display property to display the image itself.

It's annoying that `C-s' lands in the middle of the base64 image data,
so I wondered whether there was any way to control that, and search
invisible seemed like the nearest thing.

But it only works for text that's, well, invisible, and not text that's
hidden by a display property.

Simple test:

(insert (propertize "foo" 'display "bar"))
(setq search-invisible nil)

C-r foo will then land in that string.

Does Emacs have this functionality somewhere?  If not, I think it
should, and perhaps (or perhaps not?) by piggy-backing on the
search-invisible variable.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: search-invisible and friends

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Wed, 09 Sep 2020 14:39:37 +0200
> 
> Does Emacs have this functionality somewhere?

I don't think so.

> If not, I think it should, and perhaps (or perhaps not?) by
> piggy-backing on the search-invisible variable.

How about extending search-invisible to support additional properties,
by providing some variable that would hold the list of such
properties?  Then we could also avoid searching in text that is
visible, if some Lisp needs that (for example, you could have a form
where you'd like to search only the writable fields).

Re: search-invisible and friends

[Stefan Monnier]

> Does Emacs have this functionality somewhere?  If not, I think it
> should, and perhaps (or perhaps not?) by piggy-backing on the
> search-invisible variable.

It'd be very natural to extend the current behavior to also ignore text
that's hidden via `display` rather than via `invisible`, but please be
careful not to consider all `display` properties as hiding the text,
since some do and some don't (not sure which is the official definition
of which do and which don't.  Maybe `display_prop_intangible_p` does
the trick?).


        Stefan

Re: search-invisible and friends

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Wed, 09 Sep 2020 11:10:33 -0400
> Cc: emacs-devel@gnu.org
> 
> please be careful not to consider all `display` properties as hiding
> the text, since some do and some don't

Right.

> (not sure which is the official definition of which do and which
> don't.

They are called "replacing display specs" in the ELisp manual, and I
believe it does describe which are and which aren't.  Type

  i replacing display specs RET

in the ELisp manual, and read there.

Re: search-invisible and friends

[Stefan Monnier]

> They are called "replacing display specs" in the ELisp manual, and I
> believe it does describe which are and which aren't.  Type
>
>   i replacing display specs RET
>
> in the ELisp manual, and read there.

Right, but we wouldn't to re-implement the test in isearch.el, so the
question is to find an existing function which gives us the answer.


        Stefan

Re: search-invisible and friends

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: larsi@gnus.org,  emacs-devel@gnu.org
> Date: Wed, 09 Sep 2020 12:11:44 -0400
> 
> Right, but we wouldn't to re-implement the test in isearch.el, so the
> question is to find an existing function which gives us the answer.

We'd need to write it, I think.

Re: search-invisible and friends

[Juri Linkov]

> Simple test:
>
> (insert (propertize "foo" 'display "bar"))
> (setq search-invisible nil)
>
> C-r foo will then land in that string.

Should then 'C-r bar' land in that string?
(the text 'bar' is visible on the screen)

Re: search-invisible and friends

[Lars Ingebrigtsen]

Juri Linkov <juri@linkov.net> writes:

>> Simple test:
>>
>> (insert (propertize "foo" 'display "bar"))
>> (setq search-invisible nil)
>>
>> C-r foo will then land in that string.
>
> Should then 'C-r bar' land in that string?
> (the text 'bar' is visible on the screen)

The use case here is images covering text -- I just used text as a
display property here for convenience.

I have no opinion on whether you should be able to isearch textual
display properties; it's a pretty orthogonal issue.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: search-invisible and friends

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> Right, but we wouldn't to re-implement the test in isearch.el, so the
>> question is to find an existing function which gives us the answer.
>
> We'd need to write it, I think.

So basically mash up this stuff into a convenient DEFUN?

	  && !NILP (val = get_char_property_and_overlay
		              (make_fixnum (PT), Qdisplay, selected_window,
			       &overlay))
	  && display_prop_intangible_p (val, overlay, PT, PT_BYTE)

Like, er, `has-replacing-text-property-p'?

I also read the "Display Specs That Replace The Text" node in the
manual, but I'm not quite sure what the relevance is here -- it talks
about how (display) text properties are collated (i.e., merged over eq
sequences of text), but does isearch have to deal with that?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: search-invisible and friends

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> If not, I think it should, and perhaps (or perhaps not?) by
>> piggy-backing on the search-invisible variable.
>
> How about extending search-invisible to support additional properties,
> by providing some variable that would hold the list of such
> properties?  Then we could also avoid searching in text that is
> visible, if some Lisp needs that (for example, you could have a form
> where you'd like to search only the writable fields).

Ah, yeah, that sounds quite useful and natural.  We could even have a
standard property like `no-search' that could be part of the default
`search-invisible' value, so that modes can easily mark out regions that
shouldn't be searched for.  In which case, the whole `display' thing
becomes moot -- in my mode, I could just slap the `no-search' property
on all the image display bits that I want isearch to avoid?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: search-invisible and friends

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  emacs-devel@gnu.org
> Date: Thu, 10 Sep 2020 14:35:50 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Right, but we wouldn't to re-implement the test in isearch.el, so the
> >> question is to find an existing function which gives us the answer.
> >
> > We'd need to write it, I think.
> 
> So basically mash up this stuff into a convenient DEFUN?
> 
> 	  && !NILP (val = get_char_property_and_overlay
> 		              (make_fixnum (PT), Qdisplay, selected_window,
> 			       &overlay))
> 	  && display_prop_intangible_p (val, overlay, PT, PT_BYTE)

Yes, something like that.

But what about 'display' properties on overlay strings? do we want to
support that as well?

> I also read the "Display Specs That Replace The Text" node in the
> manual, but I'm not quite sure what the relevance is here -- it talks
> about how (display) text properties are collated (i.e., merged over eq
> sequences of text), but does isearch have to deal with that?

The rest of the section makes a point of saying, for each display
property kind, whether it is replacing or not.

Re: search-invisible and friends

[Stefan Monnier]

>> So basically mash up this stuff into a convenient DEFUN?
>> 
>> 	  && !NILP (val = get_char_property_and_overlay
>> 		              (make_fixnum (PT), Qdisplay, selected_window,
>> 			       &overlay))
>> 	  && display_prop_intangible_p (val, overlay, PT, PT_BYTE)
> Yes, something like that.

Agreed.

> But what about 'display' properties on overlay strings? do we want to
> support that as well?

We don't need it for the `search-invisible` case because those display
properties can only ever hide text that's not really in the buffer in
the first place (it's an after/before string).


        Stefan

Re: search-invisible and friends

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Thu, 10 Sep 2020 14:38:40 +0200
> Cc: emacs-devel@gnu.org
> 
> > How about extending search-invisible to support additional properties,
> > by providing some variable that would hold the list of such
> > properties?  Then we could also avoid searching in text that is
> > visible, if some Lisp needs that (for example, you could have a form
> > where you'd like to search only the writable fields).
> 
> Ah, yeah, that sounds quite useful and natural.  We could even have a
> standard property like `no-search' that could be part of the default
> `search-invisible' value, so that modes can easily mark out regions that
> shouldn't be searched for.  In which case, the whole `display' thing
> becomes moot -- in my mode, I could just slap the `no-search' property
> on all the image display bits that I want isearch to avoid?

Yes, those were my thoughts.

Re: search-invisible and friends

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Lars Ingebrigtsen <larsi@gnus.org>,  emacs-devel@gnu.org
> Date: Thu, 10 Sep 2020 09:21:22 -0400
> 
> > But what about 'display' properties on overlay strings? do we want to
> > support that as well?
> 
> We don't need it for the `search-invisible` case because those display
> properties can only ever hide text that's not really in the buffer in
> the first place (it's an after/before string).

The proposed primitive can be used for something other than
search-invisible use cases.

Re: search-invisible and friends

[Stefan Monnier]

>> Ah, yeah, that sounds quite useful and natural.  We could even have a
>> standard property like `no-search' that could be part of the default
>> `search-invisible' value, so that modes can easily mark out regions that
>> shouldn't be searched for.  In which case, the whole `display' thing
>> becomes moot -- in my mode, I could just slap the `no-search' property
>> on all the image display bits that I want isearch to avoid?
> Yes, those were my thoughts.

It'd be good to have such a `no-search` property, but I think it's also
important to make the `search-invisible` automatically handle text
that's hidden by a `display` property, without having to add an explicit
`no-search`.

OTOH, maybe there could be unusual circumstances where it would make
sense to consider hidden text as "visible" (e.g. if the `display`
property replaces the text with something that looks very much like the
same text, tho maybe rendered a bit differently).  So, maybe there
should be a value of `no-search` (or some other property) which says
"please do consider this text as visible even tho it's technically
hidden".


        Stefan

Re: search-invisible and friends

[Juri Linkov]

>>> Ah, yeah, that sounds quite useful and natural.  We could even have a
>>> standard property like `no-search' that could be part of the default
>>> `search-invisible' value, so that modes can easily mark out regions that
>>> shouldn't be searched for.  In which case, the whole `display' thing
>>> becomes moot -- in my mode, I could just slap the `no-search' property
>>> on all the image display bits that I want isearch to avoid?
>> Yes, those were my thoughts.
>
> It'd be good to have such a `no-search` property,

Like dired-isearch-filter-filenames adds own filter,
this could be implemented like:

(defun isearch-filter-no-search (beg end)
  (not (text-property-not-all (min beg end) (max beg end)
			      'no-search nil)))

(add-function :before-while (local 'isearch-filter-predicate)
              #'isearch-filter-no-search)

(insert (propertize "foo" 'display "bar" 'no-search t))

> but I think it's also important to make the `search-invisible`
> automatically handle text that's hidden by a `display` property,
> without having to add an explicit `no-search`.

Maybe like this:

(defun isearch-filter-display (beg end)
  (not (text-property-not-all (min beg end) (max beg end)
			      'display nil)))

(add-function :before-while (local 'isearch-filter-predicate)
              #'isearch-filter-display)

(insert (propertize "foo" 'display "bar"))

> OTOH, maybe there could be unusual circumstances where it would make
> sense to consider hidden text as "visible" (e.g. if the `display`
> property replaces the text with something that looks very much like the
> same text, tho maybe rendered a bit differently).

Do you mean to match text in the display replacement, i.e. the string "bar"?
This is more complicated but it's possible by using get-text-property and
string-match to match the search string in the display property.

RE: search-invisible and friends

[Drew Adams]

> >> If not, I think it should, and perhaps (or perhaps not?) by
> >> piggy-backing on the search-invisible variable.
> >
> > How about extending search-invisible to support additional properties,
> > by providing some variable that would hold the list of such
> > properties?  Then we could also avoid searching in text that is
> > visible, if some Lisp needs that (for example, you could have a form
> > where you'd like to search only the writable fields).
> 
> Ah, yeah, that sounds quite useful and natural.

I too support the general idea.  But I do hope that
the existing code interface for search-invisible
continues to work as is.  IOW, can we add such a
feature without breaking existing code?

> We could even have a
> standard property like `no-search' that could be part of the default
> `search-invisible' value, so that modes can easily mark out regions that
> shouldn't be searched for.  In which case, the whole `display' thing
> becomes moot -- in my mode, I could just slap the `no-search' property
> on all the image display bits that I want isearch to avoid?

It may not be just what you're asking for there, but
my code (libraries `zones.el' and `isearch-prop.el')
lets you easily toggle zones of text between visible
and visible (which affects search as expected).

It also lets you toggle visibility of the anti-zones,
i.e., the complement of a set of zones.

And you can limit isearch to being inside or outside
a set of zones.  (This includes the simple case of
limiting search to the region - inside or outside it.)

And you can interactively define zones by the presence
of text properties (including with particular values).
And you can interactively put properties on the text
in a set of zones, or add properties to areas of text
that have other given properties and property values.

And you can narrow the zones to search to those with
lazy-highlighting.  Which means you can search for one
thing, and then refine the search by searching for
something else within those search hits.
____


Zones description:
https://www.emacswiki.org/emacs/Zones

zones.el:
https://www.emacswiki.org/emacs/download/zones.el

Isearch Properties:
https://www.emacswiki.org/emacs/IsearchPlus

isearch-prop.el:
https://www.emacswiki.org/emacs/download/isearch-prop.el

Re: search-invisible and friends

[Stefan Monnier]

>> OTOH, maybe there could be unusual circumstances where it would make
>> sense to consider hidden text as "visible" (e.g. if the `display`
>> property replaces the text with something that looks very much like the
>> same text, tho maybe rendered a bit differently).
> Do you mean to match text in the display replacement,

No, I mean to ignore `search-invisible` on those specific chunks (but
still only do the search on the buffer text).


        Stefan

Re: search-invisible and friends

[Ihor Radchenko]

>> Do you mean to match text in the display replacement,
>
> No, I mean to ignore `search-invisible` on those specific chunks (but
> still only do the search on the buffer text).

FYI, I proposed similar patch earlier:
https://lists.gnu.org/archive/html/emacs-devel/2020-07/msg00679.html

Best,
Ihor

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> OTOH, maybe there could be unusual circumstances where it would make
>>> sense to consider hidden text as "visible" (e.g. if the `display`
>>> property replaces the text with something that looks very much like the
>>> same text, tho maybe rendered a bit differently).
>> Do you mean to match text in the display replacement,
>
> No, I mean to ignore `search-invisible` on those specific chunks (but
> still only do the search on the buffer text).
>
>
>         Stefan

Re: search-invisible and friends

[Stefan Monnier]

>>> Do you mean to match text in the display replacement,
>> No, I mean to ignore `search-invisible` on those specific chunks (but
>> still only do the search on the buffer text).
> FYI, I proposed similar patch earlier:
> https://lists.gnu.org/archive/html/emacs-devel/2020-07/msg00679.html

OMG, yes, it's still in my TODO.

Note that what I'm suggesting is somewhat orthogonal since it's about
making it possible for a text (or overlay) property to say not to skip
over a particular (hidden) chunk of text, regardless of whether (and
how) we'd open the text to show it or not.


        Stefan

Re: search-invisible and friends

[Ihor Radchenko]

> Note that what I'm suggesting is somewhat orthogonal since it's about
> making it possible for a text (or overlay) property to say not to skip
> over a particular (hidden) chunk of text, regardless of whether (and
> how) we'd open the text to show it or not.

Isn't 'isearch-open-invisible property doing it already? If it is
non-nil, the hidden text is not skipped by isearch.

-----

Also, adding on the initial topic of skipping some technically not
invisible text. Similar functionality is required by org-mode branch I
am working on - some folded (temporarily hidden) may better not be
searchable. I implemented it by defining custom
`isearch-filter-predicate'.

The code I used is below. For context, org-fold--isearch-specs lists
'invisible property values that should be searchable.
org-fold-get-folding-specs-in-region retrieves all 'invisible property
values in region.

The code decides if isearch can search in region depending on the value
of 'invisible property, but it can generally be any property.

(defun org-fold--isearch-filter-predicate-text-properties (beg end)
  "Make sure that text hidden by any means other than `org-fold--isearch-specs' is not searchable.
This function is intended to be used as `isearch-filter-predicate'."
  (and
   ;; Check folding specs that cannot be searched
   (seq-every-p (lambda (spec) (member spec org-fold--isearch-specs)) (org-fold-get-folding-specs-in-region beg end))
   ;; Check 'invisible properties that are not folding specs
   (or (eq search-invisible t) ; User wants to search, allow it
       (let ((pos beg)
	     unknown-invisible-property)
	 (while (and (< pos end)
		     (not unknown-invisible-property))
	   (when (and (get-text-property pos 'invisible)
		      (not (member (get-text-property pos 'invisible) org-fold--isearch-specs)))
	     (setq unknown-invisible-property t))
	   (setq pos (next-single-char-property-change pos 'invisible)))
	 (not unknown-invisible-property)))
   (or (and (eq search-invisible t)
	    ;; FIXME: this opens regions permanenly for now.
            ;; I also tried to force search-invisible 'open-all around
            ;; `isearch-range-invisible', but that somehow causes
            ;; infinite loop in `isearch-lazy-highlight'.
            (prog1 t
	      ;; We still need to reveal the folded location
	      (org-fold--isearch-show-temporary (cons beg end) nil)))
       (not (isearch-range-invisible beg end)))))

Best,
Ihor

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>>> Do you mean to match text in the display replacement,
>>> No, I mean to ignore `search-invisible` on those specific chunks (but
>>> still only do the search on the buffer text).
>> FYI, I proposed similar patch earlier:
>> https://lists.gnu.org/archive/html/emacs-devel/2020-07/msg00679.html
>
> OMG, yes, it's still in my TODO.
>
> Note that what I'm suggesting is somewhat orthogonal since it's about
> making it possible for a text (or overlay) property to say not to skip
> over a particular (hidden) chunk of text, regardless of whether (and
> how) we'd open the text to show it or not.
>
>
>         Stefan

Re: search-invisible and friends

[Stefan Monnier]

>> Note that what I'm suggesting is somewhat orthogonal since it's about
>> making it possible for a text (or overlay) property to say not to skip
>> over a particular (hidden) chunk of text, regardless of whether (and
>> how) we'd open the text to show it or not.
> Isn't 'isearch-open-invisible property doing it already? If it is
> non-nil, the hidden text is not skipped by isearch.

What I'm suggesting is a way to be able to refine what is considered
hidden and what is not considered hidden, which is why it's orthogonal
to the issue of what we do when we find text that's considered as hidden.


        Stefan

Re: search-invisible and friends

[Yuri Khan]

On Thu, 10 Sep 2020 at 22:04, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> OTOH, maybe there could be unusual circumstances where it would make
> sense to consider hidden text as "visible" (e.g. if the `display`
> property replaces the text with something that looks very much like the
> same text, tho maybe rendered a bit differently).  So, maybe there
> should be a value of `no-search` (or some other property) which says
> "please do consider this text as visible even tho it's technically
> hidden".

In this thread, I see attempts to implement an ad-hoc rule for
ignoring or not ignoring parts of buffer text when searching. I wonder
what happens if we attack the issue from a different angle.

The ‘display’ property can replace buffer text with a different string
or an image for purposes of display. Maybe it makes sense to have a
property that can similarly specify a different string for purposes of
search? A property that can replace buffer text with a different
string or a waveform for purposes of text-to-speech?

A base64-encoded image would specify ‘display’ to display an image,
‘search’ to an empty string, and ‘speak’ to "image" or something
descriptive (cf. HTML <img alt>).

A mode could propertize the string ‘LaTeX’ with ‘display’ = nice image
of the LaTeX logo but no ‘search’ or ‘speak’ properties (so search and
text-to-speech work on the original representation).

A mode that replaces one word with another could decide which aspects
(display, search and/or text-to-speech) it wants to affect and how.

Re: search-invisible and friends

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> The rest of the section makes a point of saying, for each display
> property kind, whether it is replacing or not.

Are we looking at different section of the manual?  The rest of the node
(for me) just says:

---
This function gives each of the first ten characters in the buffer a
‘display’ property which is a string ‘"A"’, but they don’t all get the
same string object.  The first two characters get the same string
object, so they are replaced with one ‘A’; the fact that the display
property was assigned in two separate calls to ‘put-text-property’ is
irrelevant.  Similarly, the next two characters get a second string
(‘concat’ creates a new string object), so they are replaced with one
‘A’; and so on.  Thus, the ten characters appear as five A’s.
---



-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: search-invisible and friends

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> OTOH, maybe there could be unusual circumstances where it would make
> sense to consider hidden text as "visible" (e.g. if the `display`
> property replaces the text with something that looks very much like the
> same text, tho maybe rendered a bit differently).

Yeah, we have the recent proposed fix for Customize buffers where
symbols like `fill-column' have a bunch of display properties put on top
that makes it look like "Fill Column"...  having isearch skip past all
that would be unintended.

> So, maybe there should be a value of `no-search` (or some other
> property) which says "please do consider this text as visible even tho
> it's technically hidden".

I think perhaps the most obvious and flexible is to just make isearch
work the way it does today on text with a display property (i.e.,
agnostic), and then have a standard text property that modes can use to
make isearch skip.  I.e., just having a replacing display property in
itself isn't a strong enough hint that isearch should ignore the text.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: search-invisible and friends

[Lars Ingebrigtsen]

Yuri Khan <yuri.v.khan@gmail.com> writes:

> The ‘display’ property can replace buffer text with a different string
> or an image for purposes of display. Maybe it makes sense to have a
> property that can similarly specify a different string for purposes of
> search? A property that can replace buffer text with a different
> string or a waveform for purposes of text-to-speech?

Oh, that's smart and even more general.  Yes, I like it.

> A mode that replaces one word with another could decide which aspects
> (display, search and/or text-to-speech) it wants to affect and how.

Yup.

If the new text property is, say, `search', we could also have
special values like `ignore' to say that there is no alternative
(search) text, and `display', which means that if it's a textual
replacing display property, then isearch should use that text as the
search.

Like

(propertize "foo" 'display "bar" 'search 'display)

Or perhaps that's too complicated.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: search-invisible and friends

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Fri, 11 Sep 2020 13:37:05 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > The rest of the section makes a point of saying, for each display
> > property kind, whether it is replacing or not.
> 
> Are we looking at different section of the manual?  The rest of the node
> (for me) just says:

I meant the parent node and its sub-nodes (thus "section", not
"subsection").  Sorry for being unclear.

Re: search-invisible and friends

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  Eli Zaretskii
>  <eliz@gnu.org>,  Emacs developers <emacs-devel@gnu.org>
> Date: Fri, 11 Sep 2020 13:46:10 +0200
> 
> Yuri Khan <yuri.v.khan@gmail.com> writes:
> 
> > The ‘display’ property can replace buffer text with a different string
> > or an image for purposes of display. Maybe it makes sense to have a
> > property that can similarly specify a different string for purposes of
> > search? A property that can replace buffer text with a different
> > string or a waveform for purposes of text-to-speech?
> 
> Oh, that's smart and even more general.  Yes, I like it.
> 
> > A mode that replaces one word with another could decide which aspects
> > (display, search and/or text-to-speech) it wants to affect and how.
> 
> Yup.

My main problem with this proposal is that it places the onus on the
Lisp program which generates the text with display and other
properties, instead of giving control to the search command.  That
would mean that existing Lisp programs that use these properties will
not support these features of searching.

Re: search-invisible and friends

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> My main problem with this proposal is that it places the onus on the
> Lisp program which generates the text with display and other
> properties, instead of giving control to the search command.  That
> would mean that existing Lisp programs that use these properties will
> not support these features of searching.

It does, but I think only the code that generates the text covered by
display property knows that its semantics are.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: search-invisible and friends

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: yuri.v.khan@gmail.com,  monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Fri, 11 Sep 2020 16:40:53 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > My main problem with this proposal is that it places the onus on the
> > Lisp program which generates the text with display and other
> > properties, instead of giving control to the search command.  That
> > would mean that existing Lisp programs that use these properties will
> > not support these features of searching.
> 
> It does, but I think only the code that generates the text covered by
> display property knows that its semantics are.

What do you mean by "semantics" here?  The effect of each kind of
display property wrt hiding the covered text is known in advance.

Re: search-invisible and friends

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> It does, but I think only the code that generates the text covered by
>> display property knows that its semantics are.
>
> What do you mean by "semantics" here?  The effect of each kind of
> display property wrt hiding the covered text is known in advance.

If you have an image in the buffer, it may be something that has
meaning, or it may be decoration.  If it has meaning, it would be nice
if a blind user can search to it (for instance, if it's a button that
can be pressed, then the searchable text might be "click to execute").
If it's just decoration, then it shouldn't show up in isearch at all.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: search-invisible and friends

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: yuri.v.khan@gmail.com,  monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Fri, 11 Sep 2020 16:56:54 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> It does, but I think only the code that generates the text covered by
> >> display property knows that its semantics are.
> >
> > What do you mean by "semantics" here?  The effect of each kind of
> > display property wrt hiding the covered text is known in advance.
> 
> If you have an image in the buffer, it may be something that has
> meaning, or it may be decoration.  If it has meaning, it would be nice
> if a blind user can search to it (for instance, if it's a button that
> can be pressed, then the searchable text might be "click to execute").
> If it's just decoration, then it shouldn't show up in isearch at all.

You are talking about adding features we currently don't have, like
alternate-text for images in HTML.

But I was worried about supporting the existing features that simply
hide text, and the desire to allow the user to say whether he/she
wants or doesn't want to find search matches within such hidden text.
AFAIU, that cannot be provide by Yuri's proposal, because that
proposal requires the code which ides text to be amended.

I hope we can come up with a solution that allows both use cases.

Re: search-invisible and friends

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> You are talking about adding features we currently don't have, like
> alternate-text for images in HTML.

Yuri's suggestion is a generalisation of your idea of adding something
like a `no-search' text property...

> But I was worried about supporting the existing features that simply
> hide text, and the desire to allow the user to say whether he/she
> wants or doesn't want to find search matches within such hidden text.
> AFAIU, that cannot be provide by Yuri's proposal, because that
> proposal requires the code which ides text to be amended.
>
> I hope we can come up with a solution that allows both use cases.

After you suggested adding a `no-search' text property, my original use
case (modes being able to say whether an image `display' property should
"hide" the text beneath from isearch) became less enticing to me.  These
modes can just slap a `no-search' text property on the images instead,
so it seems unnecessary.

And then Yuri's suggestion generalises upon this, making my original use
case even less vital.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: search-invisible and friends

[Stefan Monnier]

>> > A mode that replaces one word with another could decide which aspects
>> > (display, search and/or text-to-speech) it wants to affect and how.
>> Yup.
> My main problem with this proposal is that it places the onus on the
> Lisp program which generates the text with display and other
> properties, instead of giving control to the search command.  That
> would mean that existing Lisp programs that use these properties will
> not support these features of searching.

There's also the question of writing the code that performs the search
while obeying those properties (for added fun, there's the case of
matches that span chunks of text covered and not-covered).


        Stefan