Idea: Be able to use text properties as face attributes

Idea: Be able to use text properties as face attributes

[Drew Adams]

Proposal for discussion.  I think it could be a useful
feature.

(It would no doubt involve Emacs C code, which I wouldn't be
helping with.  I could help with updating the doc, though.)

The idea is to allow the use of text properties as
attributes for the text (and overlay) properties `face',
`font-lock-face', and `mouse-face'.  This would apply to
faces used in strings (e.g., `propertize') as well as in
buffers.

It would let you, for example, use a property such as
`invisible', `keymap', or `read-only' (even `display'?) as
part of a face definition.

Any properties that either cannot be implemented or do not
make sense for faces would be excluded (i.e., ignored).  But
I'm guessing that most could be accommodated.

Setting such a face attribute would have the effect that all
occurrences of text with that face, when redisplayed, would
appear as if the text property were applied directly to the
text that has that face.

For example, this would make all doc strings invisible:

  (set-face-attribute 'font-lock-doc-face nil 'invisible t)

Face merging is already well defined, in particular for
cases where there is a conflict between the values of the
same attribute, so I don't expect any problem there.

One open question would be whether to allow the text
properties `face', `font-lock-face', and `mouse-face' as
properties for themselves and each other.  E.g., be able
to do this:

  (set-face-attribute 'face-1 nil 'face 'face-2)

or this:

  (set-face-attribute 'face-1 nil 'font-lock-face 'face-1)

These properties could be excluded (ignored), but allowing
them might prove useful.  It would provide a simple way to
(e.g. temporarily) substitute one face for another or to
give control over a face to font-lock (change `face' to
`font-lock-face') or vice versa.

WDOT?  Is this something that would be useful?  If so, would
someone like to work on implementing it?

Re: Idea: Be able to use text properties as face attributes

[Clément Pit-Claudel]

On 2017-03-26 15:14, Drew Adams wrote:
> WDOT?  Is this something that would be useful?  If so, would
> someone like to work on implementing it?

This sounds interesting, and the proposal is clear and nice.  Can you think of concrete examples where it would things much easier for package developers (your example with 'invisible is nice)?

I can think of one, if I understand correctly.  Currently, if one want font-lock to apply properties other tan face, then one must add these properties to font-lock-extra-managed-props — and then font-lock will remove these properties indiscriminately, regardless of where they came from.  This makes things difficult sometimes. With your proposal, these attributes could be tucked in a face.  On the other hand, one could imagine font-lock being smarter and removing only the properties that it added.

I wonder if your suggestion could be generalized.  Fundamentally, is there a good reason to have a distinction between face properties and text properties ? Then faces would work in a similar fashion to overlay categories (a quick way to set consistent properties on many chunks of text)

Clément.

RE: Idea: Be able to use text properties as face attributes

[Drew Adams]

> > WDOT?  Is this something that would be useful?  If so, would
> > someone like to work on implementing it?
> 
> This sounds interesting, and the proposal is clear and nice.  Can you
> think of concrete examples where it would things much easier for
> package developers (your example with 'invisible is nice)?

I haven't thought much about uses, in particular for packages.
I expect that others can come up with more possible uses.

The main point, for me, is that it provides a way to change
the properties of many stretches of text all at once, namely
all text (or all text on a given frame) that has a given face.

> I can think of one, if I understand correctly.  Currently, if one want font-
> lock to apply properties other tan face, then one must add these properties
> to font-lock-extra-managed-props — and then font-lock will remove these
> properties indiscriminately, regardless of where they came from.  This makes
> things difficult sometimes. With your proposal, these attributes could be
> tucked in a face.

Yes, that's what I meant by saying that you can turn all uses
of a given face from `face' uses to `font-lock-face' uses, i.e.,
hand its control over to font-lock.

I like your expression "these attributes could be tucked in[to]
a face".  Which means that manipulating/using the face means
also (indirectly) manipulating/using the other properties that
the face carries on board.  Wherever the face is used, so are
the other properties used.

And it means nothing beyond that.  In particular, as you say,
only the uses of that one face are affected.

And to be clear, it's not about this feature _instead of_ some
other approach.  In particular, when it suits, which is probably
most of the time, changing properties directly is fine.  You
don't _have_ to couple a face with properties that are not
face-related.

The point is only that you _can_ do that, so you can take
advantage of bulk-applying (or bulk-removing) such non-face
properties from a bunch of locations, all at once.  The face
that holds them acts as a _carrier_ for them.

> I wonder if your suggestion could be generalized.  Fundamentally,
> is there a good reason to have a distinction between face
> properties and text properties ?

Well, the suggestion is to let most (all?) text properties
_be_ face properties, in the sense that a face can hold them.
(That doesn't mean that they all need to have an effect on
the text that is related to the effect of the face on it.)

Clearly there is a distinction between text properties that
are related to faces and those that are not, where the latter
are (also or instead) related to things other than the
appearance of a face.

And let's not forget that anyone can create any text property
(e.g. `jkjkljkl'), with any value, and apply it to some text.

Some text properties that are not really related to faces
are built-in (`emacs -Q').  Others are created by user code
that alone determines how they behave - what their effect is.

But I guess you are suggesting the other direction: that
all face attributes would be applicable to text as text
properties on their own.  E.g., you would be able to do:

 (put-text-property 30 42 :family "Palatino")

That might be another possible enhancement.  But I think
that is already possible using an anonymous face (see
(elisp) `Special Properties'):

 (put-text-property 30 42 'face '(:family "Palatino"))

Is there something else you were thinking of here?

> Then faces would work in a similar fashion to overlay
> categories (a quick way to set consistent properties on
> many chunks of text)

Maybe you could elaborate, e.g., with a (fictitious) example?
Maybe show what else you think is needed for that, beyond
what I suggested.

(BTW, I just noticed that the text property `category' and
the overlay property `category' are apparently completely
different beasts.  Why did we choose to use the same name
for them?  Seems like the text property should instead be
called `char-category' or some such.)

Re: Idea: Be able to use text properties as face attributes

[Clément Pit-Claudel]

On 2017-03-26 17:42, Drew Adams wrote:
> That might be another possible enhancement.  But I think
> that is already possible using an anonymous face (see
> (elisp) `Special Properties'):
> 
>  (put-text-property 30 42 'face '(:family "Palatino"))
> 
> Is there something else you were thinking of here?

Not much: it's just an idea I had while reading your post.  it seems somewhat arbitrary to add a level of indirection for certain properties (if I want to make some text bold, I need to append to its 'face property — whereas if I want to make it invisible, I can just apply 'invisible to it.  Maybe the two concepts would gain from being unified ?

> Maybe you could elaborate, e.g., with a (fictitious) example?
> Maybe show what else you think is needed for that, beyond
> what I suggested.

I think your proposal would make what I described possible :)  I was describing a possible generalization of it ((1) suppressing the distinction between text properties and face properties, and (2) generalizing faces to be collections of arbitrary text properties — your proposal is essentially (2), and I think it enables the conceptual cleanup (1)).

Clément.

RE: Idea: Be able to use text properties as face attributes

[Drew Adams]

> > That might be another possible enhancement.  But I think
> > that is already possible using an anonymous face (see
> > (elisp) `Special Properties'):
> >  (put-text-property 30 42 'face '(:family "Palatino"))
> > Is there something else you were thinking of here?
> 
> Not much: it's just an idea I had while reading your post.  it seems
> somewhat arbitrary to add a level of indirection for certain properties (if
> I want to make some text bold, I need to append to its 'face property —
> whereas if I want to make it invisible, I can just apply 'invisible to it.
> Maybe the two concepts would gain from being unified ?

OK.  As I said, that is really a separate enhancement.  I have
no problem with it.  But for now we already have a way to set
properties, including such face-attribute properties, at
individual text locations.  What we do not have is a way to
set properties at all locations of a given face.

> > Maybe you could elaborate, e.g., with a (fictitious) example?
> > Maybe show what else you think is needed for that, beyond
> > what I suggested.
> 
> I think your proposal would make what I described possible :)

I don't see how.  It seems like another code change would be
needed to support it.  Not that both could not be done at the
same time, but I don't see that what I suggested, by itself,
would also realize what you suggest. 

> I was describing a possible generalization of it ((1) suppressing
> the distinction between text properties and face properties, and
> (2) generalizing faces to be collections of arbitrary text properties —
> your proposal is essentially (2), and I think it enables the conceptual
> cleanup (1)).

Like my suggestion, the conceptual change would require both a
code change (to support it) and a doc change (to express it).

If both enhancements were made at the same time then yes, the
doc change for my suggestion would be different.  Again, a
priori I have nothing against (someone) making both changes.

Thanks for thinking about this.

Re: Idea: Be able to use text properties as face attributes

[Clément Pit-Claudel]

On 2017-03-26 19:26, Drew Adams wrote:
>>> Maybe you could elaborate, e.g., with a (fictitious) example?
>>> Maybe show what else you think is needed for that, beyond
>>> what I suggested.
>>
>> I think your proposal would make what I described possible :)
> 
> I don't see how.  It seems like another code change would be
> needed to support it.  Not that both could not be done at the
> same time, but I don't see that what I suggested, by itself,
> would also realize what you suggest.

You're right.  I just think that implementing your suggestion would be a significant first step towards what I described (and in fact would get us most of the way there, inasmuch as the rest of the changes would be rather minimal).

> If both enhancements were made at the same time then yes, the
> doc change for my suggestion would be different.  Again, a
> priori I have nothing against (someone) making both changes.
> 
> Thanks for thinking about this.

Thank you for suggesting this!

RE: Idea: Be able to use text properties as face attributes

[Drew Adams]

(I can't check this right now, but) I wonder if it might not
be as simple as changing `get-text-property' to check whether
the character at the given POSITION either has the given PROP
or has a face property (`face', `font-lock-face', `mouse-face')
that has the given PROP as one of its attributes, and if so
to return the PROP value.

Priority should be given to the latter (PROP on the face) if 
both exist, in my view.  Dunno about priority among `face',
`font-lock-face', and `mouse-face', if more than one of them
has the PROP and those PROP values differ.

Re: Idea: Be able to use text properties as face attributes

[Yuri Khan]

On Mon, Mar 27, 2017 at 2:14 AM, Drew Adams <drew.adams@oracle.com> wrote:

> The idea is to allow the use of text properties as
> attributes for the text (and overlay) properties `face',
> `font-lock-face', and `mouse-face'.  This would apply to
> faces used in strings (e.g., `propertize') as well as in
> buffers.
>
> It would let you, for example, use a property such as
> `invisible', `keymap', or `read-only' (even `display'?) as
> part of a face definition.

I can see a direct parallel between this proposal and the spirit of CSS.

Basically, an Emacs face is analogous to a CSS class, in that it is a
collection of properties affecting display, which can be added to and
removed from a run of text atomically as a whole.

Text properties, on the other hand, are analogous to applying styling
properties directly to text. Which makes them difficult to control.

> One open question would be whether to allow the text
> properties `face', `font-lock-face', and `mouse-face' as
> properties for themselves and each other.  E.g., be able
> to do this:
>
>   (set-face-attribute 'face-1 nil 'face 'face-2)

:inherit already does something similar.

RE: Idea: Be able to use text properties as face attributes

[Drew Adams]

> (I can't check this right now, but) I wonder if it might not
> be as simple as changing `get-text-property' to check whether
> the character at the given POSITION either has the given PROP
> or has a face property (`face', `font-lock-face', `mouse-face')
> that has the given PROP as one of its attributes, and if so
> to return the PROP value.

I sent that quickly, without qualification or explanation.
I was talking only about reading the `face' property and
determining the effect to be given by the display engine.

I didn't really mean to imply that `get-text-property'
would itself be changed in any way.  It should not be.
(But perhaps it would be useful to have another function,
`get-text-property-implied', which does give you such a
value.)

In my view, the effect of a applying a face whose spec
contains non-face properties to some text would not
change the text properties of that text (other than its
`face', `font-lock-face', or `mouse-face' property).
It would change only the rendered _effect_, i.e., what
the display engine does.  The text would appear as if
the non-face properties were applied directly to the
text, but in fact they would remain encapsulated within
the `face' etc. property.

There would thus be a difference between applying a
non-face property directly to the text and "applying"
it indirectly by adding it as a face attribute for a
`face' etc. property that is applied to it.  That keeps
things clean and lets you change or undo the effect of
that property _for that face_ only, by changing its
face spec.

> Priority should be given to the latter (PROP on the face) if
> both exist, in my view.  Dunno about priority among `face',
> `font-lock-face', and `mouse-face', if more than one of them
> has the PROP and those PROP values differ.

RE: Idea: Be able to use text properties as face attributes

[Drew Adams]

> > One open question would be whether to allow the text
> > properties `face', `font-lock-face', and `mouse-face' as
> > properties for themselves and each other.  E.g., be able
> > to do this:
> >
> >   (set-face-attribute 'face-1 nil 'face 'face-2)
> 
> :inherit already does something similar.

No, I don't think it does (depending on what you mean
by "similar").  In what I'm describing that sexp would
make all occurrences of `face-1' have the effect of
being occurrences of `face-2'.  That's not what face
inheritance does.

Re: Idea: Be able to use text properties as face attributes

[Eli Zaretskii]

> Date: Sun, 26 Mar 2017 12:14:38 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> 
> The idea is to allow the use of text properties as
> attributes for the text (and overlay) properties `face',
> `font-lock-face', and `mouse-face'.  This would apply to
> faces used in strings (e.g., `propertize') as well as in
> buffers.
> 
> It would let you, for example, use a property such as
> `invisible', `keymap', or `read-only' (even `display'?) as
> part of a face definition.
> 
> Any properties that either cannot be implemented or do not
> make sense for faces would be excluded (i.e., ignored).  But
> I'm guessing that most could be accommodated.
> 
> Setting such a face attribute would have the effect that all
> occurrences of text with that face, when redisplayed, would
> appear as if the text property were applied directly to the
> text that has that face.
> 
> For example, this would make all doc strings invisible:
> 
>   (set-face-attribute 'font-lock-doc-face nil 'invisible t)

Some comments/questions about this:

First, I don't understand why you want to "burden" faces and the
display engine with this job.  It sounds like the job has nothing to
do with faces per se, and that faces were chosen simply because
font-lock is automatically applied by redisplay.  Is that right?  If
so, I think we should consider alternative, more straightforward
approaches.  For example, if what you are looking for is a way to put
a specific text property on any text with a particular FACE, I think
you can write a function to do that by searching for face property
changes and adding that property to each stretch of text with that
FACE.  Such a function can be written in Lisp, and doesn't require any
infrastructure changes.  Would such a function do the job you had in
mind?  We could make it part of Emacs, so people won't need to
reinvent it each time.

Using the display engine for this job is also problematic in several
other aspects:

 . It will slow down redisplay, because the display code will now have
   to examine faces where it previously didn't.  For example, for
   implementing invisible text: the current code looks just at the
   invisible properties of text and overlays; under your suggestion it
   will also have to look for faces.

 . Most text properties we support don't affect the display of text:
   consider 'keymap' or 'read-only' or 'intangible' or
   'modification-hooks', to name just a few.  And of course,
   applications can invent any properties they like.  How do you
   expect the display engine to make your face attributes "appear as
   if the corresponding text property was applied" to the text, if the
   corresponding text properties don't affect the text appearance on
   display?  The display engine doesn't apply any properties, it
   produces glyphs for displaying text.

In addition, text property search will also become slower, for the
same reason redisplay will become slower, as explained above.

> Face merging is already well defined, in particular for
> cases where there is a conflict between the values of the
> same attribute, so I don't expect any problem there.

Not sure if this is relevant, but if it is, then the existing face
merging won't help us, because it is only defined for visual
attributes, not for the additional ones you want to define.

> One open question would be whether to allow the text
> properties `face', `font-lock-face', and `mouse-face' as
> properties for themselves and each other.  E.g., be able
> to do this:
> 
>   (set-face-attribute 'face-1 nil 'face 'face-2)
> 
> or this:
> 
>   (set-face-attribute 'face-1 nil 'font-lock-face 'face-1)
> 
> These properties could be excluded (ignored), but allowing
> them might prove useful.  It would provide a simple way to
> (e.g. temporarily) substitute one face for another

How is this different from overriding attributes of a face by those of
another face, something I believe you can do today?

Re: Idea: Be able to use text properties as face attributes

[Eli Zaretskii]

> From: Clément Pit-Claudel <cpitclaudel@gmail.com>
> Date: Sun, 26 Mar 2017 16:01:13 -0400
> 
> This sounds interesting, and the proposal is clear and nice.  Can you think of 
> concrete examples where it would things much easier for package developers 
> (your example with 'invisible is nice)?
> 
> I can think of one, if I understand correctly.  Currently, if one want 
> font-lock to apply properties other tan face, then one must add these 
> properties to font-lock-extra-managed-props — and then font-lock will remove 
> these properties indiscriminately, regardless of where they came from.  This 
> makes things difficult sometimes. With your proposal, these attributes could be 
> tucked in a face.  On the other hand, one could imagine font-lock being smarter 
> and removing only the properties that it added.

Or we could provide a JIT mechanism to apply properties other than
face properties.

> Fundamentally, is there a good reason to have a distinction between
> face properties and text properties ?

I don't think I understand the question.  Face properties _are_ text
properties.  Or are you talking about face attributes?  If the latter,
then every text property can potentially have attributes, so face
attributes are not special in this respect.

Re: Idea: Be able to use text properties as face attributes

[Eli Zaretskii]

> From: Yuri Khan <yuri.v.khan@gmail.com>
> Date: Mon, 27 Mar 2017 14:56:17 +0700
> Cc: Emacs developers <emacs-devel@gnu.org>
> 
> Text properties, on the other hand, are analogous to applying styling
> properties directly to text.

Please don't forget that many/most text properties have no effect
whatsoever on how text is displayed.  So text properties mostly do NOT
apply style to stretches of text, they have other effects.

Re: Idea: Be able to use text properties as face attributes

[Eli Zaretskii]

> Date: Mon, 27 Mar 2017 07:01:22 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> 
> In my view, the effect of a applying a face whose spec
> contains non-face properties to some text would not
> change the text properties of that text (other than its
> `face', `font-lock-face', or `mouse-face' property).
> It would change only the rendered _effect_, i.e., what
> the display engine does.

I don't understand what that means, for most text properties, since
they don't affect the visual appearance of the text in any way.  Which
properties, except 'invisible', did you have in mind?  I'm confused.

RE: Idea: Be able to use text properties as face attributes

[Drew Adams]

> > In my view, the effect of a applying a face whose spec
> > contains non-face properties to some text would not
> > change the text properties of that text (other than its
> > `face', `font-lock-face', or `mouse-face' property).
> > It would change only the rendered _effect_, i.e., what
> > the display engine does.
> 
> I don't understand what that means, for most text properties, since
> they don't affect the visual appearance of the text in any way.  Which
> properties, except 'invisible', did you have in mind?  I'm confused.

It's about the effect of a text property, regardless of
what the effect is or where in the Emacs base code that
effect is applied.  It's not about only text properties
that visually affect the display of the text they are
applied to.

I mentioned "rendered" and "display engine" only as a
shortcut, and because I am not familiar with the actual
code that implements the various effects.

It's not important (to my general description of the
feature) which code in fact makes text that has a `keymap'
property behave as if the keymap at that place were the
keymap that is the value of that text property.

There are lots of text properties, and as you say, many
have nothing really to do with how the text they are
applied to "appears".  They do have some relation to
that text, and there is some effect of their being
present.  And some Emacs code manages that effect.

The idea of the feature is that a face property that
contains text properties as (pseudo) face attributes
would produce the same effect as applying those text
properties to the text.  But those text properties
would not in fact be applied to the text - they would
remain encapsulated in the value of the applied face
property (`face', `font-lock-face', or `mouse-face').

RE: Idea: Be able to use text properties as face attributes

[Drew Adams]

> > The idea is to allow the use of text properties as
> > attributes for the text (and overlay) properties `face',
> > `font-lock-face', and `mouse-face'.  This would apply to
> > faces used in strings (e.g., `propertize') as well as in
> > buffers.
> >
> > It would let you, for example, use a property such as
> > `invisible', `keymap', or `read-only' (even `display'?) as
> > part of a face definition.
> >
> > Any properties that either cannot be implemented or do not
> > make sense for faces would be excluded (i.e., ignored).  But
> > I'm guessing that most could be accommodated.
> >
> > Setting such a face attribute would have the effect that all
> > occurrences of text with that face, when redisplayed, would
> > appear as if the text property were applied directly to the
> > text that has that face.
> >
> > For example, this would make all doc strings invisible:
> >
> >   (set-face-attribute 'font-lock-doc-face nil 'invisible t)
> 
> Some comments/questions about this:
> 
> First, I don't understand why you want to "burden" faces and the
> display engine with this job.  It sounds like the job has nothing to
> do with faces per se, and that faces were chosen simply because
> font-lock is automatically applied by redisplay.  Is that right?

No.  It has nothing particular to do with font-lock.  As I said,
it is about (having the effect of) changing the text properties
at all occurrences of a given face, all at once (modulo redisplay
time)

> If so, I think we should consider alternative, more straightforward
> approaches.  For example, if what you are looking for is a way to put
> a specific text property on any text with a particular FACE,

The "if so" part is wrong - see above.

And this is not about actually putting a text property on the text.
This is about getting that effect without doing that.  The actual
text properties on the relevant text would be those that it had
before, with the exception that its `face' property would have
(pseudo face) attributes that correspond to non-face text properties.

The latter would be taken into account by the display engine, to
give the effect of their having been put on the text directly.

The indirection allows confining this behavior to the face
occurrences and removing it from only the face occurrences.

> I think
> you can write a function to do that by searching for face property
> changes and adding that property to each stretch of text with that
> FACE.

See above.

It is not about having a convenient way to directly alter the text
properties of the subject text.  It is about getting that effect
without altering them.  It is about associating that effect with
the particular face (only).

> Such a function can be written in Lisp, and doesn't require any
> infrastructure changes.  Would such a function do the job you had in
> mind?  We could make it part of Emacs, so people won't need to
> reinvent it each time.

No; please see above.  I'm not looking for a way to add text
properties wherever a given face property is present.  I'm
looking for a way to encapsulate a set of text properties
with a face, so they effect the display of text that has that
face, but they are not directly applied to the text.  They are
tied to presence of the face, and they can be changed or removed
by changing the face spec.

It's just about using the face as a "carrier" of text properties.

> Using the display engine for this job is also problematic in several
> other aspects:
> 
>  . It will slow down redisplay, because the display code will now have
>    to examine faces where it previously didn't.  For example, for
>    implementing invisible text: the current code looks just at the
>    invisible properties of text and overlays; under your suggestion it
>    will also have to look for faces.

Correct.

And more generally, it would affect tests for a text property.
As mentioned in my previous mail, I don't think that the current
functions (`get-text-property', `next-single-property-change, etc.)
that test for text properties should be changed.

Instead, we would add parallel functions that test for text
properties not just being directly present but also being
"present" by implication, i.e., indirectly, i.e., by way of
a face property.

I can't speak to the implementation (display engine etc.).
I'm glad that you are at least open to thinking about this,
since you are our expert on the display engine.  If the
idea is totally unfeasible or unrealistic, you would likely
be the one to speak to that.

If it affects performance significantly to also test for a
face property and check its attributes for a text property
then perhaps that extra check could be controlled by a global
variable.  IOW, if code needs this feature then it can "turn
it on", and otherwise the display engine would act as it has
in the past.

>  . Most text properties we support don't affect the display of text:
>    consider 'keymap' or 'read-only' or 'intangible' or
>    'modification-hooks', to name just a few.

Yes.  I addressed such properties specifically.  I don't know
where in the base code they are handled, so I spoke only of
"display engine".

But from a user point of view they would be handled the same
way in this suggestion.  The property would not be added to
the text, but rather it would stay as an attribute of the
`face' (or `font-lock-face' or `mouse-face') property.  It
would have the _effect_ of having been added to the text.
E.g., the text with the particular face would behave as if
it had the given `keymap' property value.

And yes, most Lisp code that tests text for such a property
would likely use a different function to test/obtain it.
For example, it would use `get-text-property-implied' (new),
which would check first for a face property that has the
given property as an attribute.  If present, that would be
returned; if absent then the `get-text-property' result
would be returned.

We could also, say, have `get-text-property-implied'
distinguish, in its result, whether the property was directly
present or was implied by virtue of being part of a face spec.

One way to do that might be to put the return value in parens
for one of the cases.  For example, if implied, the return
value could be a cons (FACE . VALUE), where FACE is the face
and VALUE is the value of the text property in its face spec.

(Maybe a slightly different convention should be used, as
that's not foolproof.  A text property that is directly
present could have a value of a cons whose car is a face.)

>    And of course,
>    applications can invent any properties they like.

I addressed that too.

>    How do you
>    expect the display engine to make your face attributes "appear as
>    if the corresponding text property was applied" to the text, if the
>    corresponding text properties don't affect the text appearance on
>    display?  The display engine doesn't apply any properties, it
>    produces glyphs for displaying text.

I said this:

  And let's not forget that anyone can create any text property
  (e.g. `jkjkljkl'), with any value, and apply it to some text.

  Some text properties that are not really related to faces
  are built-in (`emacs -Q').  Others are created by user code
  that alone determines how they behave - what their effect is.

A user-defined text property is unknown to `emacs -Q'.  It is
up to the code that defines it to do what it wants with it,
including (if it wants) any indirect use of it as a face attribute.

IOW, not Emac's problem.  User code will typically not expect
or need any such feature, and if it does then it will need to
take care of that itself.

> In addition, text property search will also become slower, for the
> same reason redisplay will become slower, as explained above.

Yes; see above, where I mention `next-single-property-change'
as one example.

> > Face merging is already well defined, in particular for
> > cases where there is a conflict between the values of the
> > same attribute, so I don't expect any problem there.
> 
> Not sure if this is relevant, but if it is, then the existing face
> merging won't help us, because it is only defined for visual
> attributes, not for the additional ones you want to define.

Either we could decide that that is acceptable, in which
case there would never be a conflict for such "attributes"
(e.g., the "latest" face spec applied would win, wrt their
effect - or only the first face spec would win), or we
could enhance the face-merging code to treat any unknown
face "attributes" similarly: have Emacs "apply" them as
if that were the only face.

> > One open question would be whether to allow the text
> > properties `face', `font-lock-face', and `mouse-face' as
> > properties for themselves and each other.  E.g., be able
> > to do this:
> >   (set-face-attribute 'face-1 nil 'face 'face-2)
> > or this:
> >   (set-face-attribute 'face-1 nil 'font-lock-face 'face-1)
> >
> > These properties could be excluded (ignored), but allowing
> > them might prove useful.  It would provide a simple way to
> > (e.g. temporarily) substitute one face for another
> 
> How is this different from overriding attributes of a face by those of
> another face, something I believe you can do today?

Not sure what you have in mind.  The above would have the
_effect_ of showing all occurrences of `face-1' as if they
were occurrences of `face-2', or (in the second example) of
handling them as if they were occurrences of `font-lock-face'.

Re: Idea: Be able to use text properties as face attributes

[Eli Zaretskii]

> Date: Mon, 27 Mar 2017 09:22:23 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: emacs-devel@gnu.org
> 
> > > Setting such a face attribute would have the effect that all
> > > occurrences of text with that face, when redisplayed, would
> > > appear as if the text property were applied directly to the
> > > text that has that face.
> > >
> > > For example, this would make all doc strings invisible:
> > >
> > >   (set-face-attribute 'font-lock-doc-face nil 'invisible t)
> > 
> > Some comments/questions about this:
> > 
> > First, I don't understand why you want to "burden" faces and the
> > display engine with this job.  It sounds like the job has nothing to
> > do with faces per se, and that faces were chosen simply because
> > font-lock is automatically applied by redisplay.  Is that right?
> 
> No.  It has nothing particular to do with font-lock.  As I said,
> it is about (having the effect of) changing the text properties
> at all occurrences of a given face, all at once (modulo redisplay
> time)

But you explicitly mention "redisplay" above, and the display engine
doesn't care about any properties unrelated to font-lock or faces.  So
now I'm even more confused about the proposal.

> And this is not about actually putting a text property on the text.
> This is about getting that effect without doing that.

What for?  You provided no rationale for having such "virtual"
properties.  The only rationale you provided is that of a convenience
feature which would allow to quickly and simply make specific portions
of text behave as if they have those properties.  I'm asking why not
actually give them those properties.  What does it gain us?

> >  . Most text properties we support don't affect the display of text:
> >    consider 'keymap' or 'read-only' or 'intangible' or
> >    'modification-hooks', to name just a few.
> 
> Yes.  I addressed such properties specifically.  I don't know
> where in the base code they are handled, so I spoke only of
> "display engine".

Text properties that have no effect on display are not handled by the
display engine.  They are handled by the features which implement the
effects of those text properties.  For example, the read-only property
is handled by code that modifies the buffer text: that code
specifically examines the text to be modified, and if it has the
read-only property, that code signals an error.  Under your proposal,
the code which implements the read-only property will now have to
examine face attributes of the text to be modified, in addition to the
text property.

And the same will have to happen with every feature which supports
some text property.  This additional handling of face attributes will
have to be replicated in every such feature; there are no single place
to do that for all of them, AFAIU.  This is clearly additional
non-trivial work, to be repeated in many Emacs features, which is a
disadvantage.  Can you tell what would be the advantages that justify
these additional costs?

> > Not sure if this is relevant, but if it is, then the existing face
> > merging won't help us, because it is only defined for visual
> > attributes, not for the additional ones you want to define.
> 
> Either we could decide that that is acceptable, in which
> case there would never be a conflict for such "attributes"
> (e.g., the "latest" face spec applied would win, wrt their
> effect - or only the first face spec would win), or we
> could enhance the face-merging code to treat any unknown
> face "attributes" similarly: have Emacs "apply" them as
> if that were the only face.

The display engine doesn't apply face attributes in the sense that you
seem to use this terminology.  It examines face attributes and
determines what typeface, colors, fonts, etc. to use to display the
characters which have those face attributes.  IOW, the only output of
face attribute examination is the description of what should be on the
screen, and that description is then used by the terminal-specific
interface to actually display that on the glass.

Do you see how any face attribute that has no effect on display will
not be "applied" as part of this processing?  The output of this
processing is not buffer text modified in some way, it's an entirely
different set of data structures which are not even visible from Lisp.
There simply isn't any place there to record the face attributes you
propose to add.

Re: Idea: Be able to use text properties as face attributes

[Eli Zaretskii]

> Date: Mon, 27 Mar 2017 08:49:47 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: emacs-devel@gnu.org
> 
> The idea of the feature is that a face property that
> contains text properties as (pseudo) face attributes
> would produce the same effect as applying those text
> properties to the text.  But those text properties
> would not in fact be applied to the text - they would
> remain encapsulated in the value of the applied face
> property (`face', `font-lock-face', or `mouse-face').

As I explained, the display engine, which is the part that implements
the faces, will be unable to do anything with attributes that don't
affect display.  And the implementation of the features which do need
to pay attention to such attributes is entirely unrelated to faces.
So I don't understand why you want this to be attributes of faces.

Moreover, you consistently mention the display engine as the main part
of this proposal.  But features that don't affect display cannot be
usefully built on top of the display code, because the display code is
designed to run and examine text only when and where there's likely to
be changes in buffer text that will affect the display.  By contrast,
the non display-related properties need to be processed regardless of
any changes that could affect display.  Do you see a conflict here
between the goals and the proposed design?

RE: Idea: Be able to use text properties as face attributes

[Drew Adams]

> > The idea of the feature is that a face property that
> > contains text properties as (pseudo) face attributes
> > would produce the same effect as applying those text
> > properties to the text.  But those text properties
> > would not in fact be applied to the text - they would
> > remain encapsulated in the value of the applied face
> > property (`face', `font-lock-face', or `mouse-face').
> 
> As I explained, the display engine, which is the part that implements
> the faces, will be unable to do anything with attributes that don't
> affect display.

Yes, you explained that.  Thanks.

Does that mean that such code could not invoke _other_ code
that _would_ be able to do something with those text properties
(that masquerade in a face spec as face attributes)?

> And the implementation of the features which do need
> to pay attention to such attributes is entirely unrelated to faces.
> So I don't understand why you want this to be attributes of faces.

I think by now you should understand why I proposed the
feature - its use.  I've made that pretty clear.

You are free to think that it's a useless feature, of course,
and you are free to think that it might be useful but is not
feasible to implement.

> Moreover, you consistently mention the display engine as the main part
> of this proposal.

See my replies to your (repeated) diatribes about my
mistakenly referring to the "display engine".  I was only
trying to describe the desired feature, not to speak to
which code really would, could, or should implement it.
Mille excuses for having naively invoked the terms
"redisplay" and "display engine".

> But features that don't affect display cannot be
> usefully built on top of the display code, because the display code is
> designed to run and examine text only when and where there's likely to
> be changes in buffer text that will affect the display.

I suppose it is natural that you immediately go down the
rabbit hole of thinking in terms of implementation.  And
that is no doubt helpful to the discourse overall.

But it doen't really jibe with your claim not to understand
what I mean by the feature itself and its raison d'etre.
So far, anyway, I haven't noticed anyone else who has not
gotten the point of the feature.  (It is true that we've
heard from only two other readers so far.)

> By contrast,
> the non display-related properties need to be processed regardless of
> any changes that could affect display.  Do you see a conflict here
> between the goals and the proposed design?

I didn't propose any design.  And you've already made clear
that you do not understand the goals.  But I guess that
doesn't stop you from seeing such a conflict.

One more time: I do not pretend to know how this feature
would, could, or should be implemented.

RE: Idea: Be able to use text properties as face attributes

[Drew Adams]

> > > First, I don't understand why you want to "burden" faces and the
> > > display engine with this job.  It sounds like the job has nothing to
> > > do with faces per se, and that faces were chosen simply because
> > > font-lock is automatically applied by redisplay.  Is that right?
> >
> > No.  It has nothing particular to do with font-lock.  As I said,
> > it is about (having the effect of) changing the text properties
> > at all occurrences of a given face, all at once (modulo redisplay
> > time)
> 
> But you explicitly mention "redisplay" above, and the display engine
> doesn't care about any properties unrelated to font-lock or faces.  So
> now I'm even more confused about the proposal.

Modulo "update" time, if you prefer.  (And this was
mentioned in paretheses, in hopes that it might help
understanding. You can ignore it if it does not help you.)

I have no idea where in the Emacs code particular text
properties are handled.  Some code, somewhere, takes care
of implementing their effects.

I don't know what triggers different kinds of such updates,
so I spoke in a general way about "redisplay".  Substitute
whatever wording you prefer, to indicate that there might
be (I don't know) some delay between (1) changing the face
property and (2) realizing the effect of doing so.

> > And this is not about actually putting a text property on the text.
> > This is about getting that effect without doing that.
> 
> What for?  You provided no rationale for having such "virtual"
> properties.  The only rationale you provided is that of a convenience
> feature which would allow to quickly and simply make specific portions
> of text behave as if they have those properties.  I'm asking why not
> actually give them those properties.  What does it gain us?

I did speak to that - several times now.  The point is to
associate the effect with the face.  One feature of that
is that you can change or remove the effect by changing
or removing (setting to nil) the property in the face spec.
Only text that has that face is to be affected.  Perhaps
it is that part that you have not yet grasped.

> > >  . Most text properties we support don't affect the display of text:
> > >    consider 'keymap' or 'read-only' or 'intangible' or
> > >    'modification-hooks', to name just a few.
> >
> > Yes.  I addressed such properties specifically.  I don't know
> > where in the base code they are handled, so I spoke only of
> > "display engine".
> 
> Text properties that have no effect on display are not handled by the
> display engine.  They are handled by the features which implement the
> effects of those text properties.  For example, the read-only property
> is handled by code that modifies the buffer text: that code
> specifically examines the text to be modified, and if it has the
> read-only property, that code signals an error.  Under your proposal,
> the code which implements the read-only property will now have to
> examine face attributes of the text to be modified, in addition to the
> text property.

Correct.  I made that clear in my sketch of checking the
"effective" text-property value: any face property on
the text would need to be checked in priority, to handle
its text-property attributes, if there are any.

> And the same will have to happen with every feature which supports
> some text property.

Correct.

> This additional handling of face attributes will
> have to be replicated in every such feature;

Correct - for every such feature we decide to support.

> there are no single place
> to do that for all of them, AFAIU.  This is clearly additional
> non-trivial work, to be repeated in many Emacs features, which is a
> disadvantage.  Can you tell what would be the advantages that justify
> these additional costs?

I can't speak to the cost.  You are doing so, which is good.
Yes, if we want to support a given text property then clearly
we will need to update the code that supports that text property

There is, however, no "requirement" that _all_ text properties
be supported.  I said clearly, in my first message:

  Any properties that either cannot be implemented or do not
  make sense for [associating with] faces would be excluded
  (i.e., ignored).  But I'm guessing that most could be
  accommodated.

In an extreme example (but I'd hope it wouldn't be so limited,
even at the outset), support could be provided for only one
or two text properties (e.g. `invisible') to begin with.
Support for more could be added if/when we (and users) see
that the support would be helpful.

> > > Not sure if this is relevant, but if it is, then the existing face
> > > merging won't help us, because it is only defined for visual
> > > attributes, not for the additional ones you want to define.
> >
> > Either we could decide that that is acceptable, in which
> > case there would never be a conflict for such "attributes"
> > (e.g., the "latest" face spec applied would win, wrt their
> > effect - or only the first face spec would win), or we
> > could enhance the face-merging code to treat any unknown
> > face "attributes" similarly: have Emacs "apply" them as
> > if that were the only face.
> 
> The display engine doesn't apply face attributes in the sense that you
> seem to use this terminology.  It examines face attributes and
> determines what typeface, colors, fonts, etc. to use to display the
> characters which have those face attributes.  IOW, the only output of
> face attribute examination is the description of what should be on the
> screen, and that description is then used by the terminal-specific
> interface to actually display that on the glass.

I didn't say anything about the display engine there.

What would needs to be done in terms of the implementation I
leave to those knowledgable in that.  Whether existing code
that "examines face attributes" should be extended to DTRT
for this feature or some other code should be brought in to
do that is not my concern.

I'm outlining a user-visible feature, not designing its
implementation.  There may be no interest in the feature.
Or there may be interest in it but it might be deemed
unfeasible or too costly in terms of implementation.

> Do you see how any face attribute that has no effect on display
> will not be "applied" as part of this processing?

I don't even know how to parst that question.  Maybe try
to rephrase it?

> The output of this processing

What processing are you talking about?

> is not buffer text modified in some way, it's an entirely
> different set of data structures which are not even visible from Lisp.
> There simply isn't any place there to record the face attributes you
> propose to add.

Sounds like you're saying that it cannot be coded.  Too bad,
in that case.  Anyway, you clearly are not interested in the
idea, if you are not outright hostile to it.  Since it would
no doubt need your participation to be implemented I guess
it's dead in the water.

Re: Idea: Be able to use text properties as face attributes

[Eli Zaretskii]

> Date: Mon, 27 Mar 2017 13:01:34 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: emacs-devel@gnu.org
> 
> > As I explained, the display engine, which is the part that implements
> > the faces, will be unable to do anything with attributes that don't
> > affect display.
> 
> Yes, you explained that.  Thanks.
> 
> Does that mean that such code could not invoke _other_ code
> that _would_ be able to do something with those text properties
> (that masquerade in a face spec as face attributes)?

We can invoke (almost) any code we like, but the question is what
would that other code do with those attributes?  Those attributes,
like their corresponding text properties, have their effect when the
corresponding Emacs features are invoked, like buffer text
modification for the 'read-only' property or keyboard input processing
for the 'keymap' property.  There's nothing we could do with these
attributes during processing of faces by the display engine, except
record those attributes somewhere, for future use by their respective
features.  Text properties are precisely a way to record such
information, but you don't want to use text properties in this case.
So some other means of recording the information will have to be
invented.  But since those means will most probably be very similar to
text properties, at least implementation-wise, what would be the point
of inventing that?

> > And the implementation of the features which do need
> > to pay attention to such attributes is entirely unrelated to faces.
> > So I don't understand why you want this to be attributes of faces.
> 
> I think by now you should understand why I proposed the
> feature - its use.  I've made that pretty clear.
>
> You are free to think that it's a useless feature, of course,
> and you are free to think that it might be useful but is not
> feasible to implement.

Sorry, I don't understand.  My questions and my confusion are real, so
please bear with me and don't assume I'm repeatedly asking the same
questions for argument's sake.  I'm asking them because I really don't
understand your motivation for proposing this feature in the form that
you proposed it.  It's your proposal, so only you can explain its
details.

Re: Idea: Be able to use text properties as face attributes

[Eli Zaretskii]

> Date: Mon, 27 Mar 2017 13:01:38 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: emacs-devel@gnu.org
> 
> I have no idea where in the Emacs code particular text
> properties are handled.  Some code, somewhere, takes care
> of implementing their effects.

I tried to explain that with the example of the 'read-only' property.
In a nutshell, every text property has special code somewhere in Emacs
that examines that property to modify the behavior of the feature as
the property instructs.  The only common infrastructure for that is
the code which allows Lisp programs to add, remove, modify, and
examine text properties at some position in a buffer or a string.  the
effect of each property must be coded separately for every property.

> I don't know what triggers different kinds of such updates,
> so I spoke in a general way about "redisplay".  Substitute
> whatever wording you prefer, to indicate that there might
> be (I don't know) some delay between (1) changing the face
> property and (2) realizing the effect of doing so.

Not sure what you mean by "update" here.  The realization of the
effect happens when the corresponding feature is used, so it isn't
time-driven, it's event-driven.  For face attributes that affect
display, that event is the next redisplay cycle, but for the
non-visual attributes you propose the events will be entirely
different, like buffer text modification for 'read-only'.

> > > And this is not about actually putting a text property on the text.
> > > This is about getting that effect without doing that.
> > 
> > What for?  You provided no rationale for having such "virtual"
> > properties.  The only rationale you provided is that of a convenience
> > feature which would allow to quickly and simply make specific portions
> > of text behave as if they have those properties.  I'm asking why not
> > actually give them those properties.  What does it gain us?
> 
> I did speak to that - several times now.  The point is to
> associate the effect with the face.  One feature of that
> is that you can change or remove the effect by changing
> or removing (setting to nil) the property in the face spec.
> Only text that has that face is to be affected.  Perhaps
> it is that part that you have not yet grasped.

Yes, I did grasp that.  But this single rationale is in effect a
convenience feature, nothing more.  It doesn't explain why this is
more convenient than just putting the corresponding properties on
those same stretches of text, nor why you'd like to avoid actually
placing text properties to achieve the same goal.  Nor does it explain
why the face property is being suggested as the vehicle for providing
this feature.  'Face' is just one of the many text properties we
support, and there's nothing in it that would make it a better means
for implementing your convenience feature.  At least I don't see why
'face' is better than any other property.  For example, would it work
for you if we define a new text property, whose attributes will have
the same effect as the corresponding text properties.  Would that be
as good as your 'face'-based proposal?  If not, why not?

> > And the same will have to happen with every feature which supports
> > some text property.
> 
> Correct.
> 
> > This additional handling of face attributes will
> > have to be replicated in every such feature;
> 
> Correct - for every such feature we decide to support.
> 
> > there are no single place
> > to do that for all of them, AFAIU.  This is clearly additional
> > non-trivial work, to be repeated in many Emacs features, which is a
> > disadvantage.  Can you tell what would be the advantages that justify
> > these additional costs?
> 
> I can't speak to the cost.

I didn't ask you to.  What I did ask is what are the user-level
advantages which would justify those costs, whatever they are.  I
don't think you answered that question.  Such an answer is part of the
rationale for the proposed changes; the absence of the answer is what
makes it hard for me to understand the rationale, which is a
prerequisite for reasoning about it.

> In an extreme example (but I'd hope it wouldn't be so limited,
> even at the outset), support could be provided for only one
> or two text properties (e.g. `invisible') to begin with.

Invisible is actually the odd one out, since it _is_ implemented by
the display engine.  It's all the rest that puzzle me.

If 'invisible' is the only one, i.e. you are asking for a feature that
would allow making text which has a certain face disappear from
display, that could perhaps be implemented without introducing any
face attributes.  IOW, it's a much narrower goal, and we don't
necessarily need to generalize it if no other use cases are apparent.

> I'm outlining a user-visible feature, not designing its
> implementation.

I'm asking you to provide the rationale for the feature, and for
attaching this feature to faces.  So far you provided only a single
rationale, for a single use case of the feature, and that use case can
be catered to by different means which don't require any additional
infrastructure.  I'm asking to provide more use cases and at least
some arguments for attaching this feature to faces.

> Sounds like you're saying that it cannot be coded.

No, I didn't say that.  I'm saying that I still don't understand why
it would make sense to provide such a feature, and why we should
attach it to faces.  Please help me understand your reasons.

> Anyway, you clearly are not interested in the idea, if you are not
> outright hostile to it.

At this point, I'm interested in understanding the idea.  Only then I
will be able to make up my mind about it.  I'm not there yet.

Re: Idea: Be able to use text properties as face attributes

[Stefan Monnier]

> for the 'keymap' property.  There's nothing we could do with these
> attributes during processing of faces by the display engine, except
> record those attributes somewhere, for future use by their respective
> features.

It's worse: the rest of the code can't assume that the display rendering
has propagated the info from the fact attributes to that
"to-be-defined-other-place" since redisplay may not have happened yet or
may not have looked at that part of the buffer yet.

This said, Drew's feature doesn't have to require changes all over
the place.  It might be possible to limit those changes to a few core
place like Fget_text_property.

This said, I think using face attributes is a bad idea.

Instead, we should consider adding some more general system of
indirection.  Drew suggests using `face` as a special meta-property.
Current code already supports such a feature but using the name
`category` and only for overlays (and with some problems linked to the
fact that modifying that meta-property is done via `get` and `put` which
are very generic operations used for many other things, so it's
hard/impossible to detect when those meta-properties are changed, which
is a fairly serious problem.  The only reason why we don't suffer too
much from that problem is because those `category` properties aren't
used very often).

My proposal for "property planes" is somewhat related to Drew's
proposal: my "property planes" allow merging various properties into one
(font-lock can place the (font-lock face) property, and some other
package can add the (mypkg face) property and they then get merged into
the one-and-only `face` property), i.e. many-to-one.  Drew wants the
reverse: adding a single meta-property which then affects several
other properties, i.e. one-to-many.

From a more general point of view, the shared desire is to have
"abstract text properties" from which "actual text properties"
get computed.


        Stefan

RE: Idea: Be able to use text properties as face attributes

[Drew Adams]

> > I have no idea where in the Emacs code particular text
> > properties are handled.  Some code, somewhere, takes care
> > of implementing their effects.
> 
> I tried to explain that with the example of the 'read-only' property.
> In a nutshell, every text property has special code somewhere in Emacs
> that examines that property to modify the behavior of the feature as
> the property instructs.  The only common infrastructure for that is
> the code which allows Lisp programs to add, remove, modify, and
> examine text properties at some position in a buffer or a string.  the
> effect of each property must be coded separately for every property.

You already made that clear.  And I already confirmed
that updates to the separate treatments for the different
properties would be needed.

Wherever the code is that handles a given text property,
if we decide that face specs should support that text
property then that code would need to be updated to
DTRT for it.

> > I don't know what triggers different kinds of such updates,
> > so I spoke in a general way about "redisplay".  Substitute
> > whatever wording you prefer, to indicate that there might
> > be (I don't know) some delay between (1) changing the face
> > property and (2) realizing the effect of doing so.
> 
> Not sure what you mean by "update" here.  The realization of the
> effect happens when the corresponding feature is used, so it isn't
> time-driven, it's event-driven.  For face attributes that affect
> display, that event is the next redisplay cycle, but for the
> non-visual attributes you propose the events will be entirely
> different, like buffer text modification for 'read-only'.

Right.  If it is an event such as buffer text modification
that checks text property `read-only' for the text where
the modification attempt is made, then the code that makes
that check during the modification attempt would need to
first check for a face property with a non-nil `read-only'
pseudo face attribute.  If present, it would do what it
normally does for a non-nil `read-only' text property (e.g.
raise an error).

For a property such as `keymap', the code that handles
a key sequence would need to check, first, for a face
property at point, to see if that property has a `keymap'
(pseudo) attribute, and if so, and if that key sequence
is bound in that map, do what it would do if the key
were bound in a `keymap' text property at that position.

The _code to check for the property_ would be the same,
for whatever property and wherever it might occur.  It
would be similar to what I showed before: check first
for property `face', `font-lock-face', or `mouse-face'
with a pseudo attribute that is the same as the text
property to check. If none, then go ahead and check
(as now), using code used now (presumably something
similar to or a C equivalent to `get-text-property').

The code to check for the property would be the same
for any property.  But the code that handles the
property, if found, would be different for each
property, of course.  That handling code would likely
be the same as today (no change or little change).

> Yes, I did grasp that.  But this single rationale is
> in effect a convenience feature, nothing more.

I don't mind if you put it that way.  All of Emacs is
convenience features...

This feature lets you do some things that you cannot
easily do now.  Call that convenience, if you like.

> It doesn't explain why this is
> more convenient than just putting the corresponding properties on
> those same stretches of text, nor why you'd like to avoid actually
> placing text properties to achieve the same goal.

Again, yes it does.  As I said, you need only change the
pseudo attribute value to nil in the face spec to undo the
change.  That undoes the change ONLY for the face occurrences.  
Likewise, for not undoing but changing to a different
property value (assuming several are possible).

That's the point.  The text-property fiddling is confined
to a particular face.  It affects all occurrences of the
face (or all on a given frame, optionally).

That's one advantage.  Another is that we have commands
that let users easily apply faces to text in various ways
(match regexps,... whatever).  Another is that users
already have lots of locations where faces are applied -
common and significant locations, for users.  So this
feature has plenty of terrain of immediate application.

Probably the most commonly used text property is `face'
or `font-lock-face'.  Let users take advantage of those
zones that are already mapped out, applying and changing
other text properties there.

> Nor does it explain
> why the face property is being suggested as the vehicle
> for providing this feature.

> 'Face' is just one of the many text properties we
> support, and there's nothing in it that would make it a
> better means for implementing your convenience feature.

This convenience feature is all about using the face
occurrence locations.  We have them.  We could use them,
in this way (and possibly in other ways).

> At least I don't see why 'face' is better than any other
> property.

Well, it's the only property that makes sense for acting
on face locations (duh).  It lets you take advantage of
those locations, whether they are there from font-locking
or from ad hoc highlighting.  It lets you give such ways
of highlighting the ability to indirectly control other
text properties.

Whether you use font-lock highlighting or you highlight
by dragging the mouse over text (a "marking-pen"
highlighter) or you highlight in some other way, you
will be able to leverage the highlighting to act on
other text properties in highlighted zones.

> For example, would it work
> for you if we define a new text property, whose attributes will have
> the same effect as the corresponding text properties.  Would that be
> as good as your 'face'-based proposal?  If not, why not?

See above.  The point is to be able to take advantage of
existing highlighting (text properties, not overlays).

> > > And the same will have to happen with every feature
> > > which supports some text property.
> >
> > Correct.
> >
> > > This additional handling of face attributes will
> > > have to be replicated in every such feature;
> >
> > Correct - for every such feature we decide to support.
> >
> > > there are no single place
> > > to do that for all of them, AFAIU.  This is clearly additional
> > > non-trivial work, to be repeated in many Emacs features, which is a
> > > disadvantage.  Can you tell what would be the advantages justify
> > > that these additional costs?
> >
> > I can't speak to the cost.
> 
> I didn't ask you to.  What I did ask is what are the user-level
> advantages which would justify those costs, whatever they are.  I
> don't think you answered that question.

I've described the point of the feature - the advantages.
You've hinted at the cost.  Do the advantages outweigh
the cost?  I have no idea, because I don't know the costs.
And even if I did, my or your or John Doe's opinion of
whether the advantages outweigh the costs might well be
different.

And I won't be doing the C development work, so my
opinion about the worthiness relative to the cost is
not so useful.

I can't speak to the question of relative weight because
I don't appreciate the cost.  I can't speak to matters
that involve the cost - sorry.

> Such an answer is part of the
> rationale for the proposed changes; the absence of the answer is what
> makes it hard for me to understand the rationale, which is a
> prerequisite for reasoning about it.

You say that you have gotten the point, but I'm not
sure you have.  Asking "why faces?" really makes me
wonder, at this point.

If you have gotten the point then you see the utility
of the proposal.  Weigh that against the costs you
see in whatever way you see fit.

> > In an extreme example (but I'd hope it wouldn't be so limited,
> > even at the outset), support could be provided for only one
> > or two text properties (e.g. `invisible') to begin with.
> 
> Invisible is actually the odd one out, since it _is_ implemented by
> the display engine.  It's all the rest that puzzle me.

It's no different wrt other properties in regard to the
question raised: We could decide to support only `read-only'
or only `keymap' or only this and that, or whatever.

The point there was to make clear that your assumption
that I'm demanding that _all_ text properties be supported
(all or none) is off the mark.  Quite the contrary, as
I stated from the beginning.  It is important that any
text properties that are not supported be at least
tolerated (ignored) as face attributes, just as unknown
(e.g. user-defined) text properties are ignored by Emacs.

The point is to try to (1) make Emacs ignore unknown face
(pseudo) attributes, in general (put in place the necessary
infrastructure), and then possibly (2) begin to support
active handling of one or more of them.

> > I'm outlining a user-visible feature, not designing its
> > implementation.
> 
> I'm asking you to provide the rationale for the feature,
> and for attaching this feature to faces.

I believe I've done so, several times now: Be able to
leverage face locations (occurrences) and highlighting
commands/features and face-attribute commands, to (in
effect) control other text properties at those locations.

> So far you provided only a single
> rationale, for a single use case of the feature,

Dunno what you see as the single rationale, or the
single use case.

You can easily highlight zones of text in several ways.
Many zones are highlighted by font-locking, for instance.
A user or a program can want to do things to/with such
zones - things that involve text properties.

Is that one use case or a thousand use cases?  One
rationale or many?

> I'm asking to provide more use cases and at least
> some arguments for attaching this feature to faces.

I have nothing more to offer than what I've described.
Apparently that is both (a) too restrictive ("only one")
and too general - too complicated to implement.  Sorry,
but I can't add more to describe the feature as I see it.
Or if I can, please let me know what it is that would
help understanding.

> > Sounds like you're saying that it cannot be coded.
> 
> No, I didn't say that.  I'm saying that I still don't understand why
> it would make sense to provide such a feature, and why we should
> attach it to faces.  Please help me understand your reasons.

Does what I've said above, which repeats what I've said
in pretty much each mail so far, help?  If not, I'm
afraid I can't help you see better what is involved.

> > Anyway, you clearly are not interested in the idea,
> > if you are not outright hostile to it.
> 
> At this point, I'm interested in understanding the idea.  Only then I
> will be able to make up my mind about it.  I'm not there yet.

Clearly.

To be clear, in case it makes any difference: I have
no "ulterior" motive in suggesting this idea.  It just
happened to occur to me out of the blue, and the more
I thought about it the more I thought it was interesting
and could be useful.

Others can certainly think differently about it.  I lose
nothing particular if it is not implemented.  If it is
implemented I'm pretty sure that I would use it.  I don't
know just how/where I would use it.  It would be a new
feature that I would need to discover the practical use
of, just like anyone else.

RE: Idea: Be able to use text properties as face attributes

[Drew Adams]

> > > As I explained, the display engine, which is the part that implements
> > > the faces, will be unable to do anything with attributes that don't
> > > affect display.
> >
> > Yes, you explained that.  Thanks.
> >
> > Does that mean that such code could not invoke _other_ code
> > that _would_ be able to do something with those text properties
> > (that masquerade in a face spec as face attributes)?
> 
> We can invoke (almost) any code we like, but the question is what
> would that other code do with those attributes?  Those attributes,
> like their corresponding text properties, have their effect when the
> corresponding Emacs features are invoked,

And that's the only time the face spec containing those
properties needs to be handled by that other code.

Instead of just looking for the text property that it
handles, as something that belongs to the text char
it checks, it would need to first check that char for
a face property.

And if there is a face property with that text property
embedded in it (as a pseudo face attribute) then whatever
that code would do with the text property it would do
with the text property from the face spec.

That's all.

> like buffer text
> modification for the 'read-only' property or keyboard
> input processing for the 'keymap' property.

Yes, exactly.  The code that checks for and handles a
`read-only' or a `keymap' text property would simply
check for it in two places instead of one: (1) first,
embedded in a face property, if there is one, (2) if
not, directly on the char.

> There's nothing we could do with these
> attributes during processing of faces by the display engine,

Yes, that's been clear for a while now.  You were thrown
off in that direction by my referring to "display engine"
and "redisplay" out of ignorance of where in the C code
this or that property is handled.

The point is that _wherever_ a given text property is
handled in the code, that code would need to take a
larger view to check for the property.  It would not
look only for the property directly on the text; it
would look also (and first) for that property as
embedded in a face property (`face', `font-lock-face',
or  `mouse-face').

> Text properties are precisely a way to record such
> information, but you don't want to use text properties in this case.

I want to use a _face_ text property to record them.
So yes, I do want to use a text property: one of
the properties `face', `font-lock-face', and 
`mouse-face'.  And I want to (be able to) record
the values of other text properties _within the
value of the face property_.

> So some other means of recording the information will
> have to be invented.

No.  Not in my view.  They would be recorded in a face
property.  No other recording needed.

> But since those means will most probably be very similar to
> text properties, at least implementation-wise, what would be the point
> of inventing that?

No such invention is needed, in my view.

> > I think by now you should understand why I proposed the
> > feature - its use.  I've made that pretty clear.
> >
> > You are free to think that it's a useless feature, of course,
> > and you are free to think that it might be useful but is not
> > feasible to implement.
> 
> Sorry, I don't understand.  My questions and my confusion are real, so
> please bear with me and don't assume I'm repeatedly asking the same
> questions for argument's sake.  I'm asking them because I really don't
> understand your motivation for proposing this feature in the form that
> you proposed it.  It's your proposal, so only you can explain its
> details.

OK, glad to hear that.  Sometimes it's not clear.  Do you
understand now?  If not, do you think there is something
else I can add that would make things clearer?

The motivation is to be able to leverage face-occurrence
locations to manipulate (indirectly) text properties at
those locations.

We have functions - even user commands - that you can use
to change the attributes of a face, and doing that affects
all occurrences of the face (or all on a given frame,
optionally).

We can leverage these features to let users and code
(in effect) manipulate other text properties at whole
sets of locations at once.  Which locations?  Those of
a given face.

And we have umpteen ways to apply a face to different
locations - again, either by Lisp or user commands.

So users will be able to:

1. Put a face wherever they want (they can do that now).

2. Manipulate any text properties (that are supported)
   at those locations.

Please let me know if this is becoming any clearer
to you.  Neither of us wants to waste the time of the
other (and others), if there is no real communication
going on.

RE: Idea: Be able to use text properties as face attributes

[Drew Adams]

> > for the 'keymap' property.  There's nothing we could do with these
> > attributes during processing of faces by the display engine, except
> > record those attributes somewhere, for future use by their respective
> > features.
> 
> It's worse: the rest of the code can't assume that the display rendering
> has propagated the info from the fact attributes to that
> "to-be-defined-other-place" since redisplay may not have happened yet or
> may not have looked at that part of the buffer yet.

Please reread what I wrote.  I think there has been a
misunderstanding.  This feature has nothing to do with
the display engine per se.  It is only about leveraging
a face's spec to manipulate other text properties at
locations where the face is used.

For that, we would extend face specs to tolerate (ignore)
unknown face attributes (call them pseudo face attributes,
if you like).  And we would support one or more such
attributes that are text-property symbols and their values.

The code that does the supporting (interpreting) of such a
(pseudo) face attribute-that-is-a-text-property is, it is
clear by now (but it was not clear to me initially),
typically _not_ in the display engine.  I think we can
remove "display engine" from the discussion.  I'm sorry
that I (naively) mentioned it when trying to make clear
that _some C code_ would be implicated.

Wherever that code is, it would check for the text
property that it handles (1) first, by looking for a
face property and checking for the text property as one
of the face property's (pseudo) attributes, and (2)
second, IF #1 did not yield the text property, looking
for it directly on the text, as it does today.

The only thing new would be #1: trying to get the text
property first from a face property.  That lets a face
spec override text properties that might already be
applied to the text directly, and it lets a face spec
provide text props that are not yet applied (the general
effect would be as if they had been applied directly).

> This said, Drew's feature doesn't have to require changes all over
> the place.  It might be possible to limit those changes to a few core
> place like Fget_text_property.

I can't speak to where or what code changes would be needed.

I do think, however, that the behavior of `get-text-property'
(and perhaps other functions) should _not_ be changed. 

Instead we should add another function that gives Lisp code
a way to do the checking described above: first, try to get
the text property from within a face property, and then, if
not obtained, use `get-text-property' to try to get it
(directly) - as is the case now.

> This said, I think using face attributes is a bad idea.

Reason?  I can be convinced, but only by reasons.

> Instead, we should consider adding some more general system of
> indirection.  Drew suggests using `face` as a special meta-property.

Yes, I suggest using `face', `font-lock-face', and
`mouse-face' text properties to indirectly specify
other text properties.

I chose face properties not by accident, but because the
feature would let you leverage both (1) existing face
locations (occurrences), and (2) existing commands and
other functions that let users and Lisp easily put a face
property on text in various ways, or change it.

Faces are special in those senses - they constitute
especially handy affordances for manipulating text at
locations that are common or significant for users.

> Current code already supports such a feature but using the name
> `category` and only for overlays (and with some problems linked to the
> fact that modifying that meta-property is done via `get` and `put` which
> are very generic operations used for many other things, so it's
> hard/impossible to detect when those meta-properties are changed, which
> is a fairly serious problem.  The only reason why we don't suffer too
> much from that problem is because those `category` properties aren't
> used very often).
> 
> My proposal for "property planes" is somewhat related to Drew's
> proposal: my "property planes" allow merging various properties into one
> (font-lock can place the (font-lock face) property, and some other
> package can add the (mypkg face) property and they then get merged into
> the one-and-only `face` property), i.e. many-to-one.  Drew wants the
> reverse: adding a single meta-property which then affects several
> other properties, i.e. one-to-many.
> 
> From a more general point of view, the shared desire is to have
> "abstract text properties" from which "actual text properties"
> get computed.

I'd be more interested in your proposal if I had a
better idea of it.  But I suspect that it might be
independent of what I am getting at.

It's true that I'm asking for a way to handle one
or more text properties as a package at different
locations.  But my proposal is purposely aimed at
FACES and where they are applied, for the reasons
given above.  It's not _only_ about packaging text
props together to act on them in an abstract way.

I fear I'm not getting the point across well, but
I'm willing to keep trying if there is a real desire
to understand.

I fear that combining discussion of your proposal
and mine in the same thread will not be helpful -
unless it is a later thread, which can take advantage
of conclusions already drawn about my proposal (e.g.
yay or nay; specific challenges identified, whatever).

Re: Idea: Be able to use text properties as face attributes

[Stefan Monnier]

>> This said, I think using face attributes is a bad idea.
> Reason?  I can be convinced, but only by reasons.

Faces (and face attributes) are handled by the display engine, which is
a delicate and performance sensitive piece of code.  Messing with that
is tricky and can easily impose serious performance problems.

> locations.  But my proposal is purposely aimed at
> FACES and where they are applied, for the reasons
> given above.  It's not _only_ about packaging text
> props together to act on them in an abstract way.

I'd rather use another property, and then make font-lock use that new
property instead.


        Stefan

RE: Idea: Be able to use text properties as face attributes

[Drew Adams]

> >> This said, I think using face attributes is a bad idea.
> > Reason?  I can be convinced, but only by reasons.
> 
> Faces (and face attributes) are handled by the display engine, which is
> a delicate and performance sensitive piece of code.  Messing with that
> is tricky and can easily impose serious performance problems.

OK.  But if I understand correctly (no indication of
that, I realize ;-)), the only thing the display engine
would need to do differently wrt faces and face attributes
would be to tolerate (ignore) any that it does not recognize
(dunno what it does now - perhaps it raises an error?).

Yes, there is also Customize, which recognizes only the
current face attributes.

Customize could be left as is, which would mean that you
could not use it to assign text-properties as (pseudo)
attributes.

Or we could decide to support unknown face attributes
in Customize in some way (i.e., let you enter them).

I'm not including that as part of the feature proposal.
Customization of a face is not where I see this feature.
I see it in Lisp code or interactive use of `set-face-*'.

> > locations.  But my proposal is purposely aimed at
> > FACES and where they are applied, for the reasons
> > given above.  It's not _only_ about packaging text
> > props together to act on them in an abstract way.
> 
> I'd rather use another property, and then make
> font-lock use that new property instead.

The devil may be in the details - please elaborate.

But a priori I don't see how that would provide the
feature I described, which needs to work, out-of-the-box,
with faces you apply (e.g., by highlighting, and _not_
just by font-lock highlighting), as well as with
`set-face-*' commands etc.

Re: Idea: Be able to use text properties as face attributes

[Stefan Monnier]

> just by font-lock highlighting), as well as with
> `set-face-*' commands etc.

The set-face-* functions are .. fundamentally flawed given the desire to
make faces work with face-specs.  We make it work, but mostly because
it's not used very much.


        Stefan

RE: Idea: Be able to use text properties as face attributes

[Drew Adams]

> > just by font-lock highlighting), as well as with
> > `set-face-*' commands etc.
> 
> The set-face-* functions are .. fundamentally flawed given the desire to
> make faces work with face-specs.  We make it work, but mostly because
> it's not used very much.

You give no explanation, so that enigmatic paragraph
doesn't mean much to me.  I don't doubt that there is
something behind your statement, but you seem to have
a habit of giving such pronouncements without any
explanation or reasons. ;-)

If there is a problem with some existing commands,
perhaps it should be fixed.

It is useful for a user or Lisp code to be able to
change a face definition on the fly.

----

We have, BTW, the various `facemenu-set-*' commands,
which set a given text property (whether `face',
`intangible', `invisible', `read-only', etc.) for
the region.

The feature I proposed provides a similar feature,
but more generally, and for all of the zones
inhabited by a given face, instead of just for the
region.

Re: Idea: Be able to use text properties as face attributes

[Eli Zaretskii]

> Date: Tue, 28 Mar 2017 14:39:36 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: emacs-devel@gnu.org
> 
> That's the point.  The text-property fiddling is confined
> to a particular face.  It affects all occurrences of the
> face (or all on a given frame, optionally).
> 
> That's one advantage.  Another is that we have commands
> that let users easily apply faces to text in various ways
> (match regexps,... whatever).  Another is that users
> already have lots of locations where faces are applied -
> common and significant locations, for users.  So this
> feature has plenty of terrain of immediate application.

Why would I want to make all stretches of text that have, say,
font-lock-constant-face have a particular 'keymap' attribute?  And how
is that more convenient than simply scanning the text for the same
regexp/syntax as the one which used to place the face and actually
putting a 'keymap' property there?  And why do you only ask this for
the 'face' properties, but not for others?

IOW, I'm still struggling to understand why you attach this feature to
faces, and to faces only.

Please also note that at least font-lock faces are by default placed
lazily, on and around the portions of the buffer that are going to be
displayed.  So Lisp programs looking for your special attributes might
not find them if JIT font-lock didn't yet fontify them.

> > > I can't speak to the cost.
> > 
> > I didn't ask you to.  What I did ask is what are the user-level
> > advantages which would justify those costs, whatever they are.  I
> > don't think you answered that question.
> 
> I've described the point of the feature - the advantages.

Am I correct concluding that the advantages, from your POV, are that
users of this feature will be able to avoid the need of specifying the
text to receive this special treatment with regexps, syntax classes,
and other similar means, and instead will be able to use the
definitions already provided for faces?  Is this the only advantage,
or are there others?

> You've hinted at the cost.  Do the advantages outweigh
> the cost?  I have no idea, because I don't know the costs.

I didn't ask you to weigh the costs against the advantages, I asked
only about the advantages.

> So users will be able to:
> 
> 1. Put a face wherever they want (they can do that now).
> 
> 2. Manipulate any text properties (that are supported)
>    at those locations.

Why is it better than just put any property you need, such as
'read-only', directly on that same text?  What non-trivial parts of
the job would be saved by using face attributes instead?

> Faces are special in those senses - they constitute
> especially handy affordances for manipulating text at
> locations that are common or significant for users.

If this is the answer to my question above, then I'm not sure I agree
with you.  E.g., Text mode has no faces by default, but that doesn't
mean Lisp programs working in Text mode won't want to manipulate text
at significant locations.

Re: Idea: Be able to use text properties as face attributes

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> IOW, I'm still struggling to understand why you attach this feature to
> faces, and to faces only.

I may well be misunderstanding what Drew is asking for, because I didn't
understand it the first four times he explained it, either.  But I think
he's asking for something that works like a face, but includes all text
properties.

That is:  He wants to say

(defprop my-prop '(:read-only t :invisible nil :font "futura"))
(set-text-prop (point-min) (+ (point-min) 4) my-prop)
(set-text-prop (+ (point-min) 10) (+ (point-min) 20) my-prop)

and then

(set-prop my-prop :invisible t)

and then these two ranges would suddenly become invisible.  So it's
like...  a CSS selector...  kinda.

The thing is, I think you can just cheat and this thing is already
available?  That is, just make my-prop a list, apply that list to the
regions you are interested in, and then just mutate that list instead of
using the property functions?  Or does set-text-properties copy
PROPERTIES before putting them on the text?  I have neither tested nor
read the source code.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Idea: Be able to use text properties as face attributes

[Clément Pit-Claudel]

On 2017-03-29 11:21, Lars Ingebrigtsen wrote:
> That is:  He wants to say
> 
> (defprop my-prop '(:read-only t :invisible nil :font "futura"))
> (set-text-prop (point-min) (+ (point-min) 4) my-prop)
> (set-text-prop (+ (point-min) 10) (+ (point-min) 20) my-prop)

I think that's what I proposed in response to Drew's original message, and what Stefan suggests (it's a natural extension of overlay categories).  But I'm not sure it's what Drew had in mind.

Specifically, Drew (IIUC) would like *faces* to behave as these hypothetical 'defprop's.  This way, all the existing machinery on faces could be reused.

Clément.

Re: Idea: Be able to use text properties as face attributes

[Andreas Politz]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Or does set-text-properties copy PROPERTIES before putting them on the
> text?  I have neither tested nor read the source code.

emacs/src/textprop.c:set_properties:

  /* Store new properties.  */
  set_interval_plist (interval, Fcopy_sequence (properties));

-ap

RE: Idea: Be able to use text properties as face attributes

[Drew Adams]

> > That's the point.  The text-property fiddling is confined
> > to a particular face.  It affects all occurrences of the
> > face (or all on a given frame, optionally).
> >
> > That's one advantage.  Another is that we have commands
> > that let users easily apply faces to text in various ways
> > (match regexps,... whatever).  Another is that users
> > already have lots of locations where faces are applied -
> > common and significant locations, for users.  So this
> > feature has plenty of terrain of immediate application.
> 
> Why would I want to make all stretches of text that have, say,
> font-lock-constant-face have a particular 'keymap' attribute?

Dunno why you would, Eli.  Maybe you wouldn't.
I probably wouldn't (why would I?).

But you might want to use font-lock to place a
`keymap' property on particular zones of text,
defined syntactically. IOW, you might want to
take advantage of font-lock's elaborate
specification possibilities to locate places to
put a given `keymap' property.

Yes, you can also do that now, by giving a spec
to font-lock  that contains the text properties,
and adding those properties to
`font-lock-extra-managed-props'.

The proposed feature provides more flexibility
for that kind of thing.  You can change a face's
spec anytime, to change the effects for only
_that face's_ occurrences.

A face need not show any highlighting, even now
(its attributes could just reflect the default
face).  With the proposed feature a face could
certainly provide _only_ non-face text properties,
if that makes sense in a given context.

E.g., a given font-lock spec could have as its
purpose to _only_ place a given `keymap' (or
`syntax-table' or whatever) property at particular
locations, defined syntactically.

(Again, you could get that effect now, using
`font-lock-extra-managed-props', but without the
flexibility just mentioned.)

And beyond font-locking, there are lots of other
ways to "highlight" text with a given face.

And for such ad hoc highlighting you do not
necessarily want the highlighting (including
application of other text props) to go away just
because you turn off font-lock mode.

Highlighting is more general than just font-locking.

Since with the proposed feature a face can be a
carrier of text properties, the notion of
"highlighting" using a face takes on more meaning.

Highlighting in the usual sense might be one of the
effects - or not.  Text properties, besides just
those affecting appearance, can now be part of any
"highlighting".

When I said "users already have lots of locations
where faces are applied - common and significant
locations" I wasn't referring only to using
locations that were chosen for text highlighting
in the ordinary sense.

I mean both that those face occurrences _can_ be
taken advantage of and that _non-highlighting
occurrences_ of a face can also exist.  The meaning
of "places where faces are applied" is expanded.

> And how is that more convenient than simply scanning the text
> for the same regexp/syntax as the one which used to place the
> face and actually putting a 'keymap' property there?

Do you ever use font-lock?  How is it more
convenient for you that explicit code that
searches for each regexp/syntax construct and
puts the relevant face on it?  Clearly it is, no?

Likewise, for the many other ways of highlighting
text.  There are convenient ways to highlight
text, but you aren't forced to use them.  If you
prefer to always "simply" scan the text and apply
a face, you are free to do that instead. 

> And why do you only ask this for the 'face'
> properties, but not for others?

I've answered that several times now.  It's hard
to believe that you are really trying to get the
message.

If you cannot see the convenience of being able
to apply text properties just by using existing
means of highlighting (and not just font-locking)
then I don't think there's a lot more I can do
to convince you. 

> IOW, I'm still struggling to understand why you
> attach this feature to faces, and to faces only.

Yeah, you keep asking me that.  And I keep telling
you my reasons, in different ways - but the reasons
remain the same.

Do you see something besides faces that offers the
same convenience for applying properties to text?
I proposed leveraging faces, but if you see another
potential affordance for doing that, maybe we can
consider that one too.

> Please also note that at least font-lock faces are by
> default placed lazily, on and around the portions of
> the buffer that are going to be displayed.  So Lisp
> programs looking for your special attributes might not
> find them if JIT font-lock didn't yet fontify them.

Yes, by default font-locking is lazy.  That would of
course apply also to the use of a face whose spec
contains text-property (pseudo) face attributes, as
well as to any other face.

And just as you see text fontified when it becomes
exposed in the window, so would the text props in
its face be realized as the text becomes exposed
in the window.  You cannot use a property such as
`keymap' without moving point to the text that has
that property, in which case the text is also shown
in the window.

If there are any text props (that we decide to support
for this feature) for which lazy realization is not
good enough then they wouldn't be good candidates for
use by font-lock (unless font-lock is used by the user
eagerly instead of lazily).

> Am I correct concluding that the advantages, from your POV,
> are that users of this feature will be able to avoid the
> need of specifying the text to receive this special
> treatment with regexps, syntax classes, and other similar
> means, and instead will be able to use the definitions
> already provided for faces?

By "this special treatment" are you referring to
(effectively) applying non-face text props?  And is
"the text to receive" that treatment the text where
faces are applied (in my proposal)?

If so, then yes: Leverage the many convenient ways
we already have to apply a face to zones of text.

(Not sure what you mean there by "the definitions
already provided for faces."  Face definitions don't
specify where the faces get applied.)

> Is this the only advantage, or are there others?

Is it one advantage or two (or several)?  I said:

  I chose face properties not by accident, but because the
  feature would let you leverage both (1) existing face
  locations (occurrences), and (2) existing commands and
  other functions that let users and Lisp easily put a face
  property on text in various ways, or change it.

  Faces are special in those senses - they constitute
  especially handy affordances for manipulating text at
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  locations that are common or significant for users.

It's about (indirectly) applying text props and
manipulating applied text props, in convenient ways.

Those are the advantages I see, for why we would turn
faces into carriers of arbitrary text properties (at
least those we decide to support).

> > So users will be able to:
> >
> > 1. Put a face wherever they want (they can do that now).
> > 2. Manipulate any text properties (that are supported)
> >    at those locations.
> 
> Why is it better than just put any property you need,
> such as 'read-only', directly on that same text?  What 
>non-trivial parts of the job would be saved by using
> face attributes instead?

See above and my other replies.  Faces are convenient.
There is a lot more provided to users for highlighting
(i.e., using faces) than there is for placing arbitrary
text properties.

And by encapsulating text props within a face we can
easily manipulate different bunches of them individually.

They are not just all `font-lock-extra-managed-props',
i.e., handled all the same, together, wherever they are
used.

> > Faces are special in those senses - they constitute
> > especially handy affordances for manipulating text at
> > locations that are common or significant for users.
> 
> If this is the answer to my question above,

It's part of the answer.  Look above, and in previous
replies, for more.

> then I'm not sure I agree
> with you.  E.g., Text mode has no faces by default, but
> that doesn't mean Lisp programs working in Text mode
> won't want to manipulate text at significant locations.

1. I didn't say that this feature would or should be
the only way for Lisp programs to manipulate text.

2. Wrt both Lisp and interactive use: Who cares what
Text mode has by default?  Please try to get beyond
font-lock and especially beyond default font-locking.

I can highlight text in Text mode in many ways helpful
and meaningful to whatever current use I'm making of
Text mode.  That includes but is not limited to
non-default font-locking.  I use ad hoc highlighting
all the time (with text props and with overlays).

Sure, maybe some users don't, and maybe some users
never fiddle with `font-lock-keywords'.  No one would
be forced to use this feature either.

RE: Idea: Be able to use text properties as face attributes

[Drew Adams]

> > IOW, I'm still struggling to understand why you attach this feature to
> > faces, and to faces only.
> 
> I may well be misunderstanding what Drew is asking for, because I didn't
> understand it the first four times he explained it, either.  But I think
> he's asking for something that works like a face, but includes all text
> properties.

Yes.  "Works like a face" is a good way to put it.

It is what I meant by leveraging existing, convenient ways
of applying (and unapplying), and specifying, faces.

Wrt "includes all text properties":

1. At least _tolerates_ all of them, ignoring any that w
decide not to teach Emacs to handle.

2. Includes _within a given face spec_.  The face definition
is used as an object that encapsulates a certain number of
non-face text properties.

Face A might use one value of text property `keymap', and
face B might use another.  In neither case would property
`keymap' be applied directly to the underlying text where
the face is applied.  Only the face property is applied
to the text (for this feature).

This lets you remove or change property `keymap' for all
occurrences of a given face.  And it lets Lisp code
distinguish properties that are attached directly to
the text and are thus not dependent on any face from
properties that are applied indirectly by way of the
face.

> That is:  He wants to say
> (defprop my-prop '(:read-only t :invisible nil :font "futura"))
> (set-text-prop (point-min) (+ (point-min) 4) my-prop)
> (set-text-prop (+ (point-min) 10) (+ (point-min) 20) my-prop)
> and then
> (set-prop my-prop :invisible t)

Nope.  That doesn't use faces at all.  It just attaches the
properties to the text directly.  It in no way lets you
leverage the features we have for applying faces, and it in
no way bundles the properties within a face spec, so that
changing the face changes (in effect) the properties of the
text where the face is applied.

> and then these two ranges would suddenly become invisible.
> So it's like...  a CSS selector...  kinda.
> 
> The thing is, I think you can just cheat and this thing is already
> available?  That is, just make my-prop a list, apply that list to the
> regions you are interested in, and then just mutate that list instead of
> using the property functions?  Or does set-text-properties copy
> PROPERTIES before putting them on the text?  I have neither tested nor
> read the source code.

See above.  Yes, you can already apply text properties
directly to text.  That's not what this feature is about.

And it's not only about being able to change a set of
props that is applied at various locations (though yes,
there is that effect from the proposal).  It's also
about taking advantage of face features, in particular,
convenient ways to apply them.

If you look at the fact that Emacs at one point added
to font-lock the ability to "manage" (to some extent)
other, non-face text properties, this feature is in
a similar vein.

Why did we add that extension to non-face props?
Because font-lock provides a _convenient way to
specify how and where to apply_ a property.

Imagine that!  Yep, font-lock applying a non-face
property is "just a convenience".  And a very
convenient one, too. ;-)  Do most users use it?
Probably not.  That doesn't mean we shouldn't
have added it.

RE: Idea: Be able to use text properties as face attributes

[Drew Adams]

> > (defprop my-prop '(:read-only t :invisible nil :font "futura"))
> > (set-text-prop (point-min) (+ (point-min) 4) my-prop)
> > (set-text-prop (+ (point-min) 10) (+ (point-min) 20) my-prop)
> 
> I think that's what I proposed in response to Drew's original message, and
> what Stefan suggests (it's a natural extension of overlay categories).
> But I'm not sure it's what Drew had in mind.
> 
> Specifically, Drew (IIUC) would like *faces* to behave as these
> hypothetical 'defprop's.  This way, all the existing machinery
> on faces could be reused.

Yup.  Reuse all of the "existing machinery" provided for faces.

_And_ (as in Lars's example above), manipulate named bundles
of props.

Those two things combined: (1) use _faces_ (2) as the objects
that _encapsulate a bundle of properties_.