Combining face and map stuff

Combining face and map stuff

[Lars Magne Ingebrigtsen]

It would often be convenient to say "this region should be in bold" and
"this partially overlapping region should also be italic", and so on.
I've been looking in the Emacs Lisp manual, but I can't find any utility
functions for doing stuff like that.  They all seem to start with a face
with some properties that you apply to a text, which isn't really what I
want.  So I would kinda of like...  a function that took a face, and
then "combined" it with any faces that were already in the region.
Possible?

Which reminds me of another thing.  I would like to define some commands
local to a region, which can be done with 'local-map in overlays or
'keymap in text properties.  But can I combine them, too, in possibly
overlapping regions, and get the aggregate keymap?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Combining face and map stuff

[Chong Yidong]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> It would often be convenient to say "this region should be in bold" and
> "this partially overlapping region should also be italic", and so on.
> I've been looking in the Emacs Lisp manual, but I can't find any utility
> functions for doing stuff like that.  They all seem to start with a face
> with some properties that you apply to a text, which isn't really what I
> want.  So I would kinda of like...  a function that took a face, and
> then "combined" it with any faces that were already in the region.

Easiest way is probably to use overlay faces.

Re: Combining face and map stuff

[Lars Magne Ingebrigtsen]

Chong Yidong <cyd@stupidchicken.com> writes:

> Easiest way is probably to use overlay faces.

Oh, combining faces with overlays worked fine.

But combining keymaps with overlays doesn't work?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Combining face and map stuff

[Miles Bader]

You can also have face properties that are lists of
faces/face-attributes, and add or remove stuff from these.

Some example functions below, `add-to-face-property' and
`remove-from-face-property', with which you can do stuff like:

  (add-to-face-property 'variable-pitch START END)
  (add-to-face-property '(:weight bold) (+ START 5) (- END 10))
  ...
  (remove-from-face-property 'variable-pitch START END)
  ...


(defun single-face-p (face)
  "Return non-nil if FACE seems to be a single face or face-attribute list
\(as oppposed to a list of faces/face-attribute-lists)."
  (or (symbolp face)			; face-symbol
      (and (listp face)
	   (keywordp (car face))	; (:attr val ...)
	   (keywordp (cadr face)))))	; (face-symbol :attr val ...)

(defun add-to-face-property (face start end)
  "Add FACE to the face text-property of the region from START to END.
Any existing face text-properties are preserved by adding FACE to
the beginning of a list, making the existin value into a list if
necessary first.  It can be removed with `remove-from-face-property'."
  (while (< start end)
    (let ((next (next-char-property-change start end)))
      (let ((cur (get-char-property start 'face)))
        (when (single-face-p cur)
	  (setq cur (list cur)))
        (push face cur)
	(put-text-property start next 'face cur))
      (setq start next))))

(defun remove-from-face-property (face start end)
  "Remove FACE from the face text-property of the region from START to END.
The assumption is that it was added previously by `add-to-face-property'."
  (while (< start end)
    (let ((next (next-char-property-change start end)))
      (let ((cur (get-char-property start 'face)))
	(when (single-face-p cur)
	  (setq cur (list cur)))
	(when (member face cur)
	  (put-text-property start next 'face (remove face cur))))
      (setq start next))))


-miles

-- 
History, n. An account mostly false, of events mostly unimportant, which are
brought about by rulers mostly knaves, and soldiers mostly fools.

Re: Combining face and map stuff

[Chong Yidong]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Chong Yidong <cyd@stupidchicken.com> writes:
>
>> Easiest way is probably to use overlay faces.
>
> Oh, combining faces with overlays worked fine.
>
> But combining keymaps with overlays doesn't work?

The face code goes to some lengths to handle inheritance, so that all
the face properties from text properties and all overlays are combined
into the final displayed result.  The way keymaps are handled is a
little simpler: the highest-priority overlay with a keymap property (or,
failing that an underlying keymap text property) is used for the local
map.

In the past, it's been pretty painful making sure face inheritance works
just right.  So I can't say I'm enthusiastic about implementing anything
similar for local keymaps...

Re: Combining face and map stuff

[Lars Magne Ingebrigtsen]

Chong Yidong <cyd@stupidchicken.com> writes:

> In the past, it's been pretty painful making sure face inheritance works
> just right.  So I can't say I'm enthusiastic about implementing anything
> similar for local keymaps...

No, it looks quite hairy.  I was just wondering whether I was
overlooking something here (like I did with combining face overlays),
but it looks like I'm not, so I'll just implement this in a different
way for the HTML renderer.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Combining face and map stuff

[Stefan Monnier]

> Which reminds me of another thing.  I would like to define some commands
> local to a region, which can be done with 'local-map in overlays or
> 'keymap in text properties.  But can I combine them, too, in possibly
> overlapping regions, and get the aggregate keymap?

Here's an idea with which I've beeing toying:

- rather than have just one set of text-properties per char, make it
  possible to have several.  E.g. font-lock would use its own set of
  text properties.  One way to do that is that a `property' can now be
  a cons whose car is a "plane" and whose cdr is a property.
  So for example, the special font-lock-face (which currently gets
  "mapped" to face' via char-property-alias-alist) would just be `face'
  because font-lock would use (font-lock . face).
  A property like `face' would really be equivalent to (nil . face).

- then you add property-specific merge functions.  I.e. when looking up
  `face' you'd get the merge of all the `face' properties of the various
  text-property planes.  You'd probably want those merge functions to be
  written in Elisp and customizable.  So merging `keymap' properties
  becomes easy (well, it may benefit from multiple-keymap inheritance,
  which is a completely different topic).

- now font-lock can erase all the properties of the `font-lock' plane
  without worrying about erasing properties installed by other packages.

- to avoid calling Elisp code to merge things during text-property
  lookups, the merge would take place in put-text-property (i.e. no need
  to change the redisplay code at all).

- modifying a plane other than the default (nil) one could be considered
  as "not modifying the buffer" (just like adding/removing overlays).

- we could obsolete char-property-alias-alist which is only ever used
  by/for font-lock-face.

- maybe outline-minor-mode could use text-properties rather than
  overlays to make text invisible (tho it would make it impossible(?) to
  use outline-minor-mode in an indirect buffer without affecting the
  base buffer and the other indirect buffers).  So it might reduce the
  need for overlays (which have the disadvantage of being
  algorithmically slow/costly).


        Stefan

Re: Combining face and map stuff

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Mon, 04 Oct 2010 01:37:26 +0200
> 
> - rather than have just one set of text-properties per char, make it
>   possible to have several.  E.g. font-lock would use its own set of
>   text properties.  One way to do that is that a `property' can now be
>   a cons whose car is a "plane" and whose cdr is a property.

How would the display engine know which "plane" to use for rendering
each character?  Or are you suggesting to hard-code a fixed list of
"planes" in the display code?

Re: Combining face and map stuff

[Miles Bader]

Eli Zaretskii <eliz@gnu.org> writes:
> How would the display engine know which "plane" to use for rendering
> each character?  Or are you suggesting to hard-code a fixed list of
> "planes" in the display code?

It sounds like he's saying it would use their union, with a
property-specific function used to compute that union.

[and that the union computation could be done at "property setting time"
rather than at display time, to avoid speed problems.]

-miles

-- 
Future, n. That period of time in which our affairs prosper, our friends
are true and our happiness is assured.

Re: Combining face and map stuff

[Eli Zaretskii]

> From: Miles Bader <miles@gnu.org>
> Date: Mon, 04 Oct 2010 18:03:04 +0900
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel@gnu.org
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> > How would the display engine know which "plane" to use for rendering
> > each character?  Or are you suggesting to hard-code a fixed list of
> > "planes" in the display code?
> 
> It sounds like he's saying it would use their union, with a
> property-specific function used to compute that union.
> 
> [and that the union computation could be done at "property setting time"
> rather than at display time, to avoid speed problems.]

I'm probably missing something here: if the union is computed at
put-text-property time, then how is this different from what we have
now?  That union will have all the properties of the character lumped
together, just like what we have now, right?  What am I missing?

Re: Combining face and map stuff

[Miles Bader]

Eli Zaretskii <eliz@gnu.org> writes:
>> It sounds like he's saying it would use their union, with a
>> property-specific function used to compute that union.
>> 
>> [and that the union computation could be done at "property setting time"
>> rather than at display time, to avoid speed problems.]
>
> I'm probably missing something here: if the union is computed at
> put-text-property time, then how is this different from what we have
> now?  That union will have all the properties of the character lumped
> together, just like what we have now, right?  What am I missing?

Who knows, maybe it _could_ be implemented completely in lisp given
todays primitives ... but in order to make the layering thing work, all
the users would need to cooperate, presumably by means of a wrapper
layer.

(i.e., rename current setting primitives foo to "internal-foo", and
redefine foo in lisp to do the layering/merging stuff)

Anyway, I'm just guessing ... Stefan should answer :)

-miles

-- 
Carefully crafted initial estimates reward you not only with
reduced computational effort, but also with understanding and
increased self-esteem.         -- Numerical methods in C,
  Chapter 9. "Root Finding and Nonlinear Sets of Equations"

Re: Combining face and map stuff

[Stefan Monnier]

>>> It sounds like he's saying it would use their union, with a
>>> property-specific function used to compute that union.
>>> [and that the union computation could be done at "property setting time"
>>> rather than at display time, to avoid speed problems.]
>> I'm probably missing something here: if the union is computed at
>> put-text-property time, then how is this different from what we have
>> now?  That union will have all the properties of the character lumped
>> together, just like what we have now, right?  What am I missing?

You're missing that in order o properly update the merge when some
property is added/removed in a plane, all users of text properties will
have to follow some new conventions.

Of course, those conventions could be enforced via defadvices, I guess.

But there's probably going to be some slightly subtle issues: when
setting the `face' property, we can assume it's in the default nil
plane, but reading the `face' property might mean "the `face' property
in the nil plane" or "the merged `face' property".

> Who knows, maybe it _could_ be implemented completely in lisp given
> todays primitives ... but in order to make the layering thing work,
> all the users would need to cooperate, presumably by means of
> a wrapper layer.

Right.
Actually, that's why I like this idea: most of the work is done at
a time when Elisp can be used, so you get a lot of flexibility, and you
don't have to worry about interaction with tricky C code.
E.g. maybe the merge function could be used to enforce priorities
between planes, or to enable/disable planes (kind of like
add-to-invisibility-spec).


        Stefan

Re: Combining face and map stuff

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>,  emacs-devel@gnu.org
> Date: Wed, 06 Oct 2010 01:43:03 +0200
> 
> >>> It sounds like he's saying it would use their union, with a
> >>> property-specific function used to compute that union.
> >>> [and that the union computation could be done at "property setting time"
> >>> rather than at display time, to avoid speed problems.]
> >> I'm probably missing something here: if the union is computed at
> >> put-text-property time, then how is this different from what we have
> >> now?  That union will have all the properties of the character lumped
> >> together, just like what we have now, right?  What am I missing?
> 
> You're missing that in order o properly update the merge when some
> property is added/removed in a plane, all users of text properties will
> have to follow some new conventions.

When I said "what am I missing", I meant the advantages of what you
propose.  Having to follow some new conventions sounds like the price
one has to pay for having these advantages, but what are the
advantages themselves?

If the new protocol itself is the advantage, then perhaps what I'm
missing is the essence of the problems that you proposal is trying to
solve.

Re: Combining face and map stuff

[Stefan Monnier]

> If the new protocol itself is the advantage, then perhaps what I'm
> missing is the essence of the problems that you proposal is trying to
> solve.

Here are some problems that this protocol solves:

- there can currently only be one owner of a text-property, which
  creates conflicts.  We have various workarounds in use for that:
  - use overlays instead of text-properties.
  - use char-property-alias-alist.
  - add hooks in one of the packages and let the other work through
    those hooks (e.g. hi-lock using font-lock).
  - add a secondary property to try and record who put the property, so
    as to be able to recognize one's own properties when time comes to
    remove them.

- there is no way currently to combine various property values (the OP's
  question).


        Stefan

Re: Combining face and map stuff

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: miles@gnu.org,  emacs-devel@gnu.org
> Date: Thu, 07 Oct 2010 09:40:33 +0200
> 
> Here are some problems that this protocol solves:
> 
> - there can currently only be one owner of a text-property, which
>   creates conflicts.

What do you mean by "owner" in this context?

> - there is no way currently to combine various property values (the OP's
>   question).

What's wrong with add-text-properties?

Re: Combining face and map stuff

[Stefan Monnier]

>> Here are some problems that this protocol solves:
>> - there can currently only be one owner of a text-property, which
>> creates conflicts.
> What do you mean by "owner" in this context?

Package foo wants to add `face' stuff and package bar also wants to add
`face' stuff.  If both add it to the same piece of text, we have
a problem, and if they want to update it later they have to know who
"owns" the particular value (i.e. who it comes from) to know whether it
should be updated or not.

>> - there is no way currently to combine various property values (the OP's
>> question).
> What's wrong with add-text-properties?

It never combines values of a single property, just overwrites an old
value with a new one.


        Stefan

Re: Combining face and map stuff

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: miles@gnu.org,  emacs-devel@gnu.org
> Date: Thu, 07 Oct 2010 20:08:26 +0200
> 
> Package foo wants to add `face' stuff and package bar also wants to add
> `face' stuff.  If both add it to the same piece of text, we have
> a problem, and if they want to update it later they have to know who
> "owns" the particular value (i.e. who it comes from) to know whether it
> should be updated or not.

In that case, I'm sorry, but I must ask the same question again: how
will the display engine know which face to display?

Let's say we have a buffer, in which both package foo and package bar
defined different faces for the same piece of text.  Which one of them
will be displayed, and how would the display engine know to choose it?

> >> - there is no way currently to combine various property values (the OP's
> >> question).
> > What's wrong with add-text-properties?
> 
> It never combines values of a single property, just overwrites an old
> value with a new one.

Light begins slowly to turn on...

Re: Combining face and map stuff

[Stefan Monnier]

>> Package foo wants to add `face' stuff and package bar also wants to add
>> `face' stuff.  If both add it to the same piece of text, we have
>> a problem, and if they want to update it later they have to know who
>> "owns" the particular value (i.e. who it comes from) to know whether it
>> should be updated or not.

> In that case, I'm sorry, but I must ask the same question again: how
> will the display engine know which face to display?

> Let's say we have a buffer, in which both package foo and package bar
> defined different faces for the same piece of text.  Which one of them
> will be displayed, and how would the display engine know to choose it?

Internally, one of them sets the property (foo . face) to <face1>, the
other one sets the property (bar . face) to <face2>, and when these are
set, the `face' property gets set to (<face1> <face2>).  The display
engine only looks at the `face' property, just as before.


        Stefan

Re: Combining face and map stuff

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: miles@gnu.org,  emacs-devel@gnu.org
> Date: Tue, 12 Oct 2010 16:23:57 -0400
> 
> > Let's say we have a buffer, in which both package foo and package bar
> > defined different faces for the same piece of text.  Which one of them
> > will be displayed, and how would the display engine know to choose it?
> 
> Internally, one of them sets the property (foo . face) to <face1>, the
> other one sets the property (bar . face) to <face2>, and when these are
> set, the `face' property gets set to (<face1> <face2>).  The display
> engine only looks at the `face' property, just as before.

So it's just a convenience feature, to avoid something like

  get existing text property value
  merge with another value
  put it back with add-text-property

in every package?

Re: Combining face and map stuff

[Stefan Monnier]

>> > Let's say we have a buffer, in which both package foo and package bar
>> > defined different faces for the same piece of text.  Which one of them
>> > will be displayed, and how would the display engine know to choose it?
>> 
>> Internally, one of them sets the property (foo . face) to <face1>, the
>> other one sets the property (bar . face) to <face2>, and when these are
>> set, the `face' property gets set to (<face1> <face2>).  The display
>> engine only looks at the `face' property, just as before.

> So it's just a convenience feature, to avoid something like

>   get existing text property value
>   merge with another value
>   put it back with add-text-property

> in every package?

Yes, tho it goes further than that, since the above steps still leave
some questions unanswered, e.g.:
- how to make sure all packages use the same "merge" function?
- how should a package remove its property values without also removing
  the other packages's property values.


        Stefan