[jbw@macs.hw.ac.uk: overlay face property not used for after-string property]

[jbw@macs.hw.ac.uk: overlay face property not used for after-string property]

[Richard Stallman]

Would someone please fix this, then ack?

------- Start of forwarded message -------
X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY 
	autolearn=failed version=3.1.0
To: bug-gnu-emacs@gnu.org
From: Joe Wells <jbw@macs.hw.ac.uk>
Date: Sun, 21 Oct 2007 04:14:55 +0100
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Subject: overlay face property not used for after-string property

BUG:  An overlay's face property is applied to its before-string
and display properties, but not to its after-string property.

Reproduce with this expression (see below for definition of
test-in-fresh-buffer-and-window):

  (test-in-fresh-buffer-and-window
   (insert "123")
   (let ((o (make-overlay 2 3)))
     (overlay-put o 'before-string "B")
     (overlay-put o 'after-string "A")
     (overlay-put o 'display "D")
     (overlay-put o 'face 'highlight)))

This displays ?1BDA3?.  The ?BDA? portion should be displayed
with face ?highlight?, which means that the background should be
colored ?darkseagreen2?.  However, only the ?BD? portion is
displayed with face ?highlight? and the ?A? portion is wrongly
not displayed with any special face.

I'm assuming the following macro definition in my reproduction
code above:

  (defmacro test-in-fresh-buffer-and-window (&rest body)
    `(progn
       (delete-other-windows)
       (kill-buffer (get-buffer-create "xyzzy"))
       (let ((xyzzy-buf (get-buffer-create "xyzzy")))
         (set-buffer xyzzy-buf)
         (display-buffer xyzzy-buf)
         ,@body
         )))

I hope this bug report is helpful.

Joe

======================================================================
In GNU Emacs 22.1.1 (i686-pc-linux-gnu, GTK+ Version 2.8.20)
 of 2007-06-27 on artemis
Windowing system distributor `The X.Org Foundation', version 11.0.70000000
configured using `configure  '--prefix=/home/jbw/local2' '--enable-debug' '--disable-nls' '--with-x-toolkit=gtk' 'CFLAGS=-O0 -g3 -ggdb''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: en_US.UTF-8
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: jbw
  value of $LANG: nil
  locale-coding-system: utf-8
  default-enable-multibyte-characters: t

Minor modes in effect:
  TeX-source-specials-mode: t
  outline-minor-mode: t
  desktop-save-mode: t
  url-handler-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  unify-8859-on-encoding-mode: t
  utf-translate-cjk-mode: t
  auto-compression-mode: t
  temp-buffer-resize-mode: t
  size-indication-mode: t
  line-number-mode: t
  transient-mark-mode: t
------- End of forwarded message -------

Re: Fwd: overlay face property not used for after-string property

[Stefan Monnier]

> Would someone please fix this, then ack?

I don't think I can fix it, but I remember discussion of something similar
a few years ago where the conclusion was thatit was better to not apply a
property of the overlay (e.g. `face' although I believe the discussion was
about `invisible') to the (after|before)-string because it's easier to add
this property manually to the relevant string than to "undo" the effect of
the overlay's property from the string.


        Stefan


> ------- Start of forwarded message -------
> X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY 
> 	autolearn=failed version=3.1.0
> To: bug-gnu-emacs@gnu.org
> From: Joe Wells <jbw@macs.hw.ac.uk>
> Date: Sun, 21 Oct 2007 04:14:55 +0100
> MIME-Version: 1.0
> Content-Type: text/plain; charset=utf-8
> Subject: overlay face property not used for after-string property

> BUG:??An overlay's face property is applied to its before-string
> and display properties, but not to its after-string property.

> Reproduce with this expression (see below for definition of
> test-in-fresh-buffer-and-window):

>   (test-in-fresh-buffer-and-window
>    (insert "123")
>    (let ((o (make-overlay 2 3)))
>      (overlay-put o 'before-string "B")
>      (overlay-put o 'after-string "A")
>      (overlay-put o 'display "D")
>      (overlay-put o 'face 'highlight)))

> This displays ?1BDA3?.  The ?BDA? portion should be displayed
> with face ?highlight?, which means that the background should be
> colored ?darkseagreen2?.  However, only the ?BD? portion is
> displayed with face ?highlight? and the ?A? portion is wrongly
> not displayed with any special face.

> I'm assuming the following macro definition in my reproduction
> code above:

>   (defmacro test-in-fresh-buffer-and-window (&rest body)
>     `(progn
>        (delete-other-windows)
>        (kill-buffer (get-buffer-create "xyzzy"))
>        (let ((xyzzy-buf (get-buffer-create "xyzzy")))
>          (set-buffer xyzzy-buf)
>          (display-buffer xyzzy-buf)
>          ,@body
>          )))

> I hope this bug report is helpful.

> Joe

> ======================================================================
> In GNU Emacs 22.1.1 (i686-pc-linux-gnu, GTK+ Version 2.8.20)
>  of 2007-06-27 on artemis
> Windowing system distributor `The X.Org Foundation', version 11.0.70000000
> configured using `configure  '--prefix=/home/jbw/local2' '--enable-debug' '--disable-nls' '--with-x-toolkit=gtk' 'CFLAGS=-O0 -g3 -ggdb''

> Important settings:
>   value of $LC_ALL: nil
>   value of $LC_COLLATE: nil
>   value of $LC_CTYPE: en_US.UTF-8
>   value of $LC_MESSAGES: nil
>   value of $LC_MONETARY: nil
>   value of $LC_NUMERIC: nil
>   value of $LC_TIME: jbw
>   value of $LANG: nil
>   locale-coding-system: utf-8
>   default-enable-multibyte-characters: t

> Minor modes in effect:
>   TeX-source-specials-mode: t
>   outline-minor-mode: t
>   desktop-save-mode: t
>   url-handler-mode: t
>   tooltip-mode: t
>   mouse-wheel-mode: t
>   file-name-shadow-mode: t
>   global-font-lock-mode: t
>   font-lock-mode: t
>   blink-cursor-mode: t
>   unify-8859-on-encoding-mode: t
>   utf-translate-cjk-mode: t
>   auto-compression-mode: t
>   temp-buffer-resize-mode: t
>   size-indication-mode: t
>   line-number-mode: t
>   transient-mark-mode: t
> ------- End of forwarded message -------


> _______________________________________________
> Emacs-devel mailing list
> Emacs-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: Fwd: overlay face property not used for after-string property

[Joe Wells]

Stefan Monnier writes:

>> Would someone please fix this, then ack?
>
> I don't think I can fix it, but I remember discussion of something
> similar a few years ago where the conclusion was that it was better
> to not apply a property of the overlay (e.g. `face' although I
> believe the discussion was about `invisible') to the
> (after|before)-string because it's easier to add this property
> manually to the relevant string than to "undo" the effect of the
> overlay's property from the string.

For what it is worth, I agree that it might be better if the face
property (and similar properties like invisible) did not affect the
before-string and after-string properties.  In fact, I think it might
be better if the face property also did not affect the display
property.

However, there is clearly a bug here because before-string is affected
by face while after-string is not.  Either both should be affected or
neither.

Also, it is not straightforward to add the face property manually to
the relevant string, because the string may already contain face
properties.  Doing

  (propertize STRING 'face FACE)

will blast any face properties already on STRING.  If recursive
display properties were allowed, doing

  (propertize " " 'face FACE display STRING)

would do the right thing even if STRING already had face and/or
display properties.  But recursive display properties are not allowed.
And going through the string modifying each face property to add the
correct attributes is a real pain.

It would make my life much easier if recursive display properties were
allowed.  It would be enough in practice to allow recursion to some
small fixed level, like 10 or so.

Joe

P.S.  By the way, when forwarding messages from bug-gnu-emacs@gnu.org,
it would be a big help if the original poster was CC-ed.  I'm not on
the emacs-devel@gnu.org mailing list, and it is a real pain to have to
scan through the web mailing list archives to see if anything is
happening with my bug report.  Also, it is an additional pain to
extract the Message-ID and References headers for the message I am
replying to from the web mailing list archives in order to make a
proper reply.  It would have been much easier (and I would have
replied days ago) if I had just been CC-ed.

P.P.S.  Actually, the issue in the above P.S. would be handled much
better by simply using a proper bug tracker for Emacs.  Aren't you
folks tired of bugs being dropped (in some cases for many years)
because of the lack of a bug tracker?

>> ------- Start of forwarded message -------
>> From: Joe Wells <address@hidden>
>> Date: Sun, 21 Oct 2007 04:14:55 +0100
>> Subject: overlay face property not used for after-string property
>
>> BUG:  An overlay's face property is applied to its before-string
>> and display properties, but not to its after-string property.
>
>> Reproduce with this expression (see below for definition of
>> test-in-fresh-buffer-and-window):
>
>>   (test-in-fresh-buffer-and-window
>>    (insert "123")
>>    (let ((o (make-overlay 2 3)))
>>      (overlay-put o 'before-string "B")
>
>>      (overlay-put o 'after-string "A")
>>      (overlay-put o 'display "D")
>>      (overlay-put o 'face 'highlight)))
>
>> This displays “1BDA3”.  The “BDA?” portion should be displayed
>
>> with face “highlight”, which means that the background should be
>> colored “darkseagreen2”.  However, only the “BD” portion is
>> displayed with face “highlight” and the “A” portion is wrongly
>> not displayed with any special face.
>
>> I'm assuming the following macro definition in my reproduction
>> code above:
>
>>   (defmacro test-in-fresh-buffer-and-window (&amp;rest body)
>>     `(progn
>>        (delete-other-windows)
>>        (kill-buffer (get-buffer-create "xyzzy"))
>>        (let ((xyzzy-buf (get-buffer-create "xyzzy")))
>
>>          (set-buffer xyzzy-buf)
>>          (display-buffer xyzzy-buf)
>>          ,@body
>>          )))
>
>> I hope this bug report is helpful.
>
>> Joe
>
>> ======================================================================
>> In GNU Emacs 22.1.1 (i686-pc-linux-gnu, GTK+ Version 2.8.20)
>>  of 2007-06-27 on artemis
>> Windowing system distributor `The X.Org Foundation', version 11.0.70000000
>> configured using `configure  '--prefix=/home/jbw/local2' '--enable-debug' 
>
>> '--disable-nls' '--with-x-toolkit=gtk' 'CFLAGS=-O0 -g3 -ggdb''
>
>> Important settings:
>>   value of $LC_ALL: nil
>>   value of $LC_COLLATE: nil
>>   value of $LC_CTYPE: en_US.UTF-8
>>   value of $LC_MESSAGES: nil
>
>>   value of $LC_MONETARY: nil
>>   value of $LC_NUMERIC: nil
>>   value of $LC_TIME: jbw
>>   value of $LANG: nil
>>   locale-coding-system: utf-8
>>   default-enable-multibyte-characters: t
>
>> Minor modes in effect:
>>   TeX-source-specials-mode: t
>>   outline-minor-mode: t
>>   desktop-save-mode: t
>>   url-handler-mode: t
>>   tooltip-mode: t
>
>>   mouse-wheel-mode: t
>>   file-name-shadow-mode: t
>>   global-font-lock-mode: t
>>   font-lock-mode: t
>>   blink-cursor-mode: t
>>   unify-8859-on-encoding-mode: t
>
>>   utf-translate-cjk-mode: t
>>   auto-compression-mode: t
>>   temp-buffer-resize-mode: t
>>   size-indication-mode: t
>>   line-number-mode: t
>>   transient-mark-mode: t
>
>> ------- End of forwarded message -------

Re: Fwd: overlay face property not used for after-string property

[Joe Wells]

Richard Stallman <rms@gnu.org> writes:

>     I don't think I can fix it, but I remember discussion of something similar
>     a few years ago where the conclusion was thatit was better to not apply a
>     property of the overlay (e.g. `face' although I believe the discussion was
>     about `invisible') to the (after|before)-string because it's easier to add
>     this property manually to the relevant string than to "undo" the effect of
>     the overlay's property from the string.
>
> That sounds like a valid argument -- but it applies equally to the
> before-string and the after-string.  It can't be right to apply the
> property to one and not the other.
>
> Can someone write a patch so that the overlay's face property (and
> other display-related properties) won't apply to its before-string?
>
> Joe, what do you think about that as a solution?

I think that would be an improvement.  I think the best solution would
be for the face property of an overlay (and similar properties) to not
apply to the before-string, the after-string, _and_ the display
property of the overlay.

Note that there are additional problems.  One example (which I have
already reported) is that the face of the character in the buffer
after the overlay start location (which is not even in the overlay if
the overlay is of length zero) is applied to the before-string and
display properties of the overlay.  Another example (which I think I
have not previously reported) is that the face of the character in the
buffer _after_ the overlay end location (which is _never_ in the
overlay) is applied to the after-string property.

Also, fixing this issue this way leaves behind the difficulty of
adding a face property manually to a property string (before-string,
after-string, or display) which already has face properties.

-- 
Joe

Re: Fwd: overlay face property not used for after-string property

[Johan Bockgård]

Joe Wells <jbw@macs.hw.ac.uk> writes:

> Also, fixing this issue this way leaves behind the difficulty of
> adding a face property manually to a property string (before-string,
> after-string, or display) which already has face properties.

In font-lock.el we find

      ;;; Additional text property functions.

      ;; The following text property functions should be builtins.  This
      ;; means they should be written in C and put with all the other text
      ;; property functions.  In the meantime, those that are used by
      ;; font-lock.el are defined in Lisp below and given a `font-lock-'
      ;; prefix.  Those that are not used are defined in Lisp below and
      ;; commented out.  sm.

      (defun font-lock-prepend-text-property ([...])
        "Prepend to one property of the text from START to END.
      Arguments PROP and VALUE specify the property and value to prepend
      to the value already in place.  The resulting property values are
      always lists.  Optional argument OBJECT is the string or buffer
      containing the text."
      [...]



(let ((s (propertize "abc" 'face 'bar)))
  (font-lock-prepend-text-property 1 2 'face 'foo s)
  s)

  =>  #("abc" 0 1 (face bar) 1 2 (face (foo bar)) 2 3 (face bar))

-- 
Johan Bockgård

Re: Fwd: overlay face property not used for after-string property

[Richard Stallman]

    > Can someone write a patch so that the overlay's face property (and
    > other display-related properties) won't apply to its before-string?
    >
    > Joe, what do you think about that as a solution?

    I think that would be an improvement.  I think the best solution would
    be for the face property of an overlay (and similar properties) to not
    apply to the before-string, the after-string, _and_ the display
    property of the overlay.

Can someone implement that?  It should not be fundamentally hard;
making the properties apply is what is hard.

    Note that there are additional problems.  One example (which I have
    already reported) is that the face of the character in the buffer
    after the overlay start location (which is not even in the overlay if
    the overlay is of length zero) is applied to the before-string and
    display properties of the overlay.

The fix we are discussing will get rid of this problem, right?

					Another example (which I think I
    have not previously reported) is that the face of the character in the
    buffer _after_ the overlay end location (which is _never_ in the
    overlay) is applied to the after-string property.

Let's consider that after this fix is done; perhaps this will
solve that other problem as a byproduct.

Re: Fwd: overlay face property not used for after-string property

[Joe Wells]

Richard Stallman <rms@gnu.org> writes:

>     > Can someone write a patch so that the overlay's face property (and
>     > other display-related properties) won't apply to its before-string?
>     >
>     > Joe, what do you think about that as a solution?
>
>     I think that would be an improvement.  I think the best solution would
>     be for the face property of an overlay (and similar properties) to not
>     apply to the before-string, the after-string, _and_ the display
>     property of the overlay.
>
> Can someone implement that?  It should not be fundamentally hard;
> making the properties apply is what is hard.
>
>     Note that there are additional problems.  One example (which I have
>     already reported) is that the face of the character in the buffer
>     after the overlay start location (which is not even in the overlay if
>     the overlay is of length zero) is applied to the before-string and
>     display properties of the overlay.
>
> The fix we are discussing will get rid of this problem, right?

No, the problem with the before-string/display/after-string properties
being affected by a face from the buffer appears to be a completely
unrelated issue which will need to be debugged separately.

> 					Another example (which I think I
>     have not previously reported) is that the face of the character in the
>     buffer _after_ the overlay end location (which is _never_ in the
>     overlay) is applied to the after-string property.
>
> Let's consider that after this fix is done; perhaps this will
> solve that other problem as a byproduct.

I can't see any way that the change proposed here will help with that
problem.




By the way, changing topic to something else I mentioned in my earlier
message:  Johan Bockgård pointed out in another message that the
convenience functions font-lock-prepend-text-property and
font-lock-append-text-property already exist for the purpose of adding
faces to text (strings or buffer contents) that already have face
properties.  However, I have pointed out in a new bug report that
these functions don't work correctly for all possible face property
values (and indeed they sometimes raise errors).

-- 
Joe

Re: Fwd: overlay face property not used for after-string property

[Richard Stallman]

    > The fix we are discussing will get rid of this problem, right?

    No, the problem with the before-string/display/after-string properties
    being affected by a face from the buffer appears to be a completely
    unrelated issue which will need to be debugged separately.

I see.  We have one problem with overlay properties, and
another with text properties.

I think I just fixed the most of the problem for overlay properties.
Now overlay before-strings and after-strings should be unaffected by
the face properties of all overlays.  They are, however, affected
by `face' text properties and by region highlighting.

It is not 100% clear that that is correct; you might want a longer
overlay's `face' property, or a higher priority overlay's `face'
property, to affect the before-strings and after-strings of smaller
overlays.  That seems like somewhat of a can of worms.  The rule I
implemented is simple and clear.

Do you agree?  And do you get good results from the fix?
(It is in Emacs 22.)

Do you think the same rule is right for display property strings that
come from overlays?  That is, obey `face' text properties, but not
`face' overlay properties?

Once we are sure what is right for overlay properties, I will start
dealing with the text property effects.

Re: Fwd: overlay face property not used for after-string property

[Joe Wells]

Richard Stallman <rms@gnu.org> writes:

>     > The fix we are discussing will get rid of this problem, right?
>
>     No, the problem with the before-string/display/after-string
>     properties being affected by a face from the buffer appears to
>     be a completely unrelated issue which will need to be debugged
>     separately.
>
> I see.  We have one problem with overlay properties, and
> another with text properties.
>
> I think I just fixed the most of the problem for overlay
> properties.  Now overlay before-strings and after-strings should be
> unaffected by the face properties of all overlays.  They are,
> however, affected by `face' text properties and by region
> highlighting.
>
> It is not 100% clear that that is correct; you might want a longer
> overlay's `face' property, or a higher priority overlay's `face'
> property, to affect the before-strings and after-strings of smaller
> overlays.

This is a very good point!

> That seems like somewhat of a can of worms.  The rule I implemented
> is simple and clear.
>
> Do you agree?

I agree that it is simple and clear.  However, given the point you
made above, I think it would be better if before-string and
after-string properties were affected by the face property of other
overlays (regardless of whether they should be affected by the face
property of the same overlay).

> And do you get good results from the fix?  (It is in Emacs 22.)

I will not be able to test in a new Emacs version in the near future.

> Do you think the same rule is right for display property strings that
> come from overlays?  That is, obey `face' text properties, but not
> `face' overlay properties?

No.

> Once we are sure what is right for overlay properties, I will start
> dealing with the text property effects.

By the way, the concern I raise in the message you quote above is not
about text properties in the before-string or after-string, but about
text properties in the buffer.

-- 
Joe

Re: Fwd: overlay face property not used for after-string property

[Richard Stallman]

    I agree that it is simple and clear.  However, given the point you
    made above, I think it would be better if before-string and
    after-string properties were affected by the face property of other
    overlays (regardless of whether they should be affected by the face
    property of the same overlay).

Do you mean ALL other overlays?

So if there are two overlays A and B, A's before-string shoud be
affected by B's face and vice versa?  That doesn't sound quite right
to me, but I don't have a feel for just what would be right.

So I have installed a change that makes available, to the function to
choose the base face, the overlay from which the overlay string came.
The function is now called `face_for_overlay_string', and it is in
xfaces.c.  Given that info, people could implement whatever policy
seems right.

But I don't want to try implementing a different policy myself.  I
don't have an actual use for this feature, so I could try five
different policies with no reason to expect any of them to be good for
actual use.

I will leave this part for others.

Re: Fwd: overlay face property not used for after-string property

[Joe Wells]

Richard Stallman <rms@gnu.org> writes:

>     I agree that it is simple and clear.  However, given the point
>     you made above, I think it would be better if before-string and
>     after-string properties were affected by the face property of
>     other overlays (regardless of whether they should be affected by
>     the face property of the same overlay).
>
> Do you mean ALL other overlays?
>
> So if there are two overlays A and B, A's before-string shoud be
> affected by B's face and vice versa?  That doesn't sound quite right
> to me, but I don't have a feel for just what would be right.

Neither do I.

I would expect if someone put an overlay that contains the entire
buffer, this would affect all things displayed in the buffer from
overlays at lower priority.

I would expect if overlay o2 completely contains overlay o1 and
overlay o2's priority is higher than overlay o1's priority, then
overlay o2's face would affect all things displayed by overlay o1.

The above items seem clearly right.  However, a lot of strange corner
cases arise.

What if overlay o1 and overlay o2 cover the exact same region and have
the same priority?  Should they affect the display of each other's
before-string, display, and after-string properties?

What if

  (and (<= (overlay-start o2)         (overlay-start o1))
       (<= (overlay-start o1)         (overlay-end o2))
       (<  (overlay-end o2)           (overlay-end o1))
       (<  (overlay-get o1 'priority) (overlay-get o2 'priority)))

?  Should overlay o2's face affect overlay o1's before-string (but not
its after-string)?

And there are other cases to consider.

Comments from people who have implemented stuff using overlays would
be welcome here.

> So I have installed a change that makes available, to the function to
> choose the base face, the overlay from which the overlay string came.
> The function is now called `face_for_overlay_string', and it is in
> xfaces.c.  Given that info, people could implement whatever policy
> seems right.

I think this information may not be enough.  I think the policy would
also need to know which of the overlay properties before-string,
display, or after-string the string comes from.  (Note that coming
from an overlay's display string is distinct from coming from a
display string as a text property on an overlay's before-string, which
should count as coming from the before-string.)

-- 
Joe

> But I don't want to try implementing a different policy myself.  I
> don't have an actual use for this feature, so I could try five
> different policies with no reason to expect any of them to be good for
> actual use.
>
> I will leave this part for others.

Re: Fwd: overlay face property not used for after-string property

[Richard Stallman]

    I think this information may not be enough.  I think the policy would
    also need to know which of the overlay properties before-string,
    display, or after-string the string comes from.

In most cases you can determine this by looking at the overlay's
properties and seeing where the string appears.  That will work except
when the overlay has two properties that contain strings that are eq.

That is not good enough for the final implementation, but it is good
enough for experimenting to find the right criteria.  If the right
criteria really depend on this information, we can add a mechanism to
provide the information efficiently and reliably.  So I see no need to
do that until we know whether it is needed.

Re: Fwd: overlay face property not used for after-string property

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

>     I think this information may not be enough.  I think the policy would
>     also need to know which of the overlay properties before-string,
>     display, or after-string the string comes from.
>
> In most cases you can determine this by looking at the overlay's
> properties and seeing where the string appears.  That will work except
> when the overlay has two properties that contain strings that are eq.
>
> That is not good enough for the final implementation, but it is good
> enough for experimenting to find the right criteria.  If the right
> criteria really depend on this information, we can add a mechanism to
> provide the information efficiently and reliably.  So I see no need to
> do that until we know whether it is needed.

I think that the before-string should, in effect, use
(get-char-property (overlay-start ov) 'face)
to determine the face to use if no fully specified face is in the
before-string.

Similarly for after-string.  That's simple, clear and straightforward
for the user to understand.  It also has the "right thing" feeling to
it, which ignoring surrounding faces doesn't.

-- 
David Kastrup

Re: Fwd: overlay face property not used for after-string property

[Joe Wells]

David Kastrup <dak@gnu.org> writes:

> Richard Stallman <rms@gnu.org> writes:
>
>>     I think this information may not be enough.  I think the policy
>>     would also need to know which of the overlay properties
>>     before-string, display, or after-string the string comes from.
>>
>> In most cases you can determine this by looking at the overlay's
>> properties and seeing where the string appears.  That will work except
>> when the overlay has two properties that contain strings that are eq.
>>
>> That is not good enough for the final implementation, but it is good
>> enough for experimenting to find the right criteria.  If the right
>> criteria really depend on this information, we can add a mechanism to
>> provide the information efficiently and reliably.  So I see no need to
>> do that until we know whether it is needed.
>
> I think that the before-string should, in effect, use
> (get-char-property (overlay-start ov) 'face)
> to determine the face to use if no fully specified face is in the
> before-string.

The behavior you propose is currently used and causes problems.  I
have complained about it (in separate messages) as a bug.  I have seen
it causing problems for linum.el and tex-fold.el.

When (eq (overlay-start o) (overlay-end o)), the character after
(overlay-start o) is not even inside the overlay.  It makes no logical
sense for the face of a character not covered by the overlay to affect
the display of things specified by the overlay.

Requiring the face of a string displayed by an overlay to be fully
specified means that such a face can never adjust automatically to
changes in other faces.  Yet this is needed.  For example, if the
default face is changed (e.g., to change the overall size of the text
for easier viewing), it is desired that all faces will automatically
adjust.  Similarly, if the region is activated, it would be nice if
all faces that do not need to specify a background would get the
background of face “region”.  With your proposal (which is already
implemented), an overlay's before-string would have to specify a
background color (even if merely white) and could not automatically
pick up the background of the region face.

I would like to strongly suggest that overlay before-string, display,
and after-string properties should *not* be affected by the
text-property face properties in the buffer.

> Similarly for after-string.  That's simple, clear and straightforward
> for the user to understand.  It also has the "right thing" feeling to
> it, which ignoring surrounding faces doesn't.

It definitely has the “wrong thing” feeling to it for me.

-- 
Joe

Re: Fwd: overlay face property not used for after-string property

[Joe Wells]

Joe Wells <jbw@macs.hw.ac.uk> writes:

> David Kastrup <dak@gnu.org> writes:
>
>> Richard Stallman <rms@gnu.org> writes:
>>
>>>     I think this information may not be enough.  I think the policy
>>>     would also need to know which of the overlay properties
>>>     before-string, display, or after-string the string comes from.
>>>
>>> In most cases you can determine this by looking at the overlay's
>>> properties and seeing where the string appears.  That will work except
>>> when the overlay has two properties that contain strings that are eq.
>>>
>>> That is not good enough for the final implementation, but it is good
>>> enough for experimenting to find the right criteria.  If the right
>>> criteria really depend on this information, we can add a mechanism to
>>> provide the information efficiently and reliably.  So I see no need to
>>> do that until we know whether it is needed.
>>
>> I think that the before-string should, in effect, use
>> (get-char-property (overlay-start ov) 'face)
>> to determine the face to use if no fully specified face is in the
>> before-string.
>
> The behavior you propose is currently used and causes problems.  I
> have complained about it (in separate messages) as a bug.  I have seen
> it causing problems for linum.el and tex-fold.el.
>
> When (eq (overlay-start o) (overlay-end o)), the character after
> (overlay-start o) is not even inside the overlay.  It makes no logical
> sense for the face of a character not covered by the overlay to affect
> the display of things specified by the overlay.

An additional point.  Suppose the overlay has both before-string and
display properties.  (Yes, you need to do this if you want to use an
overlay to replace the text in the buffer and display multiple images
or a mixture of text and images.)  The display property replaces the
display of the text in the buffer (and possibly specifies a different
face for it).  It makes no logical sense in this case for the display
of the before-string to use the face of a character in the buffer that
is being concealed.

-- 
Joe

> Requiring the face of a string displayed by an overlay to be fully
> specified means that such a face can never adjust automatically to
> changes in other faces.  Yet this is needed.  For example, if the
> default face is changed (e.g., to change the overall size of the text
> for easier viewing), it is desired that all faces will automatically
> adjust.  Similarly, if the region is activated, it would be nice if
> all faces that do not need to specify a background would get the
> background of face “region”.  With your proposal (which is already
> implemented), an overlay's before-string would have to specify a
> background color (even if merely white) and could not automatically
> pick up the background of the region face.
>
> I would like to strongly suggest that overlay before-string, display,
> and after-string properties should *not* be affected by the
> text-property face properties in the buffer.
>
>> Similarly for after-string.  That's simple, clear and straightforward
>> for the user to understand.  It also has the "right thing" feeling to
>> it, which ignoring surrounding faces doesn't.
>
> It definitely has the “wrong thing” feeling to it for me.
>
> -- 
> Joe

Re: Fwd: overlay face property not used for after-string property

[Richard Stallman]

    An additional point.  Suppose the overlay has both before-string and
    display properties.  (Yes, you need to do this if you want to use an
    overlay to replace the text in the buffer and display multiple images
    or a mixture of text and images.)  The display property replaces the
    display of the text in the buffer (and possibly specifies a different
    face for it).  It makes no logical sense in this case for the display
    of the before-string to use the face of a character in the buffer that
    is being concealed.

In some cases it does.  Consider a paragraph that is italicized
with a text property.  Shouldn't substitute text be italic by default?

Re: Fwd: overlay face property not used for after-string property

[Joe Wells]

Richard Stallman <rms@gnu.org> writes:

>     An additional point.  Suppose the overlay has both before-string
>     and display properties.  (Yes, you need to do this if you want
>     to use an overlay to replace the text in the buffer and display
>     multiple images or a mixture of text and images.)

(And by the way, this would be easier and less awkward if at least one
additional level of string display property inside overlay display
property was supported.  But this doesn't affect the issue we are
discussing here.)

>     The display property replaces the display of the text in the
>     buffer (and possibly specifies a different face for it).  It
>     makes no logical sense in this case for the display of the
>     before-string to use the face of a character in the buffer that
>     is being concealed.
>
> In some cases it does.  Consider a paragraph that is italicized
> with a text property.  Shouldn't substitute text be italic by
> default?

Counterexample: linum.el.

I think priorities are probably the solution.  Here is an idea.  Each
face property and each piece of text has a priority.  A face property
can only affect text of strictly lower priority.  The before-string,
display (if a string), and after-string properties of an overlay have
the priority of their overlay.  Text in the buffer has priority 0.
The face property of an overlay has the priority of that overlay.  A
text property face property (i.e., in a string or in the buffer) has
the priority of the piece of text plus 0.5 (so that it will affect the
text it belongs to).

Comments?

-- 
Joe

Re: Fwd: overlay face property not used for after-string property

[Stefan Monnier]

> Counterexample: linum.el.

BTW, IIUC the problem with linum.el is Emacs's handling of margin text: it
makes no sense (to me, and apparently to linum.el) for the text displayed in
the margin to inherit face properties from the buffer point where that text
is "located".


        Stefan

Re: Fwd: overlay face property not used for after-string property

[David Kastrup]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Counterexample: linum.el.
>
> BTW, IIUC the problem with linum.el is Emacs's handling of margin
> text: it makes no sense (to me, and apparently to linum.el) for the
> text displayed in the margin to inherit face properties from the
> buffer point where that text is "located".

If you have tabulated text in alternately colored background lines, it
might well make sense to inherit that.

Please: be aware that we are talking about a generic mechanism here.
It does not make sense to hard-code behavior for a single application
into it.  There is more than one behavior that makes sense, and if we
find that we can only cater for a single behavior of several possibly
making sense (or being consistent), then we need to extend the API
rather than fix the behavior to a single possibility.

-- 
David Kastrup

Re: Fwd: overlay face property not used for after-string property

[Stefan Monnier]

>> BTW, IIUC the problem with linum.el is Emacs's handling of margin
>> text: it makes no sense (to me, and apparently to linum.el) for the
>> text displayed in the margin to inherit face properties from the
>> buffer point where that text is "located".

> If you have tabulated text in alternately colored background lines, it
> might well make sense to inherit that.

But then this background should be inherited by the margin rather than by
the margin's text, right? (i.e. it should be inherited even if there's no
text placed in the margin)

> Please: be aware that we are talking about a generic mechanism here.
> It does not make sense to hard-code behavior for a single application
> into it.  There is more than one behavior that makes sense, and if we
> find that we can only cater for a single behavior of several possibly
> making sense (or being consistent), then we need to extend the API
> rather than fix the behavior to a single possibility.

Don't worry: it's always at the top of my concerns,


        Stefan

Re: Fwd: overlay face property not used for after-string property

[Richard Stallman]

    BTW, IIUC the problem with linum.el is Emacs's handling of margin text: it
    makes no sense (to me, and apparently to linum.el) for the text displayed in
    the margin to inherit face properties from the buffer point where that text
    is "located".

We could easily turn off this inheritance entirely for margin text.
If we are sure that is the right thing to do.

Re: Fwd: overlay face property not used for after-string property

[Richard Stallman]

    I think priorities are probably the solution.  Here is an idea.  Each
    face property and each piece of text has a priority.  A face property
    can only affect text of strictly lower priority.  The before-string,
    display (if a string), and after-string properties of an overlay have
    the priority of their overlay.  Text in the buffer has priority 0.
    The face property of an overlay has the priority of that overlay.  A
    text property face property (i.e., in a string or in the buffer) has
    the priority of the piece of text plus 0.5 (so that it will affect the
    text it belongs to).

It sounds plausible.  Could you figure out how you would use this and
verify that it would give correct results in the cases you know about?

Re: Fwd: overlay face property not used for after-string property

[Richard Stallman]

    I would like to strongly suggest that overlay before-string, display,
    and after-string properties should *not* be affected by the
    text-property face properties in the buffer.

That would be easy to implement, but it seems to me that in some cases
text properties should influence these strings, just as region
highlighting should.

Since various features use overlays to do something analogous to
region highlighting, that suggests that in SOME cases we do want one
overlay's faces to affect the strings provided by another overlay, as
you asked for recently.  A study of the interaction of those features
with something like linum might make it possible to figure out some
good rules for such interaction.

Re: Fwd: overlay face property not used for after-string property

[Joe Wells]

Richard Stallman <rms@gnu.org> writes:

>     I would like to strongly suggest that overlay before-string,
>     display, and after-string properties should *not* be affected by
>     the text-property face properties in the buffer.
>
> That would be easy to implement, but it seems to me that in some
> cases text properties should influence these strings, just as region
> highlighting should.

How about negative overlay priorities for this?

(I assume you mean “text properties in the buffer”, as the phrase
“text properties” includes text properties in the strings.)

> Since various features use overlays to do something analogous to
> region highlighting, that suggests that in SOME cases we do want one
> overlay's faces to affect the strings provided by another overlay, as
> you asked for recently.

Indeed.

> A study of the interaction of those features

Maybe collect a list of all the authors of code that uses overlays
heavily and CC them on this discussion?  :-)

> with something like linum might make it possible to figure out some
> good rules for such interaction.

By the way (a slight shift of topic here), I think the overlays used
by linum.el should *not* be affected by the active region face.  I
think we already discussed this in the e-mail thread on linum.el and
the idea was to solve this by giving the active region face a numeric
priority (e.g., something like 100 or so) and overlays with higher
priority would not be affected.  (Has this been done already?)

-- 
Joe

Re: Fwd: overlay face property not used for after-string property

[Stefan Monnier]

> I think that the before-string should, in effect, use
> (get-char-property (overlay-start ov) 'face)
> to determine the face to use if no fully specified face is in the
> before-string.

No.  Go read the beginning of this thread again where I explained why this
is bad: it's (much) harder to remove a face than to add one.

Also text-properties should not affect before/after-strings.  I believe the
most obviously sensible rule is to follow the precedence that we always use:
- overlays take precedence over text-properties
- overlays of higher priority take precedence over overlays of lower priority
- for overlays of equal priority if one overlay covers the other it takes
  precedence
- for the other cases of equal priority, any arbitrary choice is OK as long
  as it's deterministic

Given this, a (before|after)-string should only be affected by
invisible|face properties set by overlays of higher precedence: not by
text-properties, not be overlays of lower precedence.


-- Stefan

Re: Fwd: overlay face property not used for after-string property

[David Kastrup]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I think that the before-string should, in effect, use
>> (get-char-property (overlay-start ov) 'face)
>> to determine the face to use if no fully specified face is in the
>> before-string.
>
> No.  Go read the beginning of this thread again where I explained
> why this is bad: it's (much) harder to remove a face than to add
> one.

Then our problem is that it is much harder to remove a face than to
add one.  And that is the problem we should fix.

> Also text-properties should not affect before/after-strings.

I don't see why not if they are appropriately sticky.

> I believe the most obviously sensible rule is to follow the
> precedence that we always use: - overlays take precedence over
> text-properties - overlays of higher priority take precedence over
> overlays of lower priority - for overlays of equal priority if one
> overlay covers the other it takes precedence - for the other cases
> of equal priority, any arbitrary choice is OK as long as it's
> deterministic
>
> Given this, a (before|after)-string should only be affected by
> invisible|face properties set by overlays of higher precedence: not
> by text-properties, not be overlays of lower precedence.

Resolving partially specified faces goes through priorities.  If there
are usage cases for before/after-string that should not inherit, then
we need to add a way to say "completely resolve to 'default (or
whatever other face) here".  Then things like lineno can use that way.

But it does not make sense to substitute a missing facility with some
fixed but illogical rules that cater for some but not all use cases
because it is easier to override this in that manner.

-- 
David Kastrup

Re: Fwd: overlay face property not used for after-string property

[Joe Wells]

David Kastrup <dak@gnu.org> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>>> I think that the before-string should, in effect, use
>>> (get-char-property (overlay-start ov) 'face)
>>> to determine the face to use if no fully specified face is in the
>>> before-string.
>>
>> No.  Go read the beginning of this thread again where I explained
>> why this is bad: it's (much) harder to remove a face than to add
>> one.
>
> Then our problem is that it is much harder to remove a face than to
> add one.  And that is the problem we should fix.

It's not actually an issue of “removing” a face.  The issue is
actually “preventing inheritance” of a face.

>> Also text-properties should not affect before/after-strings.
>
> I don't see why not if they are appropriately sticky.

An example application is linum.el.  This code needs to display
strings at the beginning of each line and the strings need to be
displayed with a face independent of the face of the adjacent text
(regardless of whether there are sticky text properties there).  At
the same time, linum.el should not use a fully specified face because
the face should be affected by the default face of the frame.

>> I believe the most obviously sensible rule is to follow the
>> precedence that we always use: - overlays take precedence over
>> text-properties - overlays of higher priority take precedence over
>> overlays of lower priority - for overlays of equal priority if one
>> overlay covers the other it takes precedence - for the other cases
>> of equal priority, any arbitrary choice is OK as long as it's
>> deterministic
>>
>> Given this, a (before|after)-string should only be affected by
>> invisible|face properties set by overlays of higher precedence: not
>> by text-properties, not be overlays of lower precedence.
>
> Resolving partially specified faces goes through priorities.  If there
> are usage cases for before/after-string that should not inherit, then
> we need to add a way to say "completely resolve to 'default (or
> whatever other face) here".  Then things like lineno can use that way.

The idea of “completely resolve to default” sounds interesting
(independently of what we are discussing).

> But it does not make sense to substitute a missing facility with some
> fixed but illogical rules that cater for some but not all use cases
> because it is easier to override this in that manner.

The problem is that the existing rules are already illogical and we
are trying to figure out how to make them less illogical.

-- 
Joe

Re: Fwd: overlay face property not used for after-string property

[David Kastrup]

Joe Wells <jbw@macs.hw.ac.uk> writes:

> David Kastrup <dak@gnu.org> writes:
>
>> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>>
>>>> I think that the before-string should, in effect, use
>>>> (get-char-property (overlay-start ov) 'face)
>>>> to determine the face to use if no fully specified face is in the
>>>> before-string.
>>>
>>> No.  Go read the beginning of this thread again where I explained
>>> why this is bad: it's (much) harder to remove a face than to add
>>> one.
>>
>> Then our problem is that it is much harder to remove a face than to
>> add one.  And that is the problem we should fix.
>
> It's not actually an issue of “removing” a face.  The issue is
> actually “preventing inheritance” of a face.
>
>>> Also text-properties should not affect before/after-strings.
>>
>> I don't see why not if they are appropriately sticky.
>
> An example application is linum.el.  This code needs to display
> strings at the beginning of each line and the strings need to be
> displayed with a face independent of the face of the adjacent text
> (regardless of whether there are sticky text properties there).

That's what I say: instead of making the behavior illogical in order
to default to some behavior good for some application, we should add
the facility for the application to explicitly request the behavior it
needs.

>> Resolving partially specified faces goes through priorities.  If
>> there are usage cases for before/after-string that should not
>> inherit, then we need to add a way to say "completely resolve to
>> 'default (or whatever other face) here".  Then things like lineno
>> can use that way.
>
> The idea of “completely resolve to default” sounds interesting
> (independently of what we are discussing).
>
>> But it does not make sense to substitute a missing facility with
>> some fixed but illogical rules that cater for some but not all use
>> cases because it is easier to override this in that manner.
>
> The problem is that the existing rules are already illogical

Because of a bug.

> and we are trying to figure out how to make them less illogical.

What I see happening is that the current bug is used as an excuse to
establish a different inconsistent behavior as "correct", based on
convenience for some applications.  Basically "if we are being
inconsistent, we might as well be inconsistent in a more convenient
manner."

And I do not consider this a good idea.  If the consistent and logical
behavior is inconvenient for some applications, we need to add the
facilities for overriding it.  But the override should not become a
fixed default.

-- 
David Kastrup

Re: Fwd: overlay face property not used for after-string property

[Joe Wells]

David Kastrup <dak@gnu.org> writes:

> Joe Wells <jbw@macs.hw.ac.uk> writes:
>
>> David Kastrup <dak@gnu.org> writes:
>>
>>> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>>>
>>>>> I think that the before-string should, in effect, use
>>>>> (get-char-property (overlay-start ov) 'face)
>>>>> to determine the face to use if no fully specified face is in the
>>>>> before-string.
>>>>
>>>> No.  Go read the beginning of this thread again where I explained
>>>> why this is bad: it's (much) harder to remove a face than to add
>>>> one.
>>>
>>> Then our problem is that it is much harder to remove a face than to
>>> add one.  And that is the problem we should fix.
>>
>> It's not actually an issue of “removing” a face.  The issue is
>> actually “preventing inheritance” of a face.
>>
>>>> Also text-properties should not affect before/after-strings.
>>>
>>> I don't see why not if they are appropriately sticky.
>>
>> An example application is linum.el.  This code needs to display
>> strings at the beginning of each line and the strings need to be
>> displayed with a face independent of the face of the adjacent text
>> (regardless of whether there are sticky text properties there).
>
> That's what I say: instead of making the behavior illogical in order
> to default to some behavior good for some application, we should add
> the facility for the application to explicitly request the behavior it
> needs.

The current behavior is illogical, so I don't understand your
complaint.

Also, I believe adding a feature to try to block inheritance in some
cases will lead to a specification that is too complicated to
understand.  It is simpler to have fewer inheritances occur in the
first place.

>>> Resolving partially specified faces goes through priorities.  If
>>> there are usage cases for before/after-string that should not
>>> inherit, then we need to add a way to say "completely resolve to
>>> 'default (or whatever other face) here".  Then things like lineno
>>> can use that way.
>>
>> The idea of “completely resolve to default” sounds interesting
>> (independently of what we are discussing).
>>
>>> But it does not make sense to substitute a missing facility with
>>> some fixed but illogical rules that cater for some but not all use
>>> cases because it is easier to override this in that manner.
>>
>> The problem is that the existing rules are already illogical
>
> Because of a bug.

I agree that there is a bug.  I don't know what you think the bug is.
For me, the bug is that a face given by a text property in the buffer
can affect the display of before-string and after-string properties of
overlays.

I would be willing to allow this for overlays with negative priority.
That could be a useful meaning for negative priorities (which
currently have no meaning).

>> and we are trying to figure out how to make them less illogical.
>
> What I see happening is that the current bug is used as an excuse to
> establish a different inconsistent behavior as "correct", based on
> convenience for some applications.

Actually, we are trying to propose a more consistent behavior (which
is also more useful).

> Basically "if we are being
> inconsistent, we might as well be inconsistent in a more convenient
> manner."

I think it is perfectly consistent to think “only higher priority face
properties affect a piece of text”, with text properties in the buffer
being considered to have the lowest priority (or priority zero if we
now decide to allow negative overlay priorities).

> And I do not consider this a good idea.  If the consistent and logical
> behavior is inconvenient for some applications, we need to add the
> facilities for overriding it.  But the override should not become a
> fixed default.

As I mention above, I think adding a facility for preventing certain
cases of face inheritance will make things too complicated, and it is
better to remove the problematic cases of inheritance.

-- 
Joe

Re: Fwd: overlay face property not used for after-string property

[Joe Wells]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I think that the before-string should, in effect, use
>> (get-char-property (overlay-start ov) 'face)
>> to determine the face to use if no fully specified face is in the
>> before-string.
>
> No.  Go read the beginning of this thread again where I explained why this
> is bad: it's (much) harder to remove a face than to add one.
>
> Also text-properties should not affect before/after-strings.

Except of course that text properties inside the string itself should
have an effect.

> I believe the
> most obviously sensible rule is to follow the precedence that we always use:
> - overlays take precedence over text-properties

I'll assume by “text-properties” you mean “text properties in the
buffer”.

> - overlays of higher priority take precedence over overlays of lower
>   priority

Does the higher priority overlay need to entirely contain the
lower-priority overlay to have an effect?  What if the overlays
partially overlap, but neither contains the other?  Suppose the end of
overlay o2 is the same as the start of overlay o1:  Can overlay o2's
face affect overlay o1's before-string?

> - for overlays of equal priority if one overlay covers the other it takes
>   precedence

So covering an overlay is like having higher priority?  What if the
two overlays have the same extent?  Does this count as covering?  If
so, do two overlays with the same extent and the same priority cover
each other?

> - for the other cases of equal priority, any arbitrary choice is OK as long
>   as it's deterministic
>
> Given this, a (before|after)-string should only be affected by
> invisible|face properties set by overlays of higher precedence: not by
> text-properties, not be overlays of lower precedence.

So you propose things work by “precedence” which is derived from
“priority” and “covering”?

-- 
Joe

Re: Fwd: overlay face property not used for after-string property

[Stefan Monnier]

>> Also text-properties should not affect before/after-strings.
> Except of course that text properties inside the string itself should
> have an effect.

Indeed.

>> I believe the
>> most obviously sensible rule is to follow the precedence that we always use:
>> - overlays take precedence over text-properties

> I'll assume by “text-properties” you mean “text properties in the
> buffer”.

Indeed.

>> - overlays of higher priority take precedence over overlays of lower
>> priority

> Does the higher priority overlay need to entirely contain the
> lower-priority overlay to have an effect?

No.  Note that these rules are general rules about how overlays and
text-properties interact.

> What if the overlays partially overlap, but neither contains the
> other?

In general, this makes no difference...

> Suppose the end of overlay o2 is the same as the start of
> overlay o1:  Can overlay o2's face affect overlay o1's before-string?

.. in the case of (before|after)-strings, interpreting what this rule
implies is trickier, but I guess it'd be something like:

  the overlays that apply to an (before|after)-string are those that
  have higher precedence than the string's overlay and that would apply
  to text inserted at that position in the buffer.

>> - for overlays of equal priority if one overlay covers the other it takes
>> precedence

> So covering an overlay is like having higher priority?

Yes.

> What if the two overlays have the same extent?

Then this rule doesn't apply and the next one (the catch-all)
applies instead.

>> - for the other cases of equal priority, any arbitrary choice is OK as long
>> as it's deterministic
>> 
>> Given this, a (before|after)-string should only be affected by
>> invisible|face properties set by overlays of higher precedence: not by
>> text-properties, not be overlays of lower precedence.

> So you propose things work by “precedence” which is derived from
> “priority” and “covering”?

Not quite: instead I described how it currently works in general, and
I suggest that we solve our problem w.r.t (before|after)-string by
trying to extrapolate from the existing rules.

I'm not 100% sure the result will be the most convenient in every single
case, but at least it will have the advantage of being conceptually clean.


        Stefan

Re: Fwd: overlay face property not used for after-string property

[Joe Wells]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> Also text-properties should not affect before/after-strings.
>> Except of course that text properties inside the string itself should
>> have an effect.
>
> Indeed.
>
>>> I believe the
>>> most obviously sensible rule is to follow the precedence that we always use:
>>> - overlays take precedence over text-properties
>
>> I'll assume by “text-properties” you mean “text properties in the
>> buffer”.
>
> Indeed.
>
>>> - overlays of higher priority take precedence over overlays of lower
>>> priority
>
>> Does the higher priority overlay need to entirely contain the
>> lower-priority overlay to have an effect?
>
> No.  Note that these rules are general rules about how overlays and
> text-properties interact.

Suppose overlay o2 has higher priority than overlay o1 and covers only
part of overlay o1.  Does o2's face affect o1's display property?

>> What if the overlays partially overlap, but neither contains the
>> other?
>
> In general, this makes no difference...

What about overlays' display properties?

>> Suppose the end of overlay o2 is the same as the start of
>> overlay o1:  Can overlay o2's face affect overlay o1's before-string?
>
> .. in the case of (before|after)-strings, interpreting what this rule

What is “this rule” here?

> implies is trickier, but I guess it'd be something like:
>
>   the overlays that apply to an (before|after)-string are those that
>   have higher precedence than the string's overlay and that would apply
>   to text inserted at that position in the buffer.

I like this (at least for before-string and after-string, it is not
clear what happens for overlay display properties).

>>> - for overlays of equal priority if one overlay covers the other it takes
>>> precedence
>
>> So covering an overlay is like having higher priority?
>
> Yes.
>
>> What if the two overlays have the same extent?
>
> Then this rule doesn't apply and the next one (the catch-all)
> applies instead.

I don't understand why “this rule” doesn't apply in this case.  Can
you explain?

>>> - for the other cases of equal priority, any arbitrary choice is OK as long
>>> as it's deterministic
>>>
>>> Given this, a (before|after)-string should only be affected by
>>> invisible|face properties set by overlays of higher precedence: not by
>>> text-properties, not be overlays of lower precedence.
>
>> So you propose things work by “precedence” which is derived from
>> “priority” and “covering”?
>
> Not quite: instead I described how it currently works in general, and
> I suggest that we solve our problem w.r.t (before|after)-string by
> trying to extrapolate from the existing rules.
>
> I'm not 100% sure the result will be the most convenient in every single
> case, but at least it will have the advantage of being conceptually clean.

Can you write down the full proposed rule set in plain English?  I'm
getting lost trying to follow.

-- 
Joe

Re: Fwd: overlay face property not used for after-string property

[Richard Stallman]

    Suppose overlay o2 has higher priority than overlay o1 and covers only
    part of overlay o1.  Does o2's face affect o1's display property?

I think it is ok if the answer is "undefined".

Re: Fwd: overlay face property not used for after-string property

[Richard Stallman]

    Given this, a (before|after)-string should only be affected by
    invisible|face properties set by overlays of higher precedence: not by
    text-properties, not be overlays of lower precedence.

Would you like to try implementing that in face_for_overlay_string?
Maybe it will do the job.

Re: Fwd: overlay face property not used for after-string property

[Joe Wells]

Richard Stallman <rms@gnu.org> writes:

>     I think this information may not be enough.  I think the policy
>     would also need to know which of the overlay properties
>     before-string, display, or after-string the string comes from.
>
> In most cases you can determine this by looking at the overlay's
> properties and seeing where the string appears.  That will work except
> when the overlay has two properties that contain strings that are eq.

This would be a real pain.  What if the string being displayed is a
display property on the before-string?  You have to not just check all
3 of before-string, display, and after-string overlay properties, but
you also have to scan through both the before-string and after-string
properties looking for display text properties.

(By the way, in a separate message, I have complained about a bug that
strikes when eq strings are used in distinct display properties.)

> That is not good enough for the final implementation, but it is good
> enough for experimenting to find the right criteria.

I don't think it is good enough even for experimenting, for the reason
I indicate above.

> If the right
> criteria really depend on this information, we can add a mechanism to
> provide the information efficiently and reliably.  So I see no need to
> do that until we know whether it is needed.

-- 
Joe

Re: Fwd: overlay face property not used for after-string property

[Richard Stallman]

    > In most cases you can determine this by looking at the overlay's
    > properties and seeing where the string appears.  That will work except
    > when the overlay has two properties that contain strings that are eq.

    This would be a real pain.  What if the string being displayed is a
    display property on the before-string?

The experiments don't need to handle such complex cases
in order to serve their purpose: to figure out the right
criteria for which overlays' properties should influence
which overlays' strings.

Once we are confident of the right criteria, we could
implement them cleanly and reliably and efficiently.

Re: Fwd: overlay face property not used for after-string property

[Richard Stallman]

    By the way, the concern I raise in the message you quote above is not
    about text properties in the before-string or after-string, but about
    text properties in the buffer.

I know.  Can you send a test case for this?

Re: Fwd: overlay face property not used for after-string property

[Joe Wells]

Richard Stallman <rms@gnu.org> writes:

>     By the way, the concern I raise in the message you quote above
>     is not about text properties in the before-string or
>     after-string, but about text properties in the buffer.
>
> I know.  Can you send a test case for this?

There is a test case in this message:

  http://www.nabble.com/anomalies-of-overlays-and-before-string-and-display-properties-p12777475.html

The message at that URL has a single test case that demonstrates
several bugs.  Bugs #3 and #4 in that message are the same as the bug
I am mentioning in the quoted message above.  I haven't had time to
prepare a test case that isolates this bug all on its own.

By the way, I think bug #2 in the message at the above URL has been
fixed (or partially fixed), but all the other bugs I think have been
left untouched.

-- 
Joe

Re: Fwd: overlay face property not used for after-string property

[Stefan Monnier]

> Now overlay before-strings and after-strings should be unaffected by
> the face properties of all overlays.

This seems extreme.

> You might want a longer overlay's `face' property, or a higher
> priority overlay's `face' property, to affect the before-strings and
> after-strings of smaller overlays.

That sounds right.


        Stefan