Nested display strings

Nested display strings

[Eli Zaretskii]

The following 4 lines define two overlays with display strings, with
the second overlay "nested" inside the first one:

(defvar myov1 (make-overlay 16 51))
(overlay-put myov1 'display "STRING1")
(defvar myov2 (make-overlay 29 43))
(overlay-put myov2 'display "STRING2")

If you evaluate this in the *scratch* buffer, the result is this:

;; This buffer STRING1STRING2STRING1, and for Lisp evaluation.

That is, STRING1 is displayed twice!  Is this a bug or a feature?

I asked Gerd Möllmann, and he told me that this is an accident: what
he really meant was, once STRING1 is displayed, to skip to the end of
its overlay, so that the nested overlay myov2 would not be displayed
at all.

However, after so many years (this is how Emacs behaves since v21.1),
perhaps we should keep this behavior.  Does anyone know any real-life
use cases where this "feature" is needed or would be beneficial?

The reason I'm asking is that whether we keep or remove this feature
affects the detailed design of how bidirectional text should be
reordered in the presence of display properties and overlay strings.

Re: Nested display strings

[Andreas Röhler]

Am 23.04.2011 18:44, schrieb Eli Zaretskii:
> The following 4 lines define two overlays with display strings, with
> the second overlay "nested" inside the first one:
>
> (defvar myov1 (make-overlay 16 51))
> (overlay-put myov1 'display "STRING1")
> (defvar myov2 (make-overlay 29 43))
> (overlay-put myov2 'display "STRING2")
>
> If you evaluate this in the *scratch* buffer, the result is this:
>
> ;; This buffer STRING1STRING2STRING1, and for Lisp evaluation.
>
> That is, STRING1 is displayed twice!  Is this a bug or a feature?
>
> I asked Gerd Möllmann, and he told me that this is an accident: what
> he really meant was, once STRING1 is displayed, to skip to the end of
> its overlay, so that the nested overlay myov2 would not be displayed
> at all.
>
> However, after so many years (this is how Emacs behaves since v21.1),
> perhaps we should keep this behavior.  Does anyone know any real-life
> use cases where this "feature" is needed or would be beneficial?
>
> The reason I'm asking is that whether we keep or remove this feature
> affects the detailed design of how bidirectional text should be
> reordered in the presence of display properties and overlay strings.
>
>
>

Hi,

would consider it definitly a bug with respect of
following order of evaluation:

(defvar myov2 (make-overlay 29 43))
(defvar myov1 (make-overlay 16 51))
(overlay-put myov2 'display "STRING2")
(overlay-put myov1 'display "STRING1")

ie if the wider overlay is applied last, it should take all.

Cheers

Andreas

Re: Nested display strings

[Eli Zaretskii]

> Date: Sat, 23 Apr 2011 21:42:14 +0200
> From: Andreas Röhler <andreas.roehler@online.de>
> CC: Eli Zaretskii <eliz@gnu.org>
> 
> (defvar myov2 (make-overlay 29 43))
> (defvar myov1 (make-overlay 16 51))
> (overlay-put myov2 'display "STRING2")
> (overlay-put myov1 'display "STRING1")
> 
> ie if the wider overlay is applied last, it should take all.

Why would the order of evaluation matter here?  After everything is
evaluated, the display engine just sees a buffer with two overlays
it's got to display.  How is it to know which one was applied first,
and why should it care?

RE: Nested display strings

[Drew Adams]

> (defvar myov1 (make-overlay 16 51))
> (overlay-put myov1 'display "STRING1")
> (defvar myov2 (make-overlay 29 43))
> (overlay-put myov2 'display "STRING2")
> 
> If you evaluate this in the *scratch* buffer, the result is this:
> ;; This buffer STRING1STRING2STRING1, and for Lisp evaluation.
> That is, STRING1 is displayed twice!  Is this a bug or a feature?
> 
> I asked Gerd Möllmann, and he told me that this is an accident: what
> he really meant was, once STRING1 is displayed, to skip to the end of
> its overlay, so that the nested overlay myov2 would not be displayed
> at all.

Don't `display' and `priority' get along well, or can't they be be made to get
along?

We have two overlapping overlays here, and the usual way to determine what
happens when overlays overlap is by looking at their `priority' values (among
other things).

> However, after so many years (this is how Emacs behaves since v21.1),
> perhaps we should keep this behavior.  Does anyone know any real-life
> use cases where this "feature" is needed or would be beneficial?

Dunno whether any existing code depends on the current behavior.  But I would
tend to think of both the current behavior and what Gerd proposed as less than
ideal.

If `display' and `priority' can be made to cooperate then that would be good.
In that case, the behavior would be consistent with other uses of overlapping
overlays.

Actually, the doc seems to suggest the behavior that I would expect.
(elisp) `Display Property' says:

"If several sources (overlays and/or a text property) specify values
for the `display' property, only one of the values takes effect,
following the rules of `get-char-property'.  *Note Examining
Properties::."

And (elisp) `Examining Properties' says this about `get-char-property':

"If OBJECT is a buffer, then overlays in that buffer are
considered first, in order of decreasing priority, followed by the text
properties."

IOW, it seems to say that even in the case of overlapping overlays, each with a
`display' property, one of the overlays wins completely, and deciding which one
wins involves taking `priority' into account.

If the doc is taken as what should happen, then the current behavior would seem
to represent a bug.  Gerd's proposed behavior apparently also conflicts with the
documented behavior: `priority', not just first-come-first-served, should govern
what happens.

You raise a good question, in any case.

Re: Nested display strings

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Date: Sat, 23 Apr 2011 13:31:47 -0700
> 
> Don't `display' and `priority' get along well, or can't they be be made to get
> along?
> 
> We have two overlapping overlays here, and the usual way to determine what
> happens when overlays overlap is by looking at their `priority' values (among
> other things).

In the example that started this thread, there were no priorities at
all.  The question is, whether the behavior in that case is correct or
not.

But even if different priorities were involved, how do they help
resolve the issue with overlays that have `display' property whose
value is a string or an image?  These overlay properties _replace_ the
buffer text, not determine how to display the buffer text.  If the two
overlays were covering the same range of buffer positions, then we
could expect that the overlay with the higher priority would "win".
But what to do in the case I presented, where one of the overlays
covers only a subset of positions covered by the other?  How can
priorities help in that case?  If the "shorter" overlay has a higher
priority, what to display instead of the characters covered only by
the "longer" one?  Either nothing or you get the duplicate display of
STRING1 again.  Both alternatives look bad.

> If `display' and `priority' can be made to cooperate then that would be good.

I don't see how could they; see above.

RE: Nested display strings

[Drew Adams]

> > Don't `display' and `priority' get along well, or can't 
> > they be be made to get along?
> > 
> > We have two overlapping overlays here, and the usual way to 
> > determine what happens when overlays overlap is by looking
> > at their `priority' values (among other things).
> 
> In the example that started this thread, there were no priorities at
> all.

I realize that you said nothing about there being `priority' values present.

My point was that using priorities would normally be a way for users to handle
such conflicts.  That assumes that priorities actually function normally with
overlapping `display' overlays.

> The question is, whether the behavior in that case is
> correct or not.

That is certainly one question.  But not the most important one, to my naive
eyes.  If priorities do in fact work in this context (dunno), then whatever the
answer to your question the user (programmer) should have a way to manipulate
the behavior.

> But even if different priorities were involved, how do they help
> resolve the issue with overlays that have `display' property whose
> value is a string or an image?  These overlay properties _replace_ the
> buffer text, not determine how to display the buffer text.

Reading the doc suggests (to me) that whichever of the overlays at a given
position has higher priority controls the display behavior at that position.
The string or image value of the higher-priority overlay would win at that
position.

That's one naive reading of the doc, and it doesn't mean that there wouldn't be
problems in practice.  You likely wouldn't want different chars of contiguous
text to each display a different image, back and forth!  Still, it seems like
you could control the behavior, given the documented rule, by applying
priorities sanely.  Dunno.

> If the two overlays were covering the same range of buffer positions,
> then we could expect that the overlay with the higher priority would
> "win".

Yes, that's what I was thinking.  And any overlap is "the same range of buffer
positions".  An overlay can cover more than the overlapped range, but within
that overlap range what you say should apply (in principle), no?

> But what to do in the case I presented, where one of the overlays
> covers only a subset of positions covered by the other?  

At any given position there is one overlay or two.  If two, then at that
position the overlay with the higher priority wins (at least that's my reading
of the doc).

If a higher-priority overlay is strictly nested within a lower-priority one,
then I would naively expect the lower one to win outside the overlap and the
higher one to win inside it.

If the higher-priority overlay is the wider one, then I would expect the
lower-priority one never to show.

I would think of this in terms of visual layers (until I discovered that such an
analogy might not always help...).

> How can priorities help in that case?

See above for an attempt.

> If the "shorter" overlay has a higher priority, what to display
> instead of the characters covered only by the "longer" one?

Dunno what you mean by "instead of the characters covered only by the "longer"
one".  The shorter one only has an effect on the chars that it is applied to,
no?  What am I missing (probably a lot)?

> Either nothing or you get the duplicate display of
> STRING1 again.  Both alternatives look bad.

What appearance/behavior is it that you are aiming at?  If someone uses
overlapping overlays then s?he has to decide what happens at the overlap.  Emacs
generally decides this by priority value.  That's a simple approach.  It doesn't
mean you can't end up with "bad" looking displays.

Perhaps give an example of the behavior/appearance that you want to achieve.
It's hard to imagine what might or might not work if we don't know what you're
aiming at.
 
> > If `display' and `priority' can be made to cooperate then 
> > that would be good.
> 
> I don't see how could they; see above.

Ditto - see above.

Again, I'm no expert on this.  I'm just reading the doc and thinking that
overlay priority makes sense in the `display' case also.  (Whether it works in
practice is another story.)

An overlay specifies display characteristics at given buffer positions.  IIUC, a
determination is made (logically at least) at each position as to what should be
displayed for that position (e.g., in place of the char at that position).

I don't have a position on this.  Feel free to ignore if I'm missing the boat.
But from (just) a reading of the doc it doesn't seem so complicated (in
principle).

Re: Nested display strings

[Stefan Monnier]

> I asked Gerd Möllmann, and he told me that this is an accident: what
> he really meant was, once STRING1 is displayed, to skip to the end of
> its overlay, so that the nested overlay myov2 would not be displayed
> at all.

There's indeed little doubt that this is "as designed", it's probably
just inherited from the view of char-properties as a minor variant of
text-properties.
The result makes some sense but AFAIK in all the cases I bumped into it
it seemed like a bug rather than a feature.
So I think it's OK to consider this case as "unsupported", so any
behavior is acceptable.  Maybe the best would be to detect the situation
and signal some kind of warning.


        Stefan

Re: Nested display strings

[Michael Welsh Duggan]

Eli Zaretskii <eliz@gnu.org> writes:

> The following 4 lines define two overlays with display strings, with
> the second overlay "nested" inside the first one:
>
> (defvar myov1 (make-overlay 16 51))
> (overlay-put myov1 'display "STRING1")
> (defvar myov2 (make-overlay 29 43))
> (overlay-put myov2 'display "STRING2")
>
> If you evaluate this in the *scratch* buffer, the result is this:
>
> ;; This buffer STRING1STRING2STRING1, and for Lisp evaluation.
>
> That is, STRING1 is displayed twice!  Is this a bug or a feature?
>
> I asked Gerd Möllmann, and he told me that this is an accident: what
> he really meant was, once STRING1 is displayed, to skip to the end of
> its overlay, so that the nested overlay myov2 would not be displayed
> at all.

I'm of the opinion that it should either display: STRING1 or
STRING1STRING2STRING1, and which doesn't really matter.  In the first
case, the first overlay gets a higher priority than the second, in the
latter the second gets one that is higher than the first.  The manual
says: "You should not make assumptions about which overlay will prevail
when there is a conflict and they have the same priority."  This is what
makes me think that either is correct.  If I had to choose, however, I
would prefer that if two overlays with the same priority appear over the
same character, whichever overlay that has the earlier start should take
precedence (giving us STRING1 in this case).

Please note, that if myov1 had a specifically higher priority than
myov2, I think STRING1STRING2STRING1 is a perfectly valid result.  I see
overlays and text properties reducing down to a single set of property
ranges, with text properties at a sort-of -1 priority.  In this case we
have a range with a property of STRING1, a range with a property of
STRING2, and a range with property of STRING1 again.  The display makes
perfect sense to me.

-- 
Michael Welsh Duggan
(md5i@md5i.com)

Re: Nested display strings

[Eli Zaretskii]

> From: Michael Welsh Duggan <md5i@md5i.com>
> Cc: emacs-devel@gnu.org
> Date: Sun, 24 Apr 2011 01:19:38 -0400
> 
> I'm of the opinion that it should either display: STRING1 or
> STRING1STRING2STRING1, and which doesn't really matter.

That is, you think that doing that without setting priorities invokes
"undefined behavior".

> If I had to choose, however, I would prefer that if two overlays
> with the same priority appear over the same character, whichever
> overlay that has the earlier start should take precedence (giving us
> STRING1 in this case).

I tend to agree.

> Please note, that if myov1 had a specifically higher priority than
> myov2, I think STRING1STRING2STRING1 is a perfectly valid result.

You mean, if myov2 has higher priority, right?  Because if myov1 has
higher priority, we display just STRING1, as expected.

Re: Nested display strings

[Michael Welsh Duggan]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Michael Welsh Duggan <md5i@md5i.com>
>> Cc: emacs-devel@gnu.org
>> Date: Sun, 24 Apr 2011 01:19:38 -0400
>> 
>> I'm of the opinion that it should either display: STRING1 or
>> STRING1STRING2STRING1, and which doesn't really matter.
>
> That is, you think that doing that without setting priorities invokes
> "undefined behavior".
>
>> If I had to choose, however, I would prefer that if two overlays
>> with the same priority appear over the same character, whichever
>> overlay that has the earlier start should take precedence (giving us
>> STRING1 in this case).
>
> I tend to agree.
>
>> Please note, that if myov1 had a specifically higher priority than
>> myov2, I think STRING1STRING2STRING1 is a perfectly valid result.
>
> You mean, if myov2 has higher priority, right?  Because if myov1 has
> higher priority, we display just STRING1, as expected.

Yes, indeed.  Sorry about that.

-- 
Michael Welsh Duggan
(md5i@md5i.com)

Re: Nested display strings

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Cc: <emacs-devel@gnu.org>
> Date: Sat, 23 Apr 2011 16:20:04 -0700
> 
> > But what to do in the case I presented, where one of the overlays
> > covers only a subset of positions covered by the other?  
> 
> At any given position there is one overlay or two.

Unlike most other properties, the `display' property does not affect
each character separately.  It replaces them all as a single unit.  So
asking about "any given position" is meaningless in this case.  As you
say, having the properties alternate back and forth on adjacent
characters would produce nonsensical display, if we were to interpret
the feature that way.

> If a higher-priority overlay is strictly nested within a lower-priority one,
> then I would naively expect the lower one to win outside the overlap and the
> higher one to win inside it.
> 
> If the higher-priority overlay is the wider one, then I would expect the
> lower-priority one never to show.

This is how it works now, but I don't think it makes much sense to
show a display string more than once.  I think this should work as in
the latter case (i.e. the "narrower" overlay is never seen),
regardless of the priorities.

Re: Nested display strings

[Andreas Röhler]

Am 24.04.2011 07:19, schrieb Michael Welsh Duggan:
> Eli Zaretskii<eliz@gnu.org>  writes:
>
>> The following 4 lines define two overlays with display strings, with
>> the second overlay "nested" inside the first one:
>>
>> (defvar myov1 (make-overlay 16 51))
>> (overlay-put myov1 'display "STRING1")
>> (defvar myov2 (make-overlay 29 43))
>> (overlay-put myov2 'display "STRING2")
>>
>> If you evaluate this in the *scratch* buffer, the result is this:
>>
>> ;; This buffer STRING1STRING2STRING1, and for Lisp evaluation.
>>
>> That is, STRING1 is displayed twice!  Is this a bug or a feature?
>>
>> I asked Gerd Möllmann, and he told me that this is an accident: what
>> he really meant was, once STRING1 is displayed, to skip to the end of
>> its overlay, so that the nested overlay myov2 would not be displayed
>> at all.
>
> I'm of the opinion that it should either display: STRING1 or
> STRING1STRING2STRING1, and which doesn't really matter.  In the first
> case, the first overlay gets a higher priority than the second, in the
> latter the second gets one that is higher than the first.  The manual
> says: "You should not make assumptions about which overlay will prevail
> when there is a conflict and they have the same priority."  This is what
> makes me think that either is correct.  If I had to choose, however, I
> would prefer that if two overlays with the same priority appear over the
> same character, whichever overlay that has the earlier start should take
> precedence (giving us STRING1 in this case).

Which would be the reverse logic from setting values in nested let's for 
example.

Suggest giving the last setting precedence, in cases 'priorty can't decide.

Cheers

Andreas


>
> Please note, that if myov1 had a specifically higher priority than
> myov2, I think STRING1STRING2STRING1 is a perfectly valid result.  I see
> overlays and text properties reducing down to a single set of property
> ranges, with text properties at a sort-of -1 priority.  In this case we
> have a range with a property of STRING1, a range with a property of
> STRING2, and a range with property of STRING1 again.  The display makes
> perfect sense to me.
>

Re: Nested display strings

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: emacs-devel@gnu.org
> Date: Sun, 24 Apr 2011 02:01:34 -0300
> 
> > I asked Gerd Möllmann, and he told me that this is an accident: what
> > he really meant was, once STRING1 is displayed, to skip to the end of
> > its overlay, so that the nested overlay myov2 would not be displayed
> > at all.
> 
> There's indeed little doubt that this is "as designed", it's probably
> just inherited from the view of char-properties as a minor variant of
> text-properties.
> The result makes some sense but AFAIK in all the cases I bumped into it
> it seemed like a bug rather than a feature.
> So I think it's OK to consider this case as "unsupported", so any
> behavior is acceptable.  Maybe the best would be to detect the situation
> and signal some kind of warning.

What about the case when the "inner" overlay has higher priority than
the "outer" one?  That is, this variant:

(defvar myov1 (make-overlay 16 51))
(defvar myov2 (make-overlay 29 43))
(overlay-put myov1 'display "STRING1")
(overlay-put myov2 'display "STRING2")
(overlay-put myov1 'priority 1)
(overlay-put myov2 'priority 2)

A number of people expressed their opinion that in this case, it is
desirable to show STRING1STRING2STRING1.  (I don't think it's right,
but that's me.)  If we want to support that, then we cannot simply
jump to the end of the overlay, as the simple change below does.  With
that change, we display STRING1 once, and STRING2 is not displayed at
all, irrespective of priorities.

=== modified file 'src/xdisp.c'
--- src/xdisp.c	2011-04-13 17:41:04 +0000
+++ src/xdisp.c	2011-04-24 17:26:30 +0000
@@ -790,7 +790,7 @@ static int resize_mini_window_1 (EMACS_I
 static Lisp_Object unwind_redisplay (Lisp_Object);
 static int string_char_and_length (const unsigned char *, int *);
 static struct text_pos display_prop_end (struct it *, Lisp_Object,
-                                         struct text_pos);
+                                         struct text_pos, Lisp_Object);
 static int compute_window_start_on_continuation_line (struct window *);
 static Lisp_Object safe_eval_handler (Lisp_Object);
 static void insert_left_trunc_glyphs (struct it *);
@@ -3882,18 +3882,25 @@ handle_display_prop (struct it *it)
    at START_POS in OBJECT.  */
 
 static struct text_pos
-display_prop_end (struct it *it, Lisp_Object object, struct text_pos start_pos)
+display_prop_end (struct it *it, Lisp_Object object, struct text_pos start_pos,
+		  Lisp_Object overlay)
 {
   Lisp_Object end;
   struct text_pos end_pos;
 
-  end = Fnext_single_char_property_change (make_number (CHARPOS (start_pos)),
+  if (!NILP (overlay))
+    CHARPOS (end_pos) = OVERLAY_POSITION (OVERLAY_END (overlay));
+  else
+    {
+      end =
+	Fnext_single_char_property_change (make_number (CHARPOS (start_pos)),
 					   Qdisplay, object, Qnil);
-  CHARPOS (end_pos) = XFASTINT (end);
+      CHARPOS (end_pos) = XFASTINT (end);
+    }
   if (STRINGP (object))
     compute_string_pos (&end_pos, start_pos, it->string);
   else
-    BYTEPOS (end_pos) = CHAR_TO_BYTE (XFASTINT (end));
+    BYTEPOS (end_pos) = CHAR_TO_BYTE (CHARPOS (end_pos));
 
   return end_pos;
 }
@@ -4099,7 +4106,7 @@ handle_single_display_spec (struct it *i
   /* Characters having this form of property are not displayed, so
      we have to find the end of the property.  */
   start_pos = *position;
-  *position = display_prop_end (it, object, start_pos);
+  *position = display_prop_end (it, object, start_pos, overlay);
   value = Qnil;
 
   /* Stop the scan at that end position--we assume that all

Re: Nested display strings

[Stefan Monnier]

> What about the case when the "inner" overlay has higher priority than
> the "outer" one?

Same problem: for two overlays of equal `priority', the shorter one has
higher priority, so your original example is already one of those
cases, AFAIC.

The main issue is that a single overlays with a `display' property
normally means "display that thing *once*".  So while we can argue of
the case where the two overlays share the start or end position, in the
case where the inner overlays is really "in the middle", there's no
clearly correct way to render it.

So I think the best behavior is to detect the situation and signal
some kind of warning/error (and behave in whichever way is most
convenient for the C code).


        Stefan

Re: Nested display strings

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: emacs-devel@gnu.org
> Date: Mon, 25 Apr 2011 09:47:23 -0300
> 
> The main issue is that a single overlays with a `display' property
> normally means "display that thing *once*".  So while we can argue of
> the case where the two overlays share the start or end position, in the
> case where the inner overlays is really "in the middle", there's no
> clearly correct way to render it.

I see.  So in this modified use case, where the overlays start at the
same position:

(defvar myov1 (make-overlay 16 51))
(defvar myov2 (make-overlay 16 43))
(overlay-put myov1 'display "STRING1")
(overlay-put myov2 'display "STRING2")
(overlay-put myov1 'priority 1)
(overlay-put myov2 'priority 2)

you think it's okay to display

;; This buffer STRING2STRING1, and for Lisp evaluation.

?

And in the opposite case, where the overlays end at the same position:

(defvar myov1 (make-overlay 16 51))
(defvar myov2 (make-overlay 29 51))
(overlay-put myov1 'display "STRING1")
(overlay-put myov2 'display "STRING2")
(overlay-put myov1 'priority 1)
(overlay-put myov2 'priority 2)

it is okay to display

;; This buffer STRING1, and for Lisp evaluation.

?

This is what happens with the change I showed in my previous message.
(With the current code, we display STRING2STRING1 in the first case
and STRING1STRING2 in the second.)

If the modified display is deemed okay, I will install that change
(and probably add something to the docs to mention these caveats).

> So I think the best behavior is to detect the situation and signal
> some kind of warning/error

Hmm.. how do you signal an error during redisplay?  I can do something
like this:

    add_to_log ("Warning: overlapping `display' strings at pos %d",
                make_number (pos));

Would that be sufficient?

Re: Nested display strings

[Stefan Monnier]

>> The main issue is that a single overlays with a `display' property
>> normally means "display that thing *once*".  So while we can argue of
>> the case where the two overlays share the start or end position, in the
>> case where the inner overlays is really "in the middle", there's no
>> clearly correct way to render it.

> I see.  So in this modified use case, where the overlays start at the
> same position:

> (defvar myov1 (make-overlay 16 51))
> (defvar myov2 (make-overlay 16 43))
> (overlay-put myov1 'display "STRING1")
> (overlay-put myov2 'display "STRING2")
> (overlay-put myov1 'priority 1)
> (overlay-put myov2 'priority 2)

> you think it's okay to display

> ;; This buffer STRING2STRING1, and for Lisp evaluation.

> ?

I'm not sure if it's OK, but at least there's an argument that it
doesn't break the "display once per overlay" rule.

> And in the opposite case, where the overlays end at the same position:

> (defvar myov1 (make-overlay 16 51))
> (defvar myov2 (make-overlay 29 51))
> (overlay-put myov1 'display "STRING1")
> (overlay-put myov2 'display "STRING2")
> (overlay-put myov1 'priority 1)
> (overlay-put myov2 'priority 2)

> it is okay to display

> ;; This buffer STRING1, and for Lisp evaluation.

> ?

It seems inconsistent, but I think we should consider both cases
unsupported, so the inconsistency is not too serious.

> If the modified display is deemed okay, I will install that change
> (and probably add something to the docs to mention these caveats).

I think it's OK.

>> So I think the best behavior is to detect the situation and signal
>> some kind of warning/error

> Hmm.. how do you signal an error during redisplay?  I can do something
> like this:

>     add_to_log ("Warning: overlapping `display' strings at pos %d",
>                 make_number (pos));

> Would that be sufficient?

There was recently a discussion of how to signal warnings/errors in
a delayed fashion.  We don't have a good mechanism for it currently, but
it would be good to add one.
In the mean time, the above is better than nothing.


        Stefan

Re: Nested display strings

[Juanma Barranquero]

On Mon, Apr 25, 2011 at 16:57, Stefan Monnier <monnier@iro.umontreal.ca> wrote:


> There was recently a discussion of how to signal warnings/errors in
> a delayed fashion.

> In the mean time, the above is better than nothing.

Yes, and there were a couple of mechanisms proposed, but none was
committed for lack of a decission...


> We don't have a good mechanism for it currently, but
> it would be good to add one.

Definitely.

    Juanma

Re: Nested display strings

[Chong Yidong]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> So I think the best behavior is to detect the situation and signal
> some kind of warning/error (and behave in whichever way is most
> convenient for the C code).

I don't think there is a non-obnoxious way for redisplay to emit errors
or warnings in such a situation.  It's probably better to leave the
current behavior as-is.

Re: Nested display strings

[Eli Zaretskii]

> From: Chong Yidong <cyd@stupidchicken.com>
> Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org
> Date: Mon, 25 Apr 2011 15:41:55 -0400
> 
> I don't think there is a non-obnoxious way for redisplay to emit errors
> or warnings in such a situation.

So you object to adding a warning to *Messages*?

> It's probably better to leave the current behavior as-is.

You mean, to continue displaying the "outer" display string twice?
Surely, not being able to display a warning doesn't mean we need to
keep the current behavior.  We could behave differently (silently),
can't we?

Re: Nested display strings

[Stefan Monnier]

>> So I think the best behavior is to detect the situation and signal
>> some kind of warning/error (and behave in whichever way is most
>> convenient for the C code).
> I don't think there is a non-obnoxious way for redisplay to emit errors
> or warnings in such a situation.

It's tricky, indeed since redisplay will be performed repeatedly an will
tend to signal such warnings over and over again.  A good/natural way
for redisplay to signal such problems would be to display them directly
in the window (e.g. insert some threatening question mark or some such,
ideally with mouse-over highlighting and a help-echo-like text
explaining the problem).


        Stefan

Re: Nested display strings

[Richard Stallman]

The issue of what to do in the case of overlapping libraries,
like that of nested display strings, was clouded by the fact
that we didn't have a lot of real use cases to show us what
behavior was really useful,

If we have various use cases which need different behavior, perhaps we will
see that there are distinct kinds of uses that need distinct behavior;
perhaps they need slightly different features.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org, www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.

Re: Nested display strings

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>,  emacs-devel@gnu.org
> Date: Tue, 26 Apr 2011 09:53:04 -0300
> 
> A good/natural way for redisplay to signal such problems would be to
> display them directly in the window (e.g. insert some threatening
> question mark or some such, ideally with mouse-over highlighting and
> a help-echo-like text explaining the problem).

Now, _that_ is what I call "obnoxious".

I actually thought about queuing a message for later, but someone
would need to implement such a facility first.

Re: Nested display strings

[Chong Yidong]

Eli Zaretskii <eliz@gnu.org> writes:

>> It's probably better to leave the current behavior as-is.
>
> You mean, to continue displaying the "outer" display string twice?
> Surely, not being able to display a warning doesn't mean we need to
> keep the current behavior.  We could behave differently (silently),
> can't we?

Does changing this make it easier for bidi to do the right thing?

If not, the "correct" interpretation is ambiguous enough that it doesn't
seem worthwhile for a change in existing behavior.

Re: Nested display strings

[Ted Zlatanov]

On Tue, 26 Apr 2011 20:51:51 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Stefan Monnier <monnier@iro.umontreal.ca>
>> Cc: Eli Zaretskii <eliz@gnu.org>,  emacs-devel@gnu.org
>> Date: Tue, 26 Apr 2011 09:53:04 -0300
>> 
>> A good/natural way for redisplay to signal such problems would be to
>> display them directly in the window (e.g. insert some threatening
>> question mark or some such, ideally with mouse-over highlighting and
>> a help-echo-like text explaining the problem).

EZ> Now, _that_ is what I call "obnoxious".

EZ> I actually thought about queuing a message for later, but someone
EZ> would need to implement such a facility first.

Maybe it could be a "something is broken" indicator in the modeline,
which when clicked pops up a help buffer with the list of problems.  It
could be used for general problems with display and font-lock, which
IIRC are sometimes hard to debug.

Ted

Re: Nested display strings

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Tue, 26 Apr 2011 13:13:54 -0500
> 
> Maybe it could be a "something is broken" indicator in the modeline,
> which when clicked pops up a help buffer with the list of problems.

The idea is nice, but it won't work on a text terminal without a
mouse, whereas overlays with display strings do.

Re: Nested display strings

[Eli Zaretskii]

> From: Chong Yidong <cyd@stupidchicken.com>
> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> Date: Tue, 26 Apr 2011 14:04:30 -0400
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> It's probably better to leave the current behavior as-is.
> >
> > You mean, to continue displaying the "outer" display string twice?
> > Surely, not being able to display a warning doesn't mean we need to
> > keep the current behavior.  We could behave differently (silently),
> > can't we?
> 
> Does changing this make it easier for bidi to do the right thing?

Yes, most probably (although I cannot be 100% sure until I sit down to
write the code).

Re: Nested display strings

[chad]

On Apr 26, 2011, at 11:25 AM, Eli Zaretskii wrote:

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Tue, 26 Apr 2011 13:13:54 -0500
>> 
>> Maybe it could be a "something is broken" indicator in the modeline,
>> which when clicked pops up a help buffer with the list of problems.
> 
> The idea is nice, but it won't work on a text terminal without a
> mouse, whereas overlays with display strings do.

How about just a mode-line indicator for ``Something has been added
 to *Messages* without appearing in the echo area''?  

As a starter proposal, we could make something that turns many/all 
of the -'s from %- in mode-line format into !'s, and then bind a command 
to C-h ! that displays the messages.

If color is available, we could splash some on.  If mouse support is 
available, make it clickable.  If tooltips... etc.

*Chad

Re: Nested display strings

[Ted Zlatanov]

On Tue, 26 Apr 2011 11:35:32 -0700 chad <yandros@MIT.EDU> wrote: 

c> On Apr 26, 2011, at 11:25 AM, Eli Zaretskii wrote:

>>> From: Ted Zlatanov <tzz@lifelogs.com>
>>> Date: Tue, 26 Apr 2011 13:13:54 -0500
>>> 
>>> Maybe it could be a "something is broken" indicator in the modeline,
>>> which when clicked pops up a help buffer with the list of problems.
>> 
>> The idea is nice, but it won't work on a text terminal without a
>> mouse, whereas overlays with display strings do.

The modeline and indicators in it should work in a text terminal; in
that case all "broken display" messages should go just to *Messages* and
there won't be any clicking.  I think that's OK.

So to issue one of these special "broken display" messages you'd call
`message-broken-display' which in turn calls `message' and does the
indicator setup.

c> How about just a mode-line indicator for ``Something has been added
c>  to *Messages* without appearing in the echo area''?  

...with a message count, which is reset to 0 on redisplay, so it shows
how many problems from `message-broken-display' have shown up since then?

Ted

RE: Nested display strings

[Drew Adams]

> >> Maybe it could be a "something is broken" indicator in the 
> >> modeline, which when clicked pops up a help buffer with the
> >> list of problems.
> > 
> > The idea is nice, but it won't work on a text terminal without a
> > mouse, whereas overlays with display strings do.
> 
> How about just a mode-line indicator for ``Something has been added
> to *Messages* without appearing in the echo area''?  
> 
> As a starter proposal, we could make something that turns many/all 
> of the -'s from %- in mode-line format into !'s, and then 
> bind a command to C-h ! that displays the messages.
> 
> If color is available, we could splash some on.  If mouse support is 
> available, make it clickable.  If tooltips... etc.

Splash some on, indeed.  It's important to get the user's attention for this.

Maybe it could turn on the washing machine, the dishwasher, and the garbage
disposal; then open the garage door, fill the bathtub, and ring the alarm clock.

Users could optionally hook in additional annoyances to attract their attention:
turn on the sprinklers, open the windows, start a fire in the fireplace...

EMACS: Endlessly Manipulating Annoying Catastrophe Signals.

Re: Nested display strings

[Chong Yidong]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> So I think the best behavior is to detect the situation and signal
>>> some kind of warning/error (and behave in whichever way is most
>>> convenient for the C code).
>> I don't think there is a non-obnoxious way for redisplay to emit errors
>> or warnings in such a situation.
>
> It's tricky, indeed since redisplay will be performed repeatedly an will
> tend to signal such warnings over and over again.  A good/natural way
> for redisplay to signal such problems would be to display them directly
> in the window (e.g. insert some threatening question mark or some such,
> ideally with mouse-over highlighting and a help-echo-like text
> explaining the problem).

If such redisplay warnings are really important (which I'm still not
sure about), we could provide a Lisp-callable "debug-redisplay" built-in
command that triggers one single redisplay, accumulates all warning
messages, and puts them into a buffer afterwards.  Except when this
command is running, such redisplay warning would have no effect.  This
would help a developer who is, say, debugging a troublesome overlay,
without annoying users in other contexts.

Re: Nested display strings

[Stefan Monnier]

> If such redisplay warnings are really important (which I'm still not
> sure about), we could provide a Lisp-callable "debug-redisplay" built-in
> command that triggers one single redisplay, accumulates all warning
> messages, and puts them into a buffer afterwards.  Except when this
> command is running, such redisplay warning would have no effect.  This
> would help a developer who is, say, debugging a troublesome overlay,
> without annoying users in other contexts.

The most important thing is that it's not worth spending much time
on it, tho.


        Stefan

Re: Nested display strings

[Stefan Monnier]

>> A good/natural way for redisplay to signal such problems would be to
>> display them directly in the window (e.g. insert some threatening
>> question mark or some such, ideally with mouse-over highlighting and
>> a help-echo-like text explaining the problem).
> Now, _that_ is what I call "obnoxious".

I have no idea why you'd find it obnoxious.  After all, that's pretty
much what we do for "chars we don't know how to draw", except we put
a square box rather than a threatening question mark and we don't bother
with the help-echo and mouse highlight.
Admittedly, the mouse-highlight is probably not a good idea, but the
rest sounds really like the ideal solution to me.


        Stefan

Re: Nested display strings

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: cyd@stupidchicken.com,  emacs-devel@gnu.org
> Date: Wed, 27 Apr 2011 21:45:17 -0300
> 
> >> A good/natural way for redisplay to signal such problems would be to
> >> display them directly in the window (e.g. insert some threatening
> >> question mark or some such, ideally with mouse-over highlighting and
> >> a help-echo-like text explaining the problem).
> > Now, _that_ is what I call "obnoxious".
> 
> I have no idea why you'd find it obnoxious.  After all, that's pretty
> much what we do for "chars we don't know how to draw", except we put
> a square box rather than a threatening question mark and we don't bother
> with the help-echo and mouse highlight.

Exactly.  The things we do NOT do in the square box case are that make
your suggestion too extreme, to my taste.  The situation is not so
threatening, after all.

Re: Nested display strings

[Stefan Monnier]

>> >> A good/natural way for redisplay to signal such problems would be to
>> >> display them directly in the window (e.g. insert some threatening
>> >> question mark or some such, ideally with mouse-over highlighting and
>> >> a help-echo-like text explaining the problem).
>> > Now, _that_ is what I call "obnoxious".
>> I have no idea why you'd find it obnoxious.  After all, that's pretty
>> much what we do for "chars we don't know how to draw", except we put
>> a square box rather than a threatening question mark and we don't bother
>> with the help-echo and mouse highlight.
> Exactly.  The things we do NOT do in the square box case are that make
> your suggestion too extreme, to my taste.  The situation is not so
> threatening, after all.

Well, I mostly want it to be easy for users to recognize that it's
a redisplay problem, and also to figure out what's the cause of
the problem, hence the help-echo.  I was thinking of a mouse
highlighting to make it into a hyperlink, but I don't think we'd know
where to jump to, so that's probably a bad idea.


        Stefan