bug#22320: Overlays with an 'invisible property break stacking of overlay faces

[Eli Zaretskii <eliz@gnu.org>]

> Cc: 22320@debbugs.gnu.org
> From: Clément Pit--Claudel <clement.pitclaudel@live.com>
> Date: Thu, 7 Jan 2016 19:27:30 -0500
> 
> 1. The overlay does not intersect with the invisible area. No problems here.
> 2. The overlay covers a few characters before the invisible area and the first few characters of the invisible area, but not the last ones.

In this case, Emacs doesn't care where the overlay ends, that buffer
position is skipped without checking overlays.

> 3. The overlay covers a few characters after the invisible area and the last few characters of the invisible area, but not the first ones.

Likewise.

> 4. The overlay is a subset of the invisible area.

If it starts on the first invisible character, Emacs will take note;
otherwise it won't.

> 5. The overlay is a superset of the invisible area.

This is the case with all your use cases.

> The current solution doesn't think in these terms; instead, it ignores all overlays for the ellipses, unless the first hidden character has the same face as the preceding one.
> 
> Always applying the face of the first character to the ellipsis means applying to the ellipsis the faces of overlays in categories 2, 5, and 4 if they cover the first character. You pointed out that the special case of 4 is confusing.
> 
> Another option (the one I was trying to explain above) is to apply to the ellipsis only the faces of overlays in category 5. I feel that this should be rather uncontroversial (don't you think?).

Well, now that I've committed what I believe is a better solution, I
think we can put these issues to rest.

> IIUC, the current implementation computes faces without taking invisibility into account for the first character before and after the left boundary of the invisible section, and then compares them. At this point it's too late to merge certain faces but not others.

Yes.  More accurately, the display engine always considers faces
before it considers invisibility, so it always sees the face of the
first invisible character.