Placing eol glyph (buffer-display-table) before an overlay after-string.

Placing eol glyph (buffer-display-table) before an overlay after-string.

[Keith David Bershatsky]

My goal is to use a pilcrow to represent line endings, and this option can be set up using the appropriate buffer-display-table entry.

I have a custom minor mode that draws cross-hairs the length of the screen to track the cursor position, which requires the use of the overlay after-string property.  Emacs draws the eol glyph after the overlay after-strings are placed -- i.e., to the right of the after-string.  The result is that the eol glyph (e.g., pilcrow) is always on the far right of the after-string overlay.  I want the eol glyph to appear at the end of the line of text, and the overlay after-string should come after the eol glyph (not before).

Is it feasible to reverse the order such that the eol glyph is laid before the overlays containing the after-string property?

A hint as to the locations (in the C code) where I would start working on reversing the order would be greatly appreciated.

If there is a better way to handle this than trying to reverse the order in which things are happening, any insight would be helpful.

WORKAROUND:  I have been using a custom XPM image of a pilcrow that is the beginning of the after-string, and I change its coloration (foreground/background) as needed.  The problem with this approach is that the XPM does not look as good as a native eol glpyh, and the XPM would need to be adjusted depending upon the font and size of the font.

Thanks,

Keith David Bershatsky

Re: Placing eol glyph (buffer-display-table) before an overlay after-string.

[Eli Zaretskii]

> Date: Fri, 11 Aug 2017 23:14:31 -0700
> From: Keith David Bershatsky <esq@lawlist.com>
> 
> Is it feasible to reverse the order such that the eol glyph is laid before the overlays containing the after-string property?

Not really.  The Emacs display engine considers a screen line ended at
the last glyph before the newline, disregarding the source of that
newline, which could be buffer text, a display or overlay string, or a
display table.  When the display engine finds itself at the end of a
screen line, it performs a set of bookkeeping operations and decisions
that are important for text layout, and must be done immediately prior
to the layout of the next screen line.  Displaying anything after that
bookkeeping, or somehow deferring this bookkeeping to later, would
need extensive changes in what is already very complex piece of code,
which handles several important issues, like
overflow-newline-into-fringe.  Current code assumes that no glyphs are
produced on the current screen line after the end-of-line was
consumed.

> A hint as to the locations (in the C code) where I would start working on reversing the order would be greatly appreciated.

There are 2 places: one in display_line, the other
move_it_in_display_line_to and move_it_to.  In all of those places,
look for uses of the macro ITERATOR_AT_END_OF_LINE_P.

Once again, I advise against such changes.

> If there is a better way to handle this than trying to reverse the order in which things are happening, any insight would be helpful.

Something like the workaround you use now:

> WORKAROUND:  I have been using a custom XPM image of a pilcrow that is the beginning of the after-string, and I change its coloration (foreground/background) as needed.  The problem with this approach is that the XPM does not look as good as a native eol glpyh, and the XPM would need to be adjusted depending upon the font and size of the font.

Why not use the real pilcrow character, e.g. as display string on the
first character of your overlay string?

Re: Placing eol glyph (buffer-display-table) before an overlay after-string.

[Yuri Khan]

On Sat, Aug 12, 2017 at 1:14 PM, Keith David Bershatsky <esq@lawlist.com> wrote:
> My goal is to use a pilcrow to represent line endings, and this option can be set up using the appropriate buffer-display-table entry.
>
> I have a custom minor mode that draws cross-hairs the length of the screen to track the cursor position, which requires the use of the overlay after-string property.  Emacs draws the eol glyph after the overlay after-strings are placed -- i.e., to the right of the after-string.  The result is that the eol glyph (e.g., pilcrow) is always on the far right of the after-string overlay.  I want the eol glyph to appear at the end of the line of text, and the overlay after-string should come after the eol glyph (not before).

That explanation would benefit from a screenshot.

I am using whitespace-mode to display a pilcrow for line endings, and
hl-line-mode to highlight the current line. For me, that setup works
and places the pilcrow correctly, although I don’t highlight the
current column.

Re: Placing eol glyph (buffer-display-table) before an overlay after-string.

[Eli Zaretskii]

> From: Yuri Khan <yuri.v.khan@gmail.com>
> Date: Sat, 12 Aug 2017 16:26:52 +0700
> Cc: Emacs developers <emacs-devel@gnu.org>
> 
> I am using whitespace-mode to display a pilcrow for line endings, and
> hl-line-mode to highlight the current line. For me, that setup works
> and places the pilcrow correctly, although I don’t highlight the
> current column.

whitespace-mode doesn't use overlay strings, and it doesn't try to
replace the visual representation of the newline.

Re: Placing eol glyph (buffer-display-table) before an overlay after-string.

[Yuri Khan]

On Sat, Aug 12, 2017 at 4:52 PM, Eli Zaretskii <eliz@gnu.org> wrote:

>> I am using whitespace-mode to display a pilcrow for line endings, and
>> hl-line-mode to highlight the current line. For me, that setup works
>> and places the pilcrow correctly, although I don’t highlight the
>> current column.
>
> whitespace-mode doesn't use overlay strings, and it doesn't try to
> replace the visual representation of the newline.

It sets up a display table with entries for tab, newline, and space.
And hl-line uses an overlay. hl-line’s overlay doesn’t use the
after-string property, though, which must be why they work together.

Re: Placing eol glyph (buffer-display-table) before an overlay after-string.

[Eli Zaretskii]

> From: Yuri Khan <yuri.v.khan@gmail.com>
> Date: Sat, 12 Aug 2017 17:18:35 +0700
> Cc: Keith David Bershatsky <esq@lawlist.com>,
> 	Emacs developers <emacs-devel@gnu.org>
> 
> > whitespace-mode doesn't use overlay strings, and it doesn't try to
> > replace the visual representation of the newline.
> 
> It sets up a display table with entries for tab, newline, and space.
> And hl-line uses an overlay. hl-line’s overlay doesn’t use the
> after-string property, though, which must be why they work together.

hl-line uses overlays only for setting the face of the displayed text,
not for displaying extra stuff via after-string.

The problem, AFAIU, is to display the after-string _after_ the glyph
which represents the newline.  Which currently cannot be done.

Re: Placing eol glyph (buffer-display-table) before an overlay after-string.

[Keith David Bershatsky]

[-- Attachment #1: Type: text/plain, Size: 4974 bytes --]

Thank you, Eli, for letting me know the exact locations responsible for the behavior at issue, and also for suggesting the use of a real pilcrow in the overlay after-string as a possible workaround.

I have attached a couple of screen-shots and links to the same uploaded images, which were requested by Yuri.  One screen-shot is *without* the pilcrow buffer-display-table entry and *with* the after-string (containing an XPM pilcrow).  The other screen-shot is *with* a pilcrow buffer-display-table and *with* the after-string (containing the XPM pilcrow) -- all of the pilcrows to the immediate right of the vertical line are the pilcrow buffer-display-table entries that were laid during redisplay subsequent to the after-string.

The cross-hairs are designed to be compatible when word-wrap is non-nil and truncate-lines is nil.  It involves time costly custom vertical-motion calculations on each screen line.  I have set up a list of functions that trigger drawing cross-hairs, and also a list of functions that trigger removing cross-hairs and temporarily inhibit new cross-hairs from being drawn.  The horizontal line of the cross-bar and eol pilcrows draw each command loop for things like self-insert-command, whereas the vertical line of the cross-bar is on an idle-timer.  There are a few functions that trigger unconditional drawing of horizontal/vertical lines for a complete cross-hairs, such as switching to a window where that minor-mode is active.  I use a more recent implementation of Emacs Bug feature #22404 to o
 btain definitive window-start/end values and the cross-hairs (with corresponding calculations) regarding same are limited to the bounds of the visible window.

The idea about using a regular pilcrow as part of the overlay after-string instead of an XPM would "probably" [untested] work if I use the new multiple fake cursors feature (Emacs Bug feature #22873) to achieve a vbar cursor through the pilcrow whenever the cursor vertically lines up with an eol pilcrow.  Prior to the invention of #22873, I achieved the vertical bar through the eol pilcrow with an XPM that can change colors at different areas of the pilcrow depending upon the situation -- e.g., odd/even numbered columns, fill-column and beyond, last point at right window edge, and wrapped lines subsequent to the first wrapped screen line.  The hbar fake cursor can also be used with a real pilcrow in lieu of the horizontal line that I have been creating as part of the XPM pilcrow.

To save time with certain rapid-fire commands like scrolling and moving the real cursor around, I erase the cross-hairs overlays and inhibit vertical-motion calculation.  The pilcrows disappear as a result.  The visual affect of the pilcrows disappearing while the cross-hairs are temporarily inhibited from being drawn is something that I have never been able to get accustomed to seeing.  I previously experimented with temporarily removing the XPM pilcrows and temporarily enabling the buffer-display-table pilcrows for rapid-fire functions (e.g., scrolling and cursor movement), but there was a time cost and I seem to remember another hitch/glitch in involved in timing and also the visual effect when subsequently removing the buffer-display-table entry and re-enabling the XPM overlays.

Keeping the eol pilcrows visible when the cross-hairs are disabled is my preferred behavior (if possible), and that is what lead me to launching this particular thread.

Here are the two flavors of XPM pilcrows that I presently use for the custom cross-hairs minor-mode:

(defvar xpm--pilcrow
"/* XPM */
static char * pilcrow_special_xpm[] = {
\"11 20 10 1\",
\"+ s col1\",
\". s col2\",
\"% s col3\",
\"@ s col4\",
\"& s col5\",
\"# s col6\",
\"^ s col7\",
\"! s col8\",
\"$ s col9\",
\"* s col10\",
\"%.........@\",
\"%.........@\",
\"%.........@\",
\"%.........@\",
\"%.........@\",
\"%..!!!!!!.@\",
\"%.!!!!.!..@\",
\"%.!!!!.!..@\",
\"%.!!!!.!..@\",
\"%..!!!.!..@\",
\"%....!.!..@\",
\"%....!.!..@\",
\"%....!.!..@\",
\"%....!.!..@\",
\"%....!.!..@\",
\"%....!.!..@\",
\"%.........@\",
\"%.........@\",
\"&+++++++++#\",
\"%.........@\"};")

(defvar pilcrow
"/* XPM */
static char * pilcrow_regular_xpm[] = {
\"11 20 10 1\",
\"+ s col1\",
\". s col2\",
\"% s col3\",
\"@ s col4\",
\"& s col5\",
\"# s col6\",
\"^ s col7\",
\"! s col8\",
\"$ s col9\",
\"* s col10\",
\"%.........@\",
\"%.........@\",
\"%.........@\",
\"%.........@\",
\"%.........@\",
\"%..++++++.@\",
\"%.++++.+..@\",
\"%.++++.+..@\",
\"%.++++.+..@\",
\"%..+++.+..@\",
\"%....+.+..@\",
\"%....+.+..@\",
\"%....+.+..@\",
\"%....+.+..@\",
\"%....+.+..@\",
\"%....+.+..@\",
\"%.........@\",
\"%.........@\",
\"%.........@\",
\"%.........@\"};")

I am including links to the uploaded pictures just in case the mail list cannot handle storage of files that exceed a certain limit.

Links to the same images that are attached:

https://lawlist.com/images/xpm_only.png

https://lawlist.com/images/with_buffer_display_table.png


[-- Attachment #2: with_buffer_display_table.png --]
[-- Type: application/png, Size: 176100 bytes --]

[-- Attachment #3: xpm_only.png --]
[-- Type: application/png, Size: 172993 bytes --]