bug#13399: 24.3.50; Word-wrap can't wrap at zero-width space U-200B

[Eli Zaretskii <eliz@gnu.org>]

> Date: Fri, 11 Jan 2013 09:16:45 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: 13399@debbugs.gnu.org
> 
>  >> As can be seen in the window showing *foo*,
>  >> lines are not regularly wrapped at that character.
>  >
>  > You mean, not wrapped at all.  Witness the continuation bitmaps in the
>  > fringes, which shouldn't appear when a line is wrapped.
> 
> I thought these bitmaps appear when a line is wrapped.

Not by default.  Not unless you customize visual-line-fringe-indicators.

>  > If anything, this is a missing feature, since word-wrap is explicitly
>  > coded to break lines only on SPC and TAB characters.
> 
> The doc-string of `word-wrap' says
> 
>    When word-wrapping is on, continuation lines are wrapped at the space
>    or tab character nearest to the right window edge
> 
> Since U-200B is a space character the line should wrap at it.

No, it means literally "the space character", U+0020.

>  Also
> 
>    this character is intended for invisible word separation and for line
>    break control; it has no width, but its presence between two
>    characters does not prevent increased letter spacing in justification
> 
> and Emacs apparently does handle it specially since it reserves a few
> pixels when drawing it.

See glyphless-char-display and glyphless-char-display-control for why.

> But documentation on `word-wrap' is scarce ...

Actually, it doesn't exist, apart of the doc string.

>  > See the
>  > IT_DISPLAYING_WHITESPACE macro in xdisp.c.
> 
> I tried to understand the code but failed.

  #define IT_DISPLAYING_WHITESPACE(it)					\
    /* If the character to be displayed is SPC or TAB */
    ((it->what == IT_CHARACTER && (it->c == ' ' || it->c == '\t'))      \
    /* Or we are iterating over a display or overlay string, ... */
     || ((STRINGP (it->string)						\
    /* ... and the character at current string position is SPC or TAB */
	  && (SREF (it->string, IT_STRING_BYTEPOS (*it)) == ' '		\
	      || SREF (it->string, IT_STRING_BYTEPOS (*it)) == '\t'))	\
    /* Or we are iterating over a C string, ... */
	 || (it->s							\
    /* ... and the character at current string position is SPC or TAB */
	     && (it->s[IT_BYTEPOS (*it)] == ' '				\
		 || it->s[IT_BYTEPOS (*it)] == '\t'))			\
    /* Or the iterator is before end of buffer's reachable portion, ... */
	 || (IT_BYTEPOS (*it) < ZV_BYTE					\
    /* ... and the character at current buffer position is SPC or TAB */
	     && (*BYTE_POS_ADDR (IT_BYTEPOS (*it)) == ' '			\
		 || *BYTE_POS_ADDR (IT_BYTEPOS (*it)) == '\t'))))		\

In any case, you can clearly see that it only tests for literal SPC
and TAB characters.

>  > If we want to add more characters to the set, we should probably
>  > arrange a special char-table for this, and have it exposed to Lisp, so
>  > it could be customized.  Patches are welcome.
> 
> IIUC all breakable spaces are between U-2000 and U-200B so maybe a
> character table is not needed.

Who said we want only break at breakable space characters?  Who said
Unicode will never add more such characters in another block?  And
what about low-ASCII characters, which are already in a different
block?

In any case, even if you are right, a char-table is a way to store
character properties efficiently.  In particular, it will waste very
little storage to mark a contiguous range of characters with the same
property.  The advantage of using a char-table is that it will
dynamically expand as needed if more characters are added to the set.

> Anway, exposing displayed text to Lisp would be great.  We'd just need
> two functions - one that gets the pixel width of an arbitrary buffer
> string wrt a specific window, and one that gets the pixel height of an
> arbitrary buffer string (newlines ignored) wrt a specific window.  This
> way we could get rid of lots of problems currently hidden in the display
> engine ...

You lost me here.  By "exposing to Lisp" I meant expose the char-table
of word-wrap characters to Lisp.  What did _you_ want exposed to Lisp?