bug#13399: 24.3.50; Word-wrap can't wrap at zero-width space U-200B

[Adam Tack <adam.tack.513@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2760 bytes --]

I have a patch for the original issue of word-wrap not wrapping at a
zero-width space.  The implementation uses a character table, and is
closely based on that written by Martin Rudalics
(https://debbugs.gnu.org/cgi/bugreport.cgi?bug=13399#113), with Eli
Zaretski's suggestions regarding unicode.

The patch applies cleanly to the latest master, compiles on GNU+Linux
(Ubuntu Xenial) and appears to work — both of the following tests
result in the expected wrapping on the zero-width space character (the
first of these is taken verbatim from this bug thread, the second,
adapted from the first, checks that there is no regression of Bug#11341):

(with-current-buffer (get-buffer-create "*foo*")
  (dotimes (i 1000)
    (insert "1234")) ; U-200B
  (setq word-wrap t)
  (display-buffer "*foo*"))

(with-current-buffer (get-buffer-create "*bar*")
  (dotimes (i 1000)
    (insert "1234")) ; U-200B
  (setq word-wrap t)
  (setq whitespace-display-mappings
    '((space-mark 32
              [183]
              [46])
      (space-mark 160
              [164]
              [95])
      (space-mark 8203
              [164]
              [95])
      (newline-mark 10
            [36 10])
      (tab-mark 9
            [187 9]
            [92 9])))
  (whitespace-mode)
  (display-buffer "*bar*"))

Setting other word-wrap characters using set-char-table-range with
lisp also works as expected in the simple situations that I tested.

However, this is my first foray into modifying a serious C codebase,
so I am not sure if I have done the right thing.  In particular, I
have serious doubts about the second and third cases from
IT_DISPLAYING_WHITESPACE, especially since I don't really know when
they would be applicable.

   || ((STRINGP (it->string)                        \
    && !NILP (CHAR_TABLE_REF                    \
          (Vword_wrap_chars, STRING_CHAR            \
           (SDATA (it->string) + IT_STRING_BYTEPOS (*it)))))    \
       || (it->s && !NILP (CHAR_TABLE_REF                \
               (Vword_wrap_chars,                \
                STRING_CHAR(it->s + IT_BYTEPOS (*it)))))    \

Additionally, I'm not certain whether syms_of_character in character.c
is the right location for the definition of the char-table and whether
the range of characters U+2000 to U+200B should be in the chartable,
or if it should just be space and tab, by default.


I am aware that if this were to be accepted, I would also need to make
a change to etc/NEWS, probably the docstring of `word-wrap' and
somewhere in the Texinfo manual.

I have not yet filled out a copyright assignment form, though I will
do so if this patch (modulo changes) is considered acceptable.

Thanks!

[-- Attachment #2: word_wrap_char_table.diff --]
[-- Type: text/plain, Size: 2525 bytes --]

diff --git a/src/character.c b/src/character.c
index c8ffa2b..6e7f55a 100644
--- a/src/character.c
+++ b/src/character.c
@@ -1145,4 +1145,15 @@ All Unicode characters have one of the following values (symbol):
 See The Unicode Standard for the meaning of those values.  */);
   /* The correct char-table is setup in characters.el.  */
   Vunicode_category_table = Qnil;
+
+  DEFVAR_LISP ("word-wrap-chars", Vword_wrap_chars,
+	       doc: /* A char-table for characters at which word-wrap occurs.
+Such characters have value t in this table.
+By default these are the whitespace characters. */);
+  Vword_wrap_chars = Fmake_char_table (Qnil, Qnil);
+  Fset_char_table_range (Vword_wrap_chars, make_number (9), Qt);
+  Fset_char_table_range (Vword_wrap_chars, make_number (32), Qt);
+  Fset_char_table_range (Vword_wrap_chars,
+			 Fcons (make_number (8192),
+				make_number (8203)), Qt);
 }
diff --git a/src/xdisp.c b/src/xdisp.c
index 7e47c06..7152220 100644
--- a/src/xdisp.c
+++ b/src/xdisp.c
@@ -348,20 +348,23 @@ static Lisp_Object list_of_error;
 #endif /* HAVE_WINDOW_SYSTEM */
 
 /* Test if the display element loaded in IT, or the underlying buffer
-   or string character, is a space or a TAB character.  This is used
-   to determine where word wrapping can occur.  */
+   or string character, belongs to the word-wrap-chars char-table.
+   This is used to determine where word wrapping can occur.  */
 
 #define IT_DISPLAYING_WHITESPACE(it)					\
-  ((it->what == IT_CHARACTER && (it->c == ' ' || it->c == '\t'))	\
+  ((it->what == IT_CHARACTER						\
+    && !NILP (CHAR_TABLE_REF (Vword_wrap_chars, it->c)))		\
    || ((STRINGP (it->string)						\
-	&& (SREF (it->string, IT_STRING_BYTEPOS (*it)) == ' '		\
-	    || SREF (it->string, IT_STRING_BYTEPOS (*it)) == '\t'))	\
-       || (it->s							\
-	   && (it->s[IT_BYTEPOS (*it)] == ' '				\
-	       || it->s[IT_BYTEPOS (*it)] == '\t'))			\
+	&& !NILP (CHAR_TABLE_REF					\
+		  (Vword_wrap_chars, STRING_CHAR			\
+		   (SDATA (it->string) + IT_STRING_BYTEPOS (*it)))))	\
+       || (it->s && !NILP (CHAR_TABLE_REF				\
+			   (Vword_wrap_chars,				\
+			    STRING_CHAR(it->s + IT_BYTEPOS (*it)))))	\
        || (IT_BYTEPOS (*it) < ZV_BYTE					\
-	   && (*BYTE_POS_ADDR (IT_BYTEPOS (*it)) == ' '			\
-	       || *BYTE_POS_ADDR (IT_BYTEPOS (*it)) == '\t'))))		\
+	   && !NILP (CHAR_TABLE_REF					\
+		     (Vword_wrap_chars,					\
+		      (FETCH_CHAR(IT_BYTEPOS (*it))))))))		\
 
 /* True means print newline to stdout before next mini-buffer message.  */