bug#74178: Handle tab stops on display so they work in variable-pitch-mode

bug#74178: Handle tab stops on display so they work in variable-pitch-mode

[dannym]

[-- Attachment #1: Type: text/plain, Size: 1967 bytes --]

Hello,

Right now, when using variable-pitch-mode, there's no way for the user
to make tables.  This patch adds a feature to do that.

This patch handles tab-stop-list in the display part of emacs so that
they work as expected when using variable-pitch-mode.

The unit of the values in tab-stop-list is space characters.

The reasons why those values are in space characters are:

- Works with HiDPI and LoDPI screens the same way with the same values
inside tab-stop-list.
- Works in console emacs.
- Works in graphical emacs.
- Is backward compatible.

My assumption is that existing user files either don't have any '\t'
in their buffers or they are using the (previous) high-level version
of emacs tab-stop-list (which will replace all '\t' by some spaces),
not both.  This way, emacs without the patch is forward-compatible
to emacs with the patch.

This is a version that will use tabs in the order they appear in the
tab-stop-list, (on purpose) regardless of whether the text on the
line already exceeded the respective tab stop position (because it's
still better not to use the wrong column in the table).

Each respective tab stop, in order, will be used for a respective
'\t', per line. If these fixed tab stops are used up, it will fall
back to the automatic tab stops every tab_width that emacs also
already had done before.

The list of tab stops is buffer-local--as before.

I successfully tested both console and graphical emacs.

In GNU Emacs 29.4 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.41,
cairo version 1.18.0)
System Description: Guix System

Configured using:
  'configure
  
CONFIG_SHELL=/gnu/store/3jhfhxdf6v5ms10x5zmnl166dh3yhbr1-bash-minimal-5.1.16/bin/bash
  
SHELL=/gnu/store/3jhfhxdf6v5ms10x5zmnl166dh3yhbr1-bash-minimal-5.1.16/bin/bash
  --prefix=/gnu/store/9q1cfyj0bk0lqvx75pg9gn4isnlz2llv-emacs-pgtk-29.4
  --enable-fast-install --with-pgtk --with-cairo --with-modules
  --with-native-compilation=aot --disable-build-details'

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: emacs-tab-stops-v2.patch --]
[-- Type: text/x-diff; name=emacs-tab-stops-v2.patch, Size: 8703 bytes --]

diff -ru '-F^[_a-zA-Z0-9$]\+ *(' orig/emacs-29.4/ChangeLog emacs-29.4/ChangeLog
--- orig/emacs-29.4/ChangeLog	1970-01-01 01:00:01.000000000 +0100
+++ emacs-29.4/ChangeLog	2024-11-02 17:54:05.557873763 +0100
@@ -1,3 +1,19 @@
+2024-11-02  Danny Milosavljevic  <dannym@friendly-machines.com>
+
+	Handle tab stops in display (instead of faking it).
+
+	This makes it possible for the user to make tables in
+	variable-pitch-mode as well.
+
+	* src/buffer.h (tab_stop_list): New public variable.
+	* src/buffer.c (tab_stop_list): Initialize it.
+	* src/dispextern.h (tab_stop_list): New variable.
+	* src/term.c (find_next_tab_stop): New procedure.
+	(produce_glyphs): Use it.
+	* src/xdisp.c (init_iterator): Initialize tab_stop_list.
+	(take_next_tab_stop): New procedure.
+	(gui_produce_glyphs): Use it.
+
 2024-06-22  Stefan Kangas  <stefankangas@gmail.com>
 
 	* Version 29.4 released.
diff -ru '-F^[_a-zA-Z0-9$]\+ *(' orig/emacs-29.4/src/buffer.c emacs-29.4/src/buffer.c
--- orig/emacs-29.4/src/buffer.c	1970-01-01 01:00:01.000000000 +0100
+++ emacs-29.4/src/buffer.c	2024-11-02 12:56:16.890331961 +0100
@@ -4676,6 +4676,7 @@ init_buffer_once (void)
   XSETFASTINT (BVAR (&buffer_local_flags, selective_display), idx); ++idx;
   XSETFASTINT (BVAR (&buffer_local_flags, selective_display_ellipses), idx); ++idx;
   XSETFASTINT (BVAR (&buffer_local_flags, tab_width), idx); ++idx;
+  XSETFASTINT (BVAR (&buffer_local_flags, tab_stop_list), idx); ++idx;
   XSETFASTINT (BVAR (&buffer_local_flags, truncate_lines), idx);
   /* Make this one a permanent local.  */
   buffer_permanent_local_flags[idx++] = 1;
@@ -4776,6 +4777,7 @@ init_buffer_once (void)
   buffer_defaults.overlays = NULL;
 
   XSETFASTINT (BVAR (&buffer_defaults, tab_width), 8);
+  bset_tab_stop_list (&buffer_defaults, Qnil);
   bset_truncate_lines (&buffer_defaults, Qnil);
   bset_word_wrap (&buffer_defaults, Qnil);
   bset_ctl_arrow (&buffer_defaults, Qt);
@@ -5216,6 +5218,12 @@ syms_of_buffer (void)
 inserts one or more TAB characters, this variable will affect the
 indentation step as well, even if `indent-tabs-mode' is non-nil.  */);
 
+  DEFVAR_PER_BUFFER ("tab-stop-list", &BVAR (current_buffer, tab_stop_list),
+		     Qnil,
+		     doc: /* Where fixed tab stops are (for display of tab characters), in columns.
+This controls the positions of fixed tab stops on display.
+The value should be a list of positive integers (each from the beginning of the line).  */);
+
   DEFVAR_PER_BUFFER ("ctl-arrow", &BVAR (current_buffer, ctl_arrow), Qnil,
 		     doc: /* Non-nil means display control chars with uparrow `^'.
 A value of nil means use backslash `\\' and octal digits.
diff -ru '-F^[_a-zA-Z0-9$]\+ *(' orig/emacs-29.4/src/buffer.h emacs-29.4/src/buffer.h
--- orig/emacs-29.4/src/buffer.h	1970-01-01 01:00:01.000000000 +0100
+++ emacs-29.4/src/buffer.h	2024-11-02 12:22:03.458384616 +0100
@@ -381,6 +381,7 @@ BUF_TEMP_SET_PT (struct buffer *buffer,
      in buffers that are not current.  */
   Lisp_Object case_fold_search_;
   Lisp_Object tab_width_;
+  Lisp_Object tab_stop_list_;
   Lisp_Object fill_column_;
   Lisp_Object left_margin_;
 
@@ -832,6 +833,11 @@ bset_undo_list (struct buffer *b, Lisp_O
   b->undo_list_ = val;
 }
 INLINE void
+bset_tab_stop_list (struct buffer *b, Lisp_Object val)
+{
+  b->tab_stop_list_ = val;
+}
+INLINE void
 bset_upcase_table (struct buffer *b, Lisp_Object val)
 {
   b->upcase_table_ = val;
diff -ru '-F^[_a-zA-Z0-9$]\+ *(' orig/emacs-29.4/src/dispextern.h emacs-29.4/src/dispextern.h
--- orig/emacs-29.4/src/dispextern.h	1970-01-01 01:00:01.000000000 +0100
+++ emacs-29.4/src/dispextern.h	2024-11-02 12:51:48.394338846 +0100
@@ -2636,6 +2636,9 @@ GLYPH_CODE_P (Lisp_Object gc)
   /* Number of columns per \t.  */
   short tab_width;
 
+  /* Position of the tab stops (in columns) */
+  Lisp_Object tab_stop_list;
+
   /* Value of the `height' property, if any; nil if none.  */
   Lisp_Object font_height;
 
diff -ru '-F^[_a-zA-Z0-9$]\+ *(' orig/emacs-29.4/src/term.c emacs-29.4/src/term.c
--- orig/emacs-29.4/src/term.c	1970-01-01 01:00:01.000000000 +0100
+++ emacs-29.4/src/term.c	2024-11-02 18:01:32.877862293 +0100
@@ -1527,6 +1527,25 @@ tty_append_glyph (struct it *it)
 }
 
 
+static inline int
+take_next_tab_stop (struct it *it, int x)
+{
+  Lisp_Object tail = it->tab_stop_list;
+  if (CONSP (tail)) {
+      if (FIXNUMP (XCAR (tail))) {
+            int result = XFIXNUM (XCAR (tail));
+            it->tab_stop_list = XCDR (tail);
+            return result;
+      } else
+            return x;
+  }
+
+  /* Fall back to fixed tab stops */
+  int next_tab_x = (((1 + x + it->tab_width - 1) / it->tab_width) * it->tab_width);
+  return next_tab_x;
+}
+
+
 /* Produce glyphs for the display element described by IT.  *IT
    specifies what we want to produce a glyph for (character, image, ...),
    and where in the glyph matrix we currently are (glyph row and hpos).
@@ -1582,7 +1601,11 @@ produce_glyphs (struct it *it)
 	append_glyph (it);
     }
   else if (it->char_to_display == '\n')
-    it->pixel_width = it->nglyphs = 0;
+    {
+      it->pixel_width = it->nglyphs = 0;
+      /* Reset tab stops to the user value after each paragraph */
+      it->tab_stop_list = BVAR (current_buffer, tab_stop_list);
+    }
   else if (it->char_to_display == '\t')
     {
       int absolute_x = (it->current_x
@@ -1591,10 +1614,7 @@ produce_glyphs (struct it *it)
       /* Adjust for line numbers.  */
       if (!NILP (Vdisplay_line_numbers) && it->line_number_produced_p)
 	absolute_x -= it->lnum_pixel_width;
-      int next_tab_x
-	= (((1 + absolute_x + it->tab_width - 1)
-	    / it->tab_width)
-	   * it->tab_width);
+      int next_tab_x = take_next_tab_stop (it, absolute_x);
       if (!NILP (Vdisplay_line_numbers) && it->line_number_produced_p)
 	next_tab_x += it->lnum_pixel_width;
       int nspaces;
diff -ru '-F^[_a-zA-Z0-9$]\+ *(' orig/emacs-29.4/src/xdisp.c emacs-29.4/src/xdisp.c
--- orig/emacs-29.4/src/xdisp.c	1970-01-01 01:00:01.000000000 +0100
+++ emacs-29.4/src/xdisp.c	2024-11-02 18:01:45.273861975 +0100
@@ -3316,6 +3316,7 @@ init_iterator (struct it *it, struct win
   it->multibyte_p = !NILP (BVAR (current_buffer, enable_multibyte_characters));
 
   it->tab_width = SANE_TAB_WIDTH (current_buffer);
+  it->tab_stop_list = BVAR (current_buffer, tab_stop_list);
 
   /* Are lines in the display truncated?  */
   if (TRUNCATE != 0)
@@ -31691,6 +31692,31 @@ calc_line_height_property (struct it *it
 }
 
 
+static inline int
+take_next_tab_stop (struct it *it, int font_space_width, int x)
+{
+  Lisp_Object tail = it->tab_stop_list;
+  if (CONSP (tail)) {
+      if (FIXNUMP (XCAR (tail))) {
+            int result = XFIXNUM (XCAR (tail)) * font_space_width;
+      		it->tab_stop_list = XCDR (tail);
+      		return result;
+      } else
+            return x;
+  }
+
+  int tab_width = it->tab_width * font_space_width;
+  /* Fall back to fixed tab stops */
+  int next_tab_x = ((1 + x + tab_width - 1) / tab_width) * tab_width;
+  /* If the distance from the current position to the next tab
+	 stop is less than a space character width, use the
+	 tab stop after that.  */
+  if (next_tab_x - x < font_space_width)
+		next_tab_x += tab_width;
+  return next_tab_x;
+}
+
+
 /* Append a glyph for a glyphless character to IT->glyph_row.  FACE_ID
    is a face ID to be used for the glyph.  FOR_NO_FONT is true if
    and only if this is for a character for which no font was found.
@@ -32112,6 +32138,8 @@ gui_produce_glyphs (struct it *it)
 	  it->override_ascent = -1;
 	  it->pixel_width = 0;
 	  it->nglyphs = 0;
+	  /* Reset tab stops to the user value after each paragraph */
+	  it->tab_stop_list = BVAR (current_buffer, tab_stop_list);
 
 	  height = get_it_property (it, Qline_height);
 	  /* Split (line-height total-height) list.  */
@@ -32202,7 +32230,6 @@ gui_produce_glyphs (struct it *it)
 	{
 	  if (font->space_width > 0)
 	    {
-	      int tab_width = it->tab_width * font->space_width;
 	      int x = it->current_x + it->continuation_lines_width;
 	      int x0 = x;
 	      /* Adjust for line numbers, if needed.   */
@@ -32214,13 +32241,8 @@ gui_produce_glyphs (struct it *it)
 		    x += it->stretch_adjust;
 		}
 
-	      int next_tab_x = ((1 + x + tab_width - 1) / tab_width) * tab_width;
+	      int next_tab_x = take_next_tab_stop (it, font->space_width, x);
 
-	      /* If the distance from the current position to the next tab
-		 stop is less than a space character width, use the
-		 tab stop after that.  */
-	      if (next_tab_x - x < font->space_width)
-		next_tab_x += tab_width;
 	      if (!NILP (Vdisplay_line_numbers) && it->line_number_produced_p)
 		{
 		  next_tab_x += it->lnum_pixel_width;

bug#74178: Handle tab stops on display so they work in variable-pitch-mode

[Eli Zaretskii]

> Date: Sat, 02 Nov 2024 19:06:54 +0100
> From: dannym@friendly-machines.com
> 
> Right now, when using variable-pitch-mode, there's no way for the user
> to make tables.  This patch adds a feature to do that.
> 
> This patch handles tab-stop-list in the display part of emacs so that
> they work as expected when using variable-pitch-mode.
> 
> The unit of the values in tab-stop-list is space characters.
> 
> The reasons why those values are in space characters are:
> 
> - Works with HiDPI and LoDPI screens the same way with the same values
> inside tab-stop-list.
> - Works in console emacs.
> - Works in graphical emacs.
> - Is backward compatible.
> 
> My assumption is that existing user files either don't have any '\t'
> in their buffers or they are using the (previous) high-level version
> of emacs tab-stop-list (which will replace all '\t' by some spaces),
> not both.  This way, emacs without the patch is forward-compatible
> to emacs with the patch.
> 
> This is a version that will use tabs in the order they appear in the
> tab-stop-list, (on purpose) regardless of whether the text on the
> line already exceeded the respective tab stop position (because it's
> still better not to use the wrong column in the table).
> 
> Each respective tab stop, in order, will be used for a respective
> '\t', per line. If these fixed tab stops are used up, it will fall
> back to the automatic tab stops every tab_width that emacs also
> already had done before.
> 
> The list of tab stops is buffer-local--as before.

Thanks, but please tell how this feature is intended to be used.  In
particular, what would be the value of the new buffer-local variable,
and how will the tab stops of this value be determined by users or
Lisp programs who want to take advantage of this feature?

There are also some technical problems with the idea of your
implementation: for example, it sounds like the display iterator will
need to always start from the beginning of a physical line, otherwise
it will be unable to correctly account for the tab stops before the
position where it is initialized.  But let's ignore these aspects for
now and talk about the general idea, because it's possible that the
feature could be implemented in some other way that is free of these
problems.

P.S. There's also the issue of assigning to the FSF the copyright for
your work, without which we will be unable to accept a contribution of
this size.  Should I send you the form to fill and the instructions,
to start the legal paperwork rolling?