Implementing Vertical Text support in Emacs

Implementing Vertical Text support in Emacs

[समीर सिंह Sameer Singh]

[-- Attachment #1: Type: text/plain, Size: 1613 bytes --]

Recently while implementing the Mongolian script, I thought, maybe vertical
text support should be added to Emacs too.

Vertical text is used in scripts like the CJK characters, Mongolian, Phags
Pa, Ogham etc.
Scripts like the CJK characters do not need to be rotated when written
vertically and can be written either from ltr or rtl,
while scripts like Mongolian are rotated and written from left to right.

So my questions are: Should this be done? If so, where should I start.
How to correctly mix non vertical and vertical scripts in the same buffer,
if it is even possible.
Should this be a minor mode only?

Here are some reference links:
This UAX#50 defines default orientation for various unicode characters:
https://www.unicode.org/reports/tr50/tr50-26.html
This website defines orientation of all unicode characters:
https://www.unicode.org/Public/vertical/revision-17/VerticalOrientation-17.html

Robust Vertical Text Layout: Using the Unicode BIDI Algorithm to Handle
Complexities in Typesetting Multi-Script Vertical Text:
https://www.unicode.org/notes/tn22/RobustVerticalLayout.pdf

css-writing-modes-4: https://www.w3.org/TR/css-writing-modes-4/

OTF Tags for vertical text:
https://docs.microsoft.com/en-us/typography/opentype/spec/features_uz#tag-vert
https://docs.microsoft.com/en-us/typography/opentype/spec/features_uz#tag-vrtr

This is what the UAX#9 says about bidi text in vertical orientation:
https://unicode.org/reports/tr9/#Vertical_Text

A website that has vertical text orientation: https://wikibilig.mn/

I do not promise that this would be completed, but atleast I can try.
Thanks.

[-- Attachment #2: Type: text/html, Size: 2288 bytes --]

RE: [External] : Implementing Vertical Text support in Emacs

[Drew Adams]

> Recently while implementing the Mongolian script,
> I thought, maybe vertical text support should be
> added to Emacs too.
>
> Vertical text is used in scripts like the CJK
> characters, Mongolian, Phags Pa, Ogham etc.
> Scripts like the CJK characters do not need to be
> rotated when written vertically and can be written
> either from ltr or rtl, while scripts like
> Mongolian are rotated and written from left to right.
>
> So my questions are: Should this be done? If so,
> where should I start[?].

Cool.

I have no idea whether this will help or is in any
way really related.  Maybe not.  But I'll mention
it in case it might be.

Library `find-where.el' defines commands to move
over words that are written vertically:
`fw-downward-word' and `fw-upward-word'.  (They're
written mainly as a way of illustrating some of the
library's features.)

They're like `forward-word' and `backward-word',
but they move downward and upward, not across.
___

More general are commands `fw-to-next-where-vertical'
and `fw-to-previous-where-vertical'.  They move to
the first buffer position directly below/above point
where a given predicate is true.  They prompt you for
the predicate.
___

Doc:
https://www.emacswiki.org/emacs/FindWhere

Code:
https://www.emacswiki.org/emacs/download/find-where.el

Re: Implementing Vertical Text support in Emacs

[Eli Zaretskii]

> From: समीर सिंह Sameer Singh
>  <lumarzeli30@gmail.com>
> Date: Mon, 13 Jun 2022 20:59:45 +0530
> 
> Recently while implementing the Mongolian script, I thought, maybe vertical text support should be added to
> Emacs too.
> 
> Vertical text is used in scripts like the CJK characters, Mongolian, Phags Pa, Ogham etc.
> Scripts like the CJK characters do not need to be rotated when written vertically and can be written either
> from ltr or rtl,
> while scripts like Mongolian are rotated and written from left to right.
> 
> So my questions are: Should this be done?

No one but yourself can answer that question.  Obviously, scripts that
are written vertically do exist, so adding such capabilities to Emacs
will make Emacs more powerful, and is therefore a Good Thing, at least
in principle.  Whether that advantage is worth the effort and the
additional complexities in the Emacs display code and the related
parts (see below), is not something that can be easily answered, at
least not until we understand the implications better.  In general,
I'd say YES, assuming there's motivation and will to see this through.

> If so, where should I start.

Start by reading the large commentary at the beginning of xdisp.c,
which providers a general overview of the display engine and its basic
structure and algorithms.  After you digest that, we'd need to talk
about the details, because they are non-trivial.  Let me show you the
tip of the iceberg (and it could be that what I say below won't make
sense until you read that commentary).

Emacs display is basically built around a 2D structure called "glyph
matrix".  The rows of each matrix correspond to screen lines of the
window to which the matrix belongs.  The display engine works by
updating the matrix given the up-to-date buffer text, then reflecting
its rows on the glass.

For vertical layout, we'd need to consider each row to correspond to a
screen _column_.  This will require to write a vertical version of the
display_line function, whose job is to produce a single row of the
glyph matrix, because the layout and dimensions of the vertical text
are different, and so are the various layout decisions.  There will be
secondary, but nevertheless important, issues, like how to wrap
columns, how to indicate the wrap-around (what we currently do with
fringe bitmaps for continued lines), etc.  The code which converts
screen coordinates to the cell in the glyph matrix and to buffer
positions (it is used to process mouse clicks, for example) will also
need to have its own vertical version.

AFAIU, some vertical scripts are written in rows left-to-right and
some right-to-left, so you actually have 2 different layouts on your
hands.

Some redisplay optimizations will need to be rewritten for vertical
text, since at least some of them are based on window-system features
like moving blocks of pixels.  Or maybe we will decide that some
optimizations will not be used for vertical text.

Then there will be issues with scrolling: do we swap horizontal and
vertical scrolling for vertical-layout text, or do we keep the scroll
direction (in which case we need to decide what it means to scroll up
or down)?

There will be more -- there's always more when Emacs display is
concerned.  The iceberg is very large, not unlike the one we faced
when support for bidirectional display was designed and implemented.

> How to correctly mix non vertical and vertical scripts in the same buffer, if it is even possible.

This is an even tougher issue.  Is support for such mixed scripts a
necessity without which the entire feature won't make sense?  If not,
I'd suggest to limit yourself to either vertical or horizontal layout,
leaving the mix of the two unsupported for starters.  If mixing them
is a must, everything I said above needs to be rethought, and it could
even be that the current design of the display engine simply cannot be
extended to support that.

> Should this be a minor mode only?

Why does it matter how we turn on or off this feature?  This is the
easiest decision to make, negligible when compared to the effort
required to implement the stuff in the first place.

> OTF Tags for vertical text:
> https://docs.microsoft.com/en-us/typography/opentype/spec/features_uz#tag-vert
> https://docs.microsoft.com/en-us/typography/opentype/spec/features_uz#tag-vrtr

Do vertical-layout scripts require complex text shaping
(a.k.a. "character composition")?  If so, does HarfBuzz support such
scripts and their corresponding fonts?  I don't know.  We may need a
different shaping engine (if one exists).

> This is what the UAX#9 says about bidi text in vertical orientation:
> https://unicode.org/reports/tr9/#Vertical_Text

Bidirectional reordering of vertical-layout text is the least of your
concerns.  Either it is reordered exactly as our implementation
already does, and just drawn vertiually, in which case the problem is
already solved; or it isn't reordered at all, in which case the
problem doesn't exist in the first place.

Thanks.

Re: Implementing Vertical Text support in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Are there any scripts that are always and only written vertically?
Are there any scripts that people actually write in
that people do not nowadays write horizontally in word processors?

If people have learned to accept writing those scripts horizontally, I
think that way is good enough for Emacs too.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Implementing Vertical Text support in Emacs

[Po Lu]

Richard Stallman <rms@gnu.org> writes:

> Are there any scripts that are always and only written vertically?

I think Mongolian script is always written that way.

Re: Implementing Vertical Text support in Emacs

[समीर सिंह Sameer Singh]

[-- Attachment #1: Type: text/plain, Size: 1137 bytes --]

>
> > Are there any scripts that are always and only written vertically?
>
> I think Mongolian script is always written that way.


Yes I think scripts like Mongolian and Phags Pa are always written
vertically.

Are there any scripts that people actually write in
> that people do not nowadays write horizontally in word processors?
>

I am sorry I do not understand the question, did you meant to say
"vertically"?
btw word processors like libreoffice support vertical text.

If people have learned to accept writing those scripts horizontally, I
> think that way is good enough for Emacs too.
>

Wouldn't it be better to support vertical text in Emacs, the only reason
people would have "learned to accept" to write these scripts horizontally
is because their preferred way of writing is not supported in many
softwares/websites.
I know that this is a complex process, but at least we can try.

On Wed, Jun 15, 2022 at 3:52 PM Po Lu <luangruo@yahoo.com> wrote:

> Richard Stallman <rms@gnu.org> writes:
>
> > Are there any scripts that are always and only written vertically?
>
> I think Mongolian script is always written that way.
>

[-- Attachment #2: Type: text/html, Size: 2240 bytes --]

Re: Implementing Vertical Text support in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Wouldn't it be better to support vertical text in Emacs,

In a world in which we had an infinite supply of very good programmers
working on Emacs, it would be better to support vertical text.

But in the actual world, we need to ask whether it is worth the cost,
and whether some other feature would be more worth while.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Implementing Vertical Text support in Emacs

[समीर सिंह Sameer Singh]

[-- Attachment #1: Type: text/plain, Size: 3107 bytes --]

Thanks for such a detailed response.

Start by reading the large commentary at the beginning of xdisp.c,
> which providers a general overview of the display engine and its basic
> structure and algorithms.  After you digest that, we'd need to talk
> about the details, because they are non-trivial.  Let me show you the
> tip of the iceberg (and it could be that what I say below won't make
> sense until you read that commentary).
>

I have read the full commentary and have understood it somewhat, but at
least I can understand what you have written later.
Maybe while writing the implementation it will make more sense.

Then there will be issues with scrolling: do we swap horizontal and
> vertical scrolling for vertical-layout text, or do we keep the scroll
> direction (in which case we need to decide what it means to scroll up
> or down)?
>

I think we should swap horizontal and vertical scrolling.
For a reference see this website: wikibilig.mn/

AFAIU, some vertical scripts are written in rows left-to-right and
> some right-to-left, so you actually have 2 different layouts on your
> hands.
>

Scripts like Mongolian are written only in one direction: ltr, but CJK
characters are bidirectional i.e.
they can either be written from ltr or from rtl .How should we support that?

This is an even tougher issue.  Is support for such mixed scripts a
> necessity without which the entire feature won't make sense?  If not,
> I'd suggest to limit yourself to either vertical or horizontal layout,
> leaving the mix of the two unsupported for starters.  If mixing them
> is a must, everything I said above needs to be rethought, and it could
> even be that the current design of the display engine simply cannot be
> extended to support that.
>

Yeah, I should not mix them.

Do vertical-layout scripts require complex text shaping
> (a.k.a. "character composition")?  If so, does HarfBuzz support such
> scripts and their corresponding fonts?  I don't know.  We may need a
> different shaping engine (if one exists).
>

Yes they require complex text shaping and Harfbuzz supports it.
I even had a Mongolian patch ready but accidently deleted my emacs source
directory. :(

Also how would I test the new changes I would make in xdisp.c?

Thanks.

On Fri, Jun 17, 2022 at 4:18 AM Richard Stallman <rms@gnu.org> wrote:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > Wouldn't it be better to support vertical text in Emacs,
>
> In a world in which we had an infinite supply of very good programmers
> working on Emacs, it would be better to support vertical text.
>
> But in the actual world, we need to ask whether it is worth the cost,
> and whether some other feature would be more worth while.
>
> --
> Dr Richard Stallman (https://stallman.org)
> Chief GNUisance of the GNU Project (https://gnu.org)
> Founder, Free Software Foundation (https://fsf.org)
> Internet Hall-of-Famer (https://internethalloffame.org)
>
>
>

[-- Attachment #2: Type: text/html, Size: 4939 bytes --]

Re: Implementing Vertical Text support in Emacs

[Eli Zaretskii]

> From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> Date: Fri, 17 Jun 2022 07:13:33 +0530
> Cc: Po Lu <luangruo@yahoo.com>, emacs-devel@gnu.org, Richard Stallman <rms@gnu.org>
> 
> > AFAIU, some vertical scripts are written in rows left-to-right and
> > some right-to-left, so you actually have 2 different layouts on your
> > hands.
> 
> Scripts like Mongolian are written only in one direction: ltr, but CJK
> characters are bidirectional i.e.
> they can either be written from ltr or from rtl .How should we support that?

The low-level terminal-specific code which outputs the series of
glyphs (a.k.a. "glyph strings") to the glass will have to figure it
out, as part of the general support for vertical layout.  This is the
code you will find in the various *term.[cm] files.

The current terminal-specific code which writes to the glass always
outputs stuff left-to-right, one screen line after another and in
generally top-to-bottom order.  (The bidi display doesn't change it,
it just arranges for the glyph rows to have a stretch glyph (which
displays as a stretch of background-color space) of a suitably
calculated width as the first glyph of the glyph row, and the rest of
the glyphs are put into glyph rows in their visual order.)

The code which will support vertical layout will probably need to
change that, and draw "glyph columns" top to bottom.  Then it could
decide whether to start at the left or the right side of the window.

Alternatively, there could be an intermediate step that converts
column-wise series of glyphs into glyph rows (or maybe fills the glyph
rows directly in the first place); then the level that writes to the
glass could remain largely unchanged.  This is actually preferable if
we want to support vertical layout on TTY frames, because any method
other than writing characters left-to-right will be much slower on
text-mode terminals, due to the need of sending commands to move the
output cursor.

> > Do vertical-layout scripts require complex text shaping
> > (a.k.a. "character composition")?  If so, does HarfBuzz support such
> > scripts and their corresponding fonts?  I don't know.  We may need a
> > different shaping engine (if one exists).
> 
> Yes they require complex text shaping and Harfbuzz supports it.

Then you will need to make sure that HarfBuzz returns in that case the
same information as in the horizontal case, regarding the pixel
offsets of the individual parts of a grapheme cluster.  For example,
is the X offset still measured horizontally in that case and the Y
offset still measured vertically, or did they swap the directions?

It's also possible that composite.el will have to be modified, if we
want the likes of compose-gstring-for-graphic to work with the
vertical layout.

> Also how would I test the new changes I would make in xdisp.c?

There's a small test suite in test/manual/redisplay-testsuite.el, and
another one in test/src/xdisp-tests.el, but by and large the only way
is to just use Emacs and fix any problems that pop up.  We don't
currently have a good way of testing the display engine, because that
requires some infrastructure to detect display effects automatically.

Btw, in addition to display_line, the other places that will probably
need changes are the move_it_* family of functions, starting from
move_it_in_display_line_to -- those are used in code that needs to
make layout decisions without displaying anything, for example when
Emacs decides where to set the window-start point after scrolling the
window.

Re: Implementing Vertical Text support in Emacs

[समीर सिंह Sameer Singh]

[-- Attachment #1: Type: text/plain, Size: 4475 bytes --]

Well after getting some free time I started to work on this for a week or
so and I have to say I am quite a bit lost :(

Right now I don't care about harfbuzz or correct character composition, I
only want to display rows vertically, but
I don't know where to start, does modifying only display_line and
move_it_in_display_line_to work?
Even if this is the case, I don't know how to parse the huge display_line
function and find the area of interest
among the lines of wrap and hscroll code? where is the code where the
matrix is filled?

Regrettably my inexperience  is causing hurdles therefore any guidance
would be a huge help.

Thank you.

On Fri, Jun 17, 2022 at 11:42 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> > Date: Fri, 17 Jun 2022 07:13:33 +0530
> > Cc: Po Lu <luangruo@yahoo.com>, emacs-devel@gnu.org, Richard Stallman <
> rms@gnu.org>
> >
> > > AFAIU, some vertical scripts are written in rows left-to-right and
> > > some right-to-left, so you actually have 2 different layouts on your
> > > hands.
> >
> > Scripts like Mongolian are written only in one direction: ltr, but CJK
> > characters are bidirectional i.e.
> > they can either be written from ltr or from rtl .How should we support
> that?
>
> The low-level terminal-specific code which outputs the series of
> glyphs (a.k.a. "glyph strings") to the glass will have to figure it
> out, as part of the general support for vertical layout.  This is the
> code you will find in the various *term.[cm] files.
>
> The current terminal-specific code which writes to the glass always
> outputs stuff left-to-right, one screen line after another and in
> generally top-to-bottom order.  (The bidi display doesn't change it,
> it just arranges for the glyph rows to have a stretch glyph (which
> displays as a stretch of background-color space) of a suitably
> calculated width as the first glyph of the glyph row, and the rest of
> the glyphs are put into glyph rows in their visual order.)
>
> The code which will support vertical layout will probably need to
> change that, and draw "glyph columns" top to bottom.  Then it could
> decide whether to start at the left or the right side of the window.
>
> Alternatively, there could be an intermediate step that converts
> column-wise series of glyphs into glyph rows (or maybe fills the glyph
> rows directly in the first place); then the level that writes to the
> glass could remain largely unchanged.  This is actually preferable if
> we want to support vertical layout on TTY frames, because any method
> other than writing characters left-to-right will be much slower on
> text-mode terminals, due to the need of sending commands to move the
> output cursor.
>
> > > Do vertical-layout scripts require complex text shaping
> > > (a.k.a. "character composition")?  If so, does HarfBuzz support such
> > > scripts and their corresponding fonts?  I don't know.  We may need a
> > > different shaping engine (if one exists).
> >
> > Yes they require complex text shaping and Harfbuzz supports it.
>
> Then you will need to make sure that HarfBuzz returns in that case the
> same information as in the horizontal case, regarding the pixel
> offsets of the individual parts of a grapheme cluster.  For example,
> is the X offset still measured horizontally in that case and the Y
> offset still measured vertically, or did they swap the directions?
>
> It's also possible that composite.el will have to be modified, if we
> want the likes of compose-gstring-for-graphic to work with the
> vertical layout.
>
> > Also how would I test the new changes I would make in xdisp.c?
>
> There's a small test suite in test/manual/redisplay-testsuite.el, and
> another one in test/src/xdisp-tests.el, but by and large the only way
> is to just use Emacs and fix any problems that pop up.  We don't
> currently have a good way of testing the display engine, because that
> requires some infrastructure to detect display effects automatically.
>
> Btw, in addition to display_line, the other places that will probably
> need changes are the move_it_* family of functions, starting from
> move_it_in_display_line_to -- those are used in code that needs to
> make layout decisions without displaying anything, for example when
> Emacs decides where to set the window-start point after scrolling the
> window.
>

[-- Attachment #2: Type: text/html, Size: 5377 bytes --]

Re: Implementing Vertical Text support in Emacs

[Eli Zaretskii]

> From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> Date: Fri, 9 Sep 2022 00:14:45 +0530
> Cc: emacs-devel@gnu.org
> 
> Well after getting some free time I started to work on this for a week or so and I have to say I am quite a bit
> lost :( 
> 
> Right now I don't care about harfbuzz or correct character composition, I only want to display rows vertically,
> but
> I don't know where to start, does modifying only display_line and move_it_in_display_line_to work?
> Even if this is the case, I don't know how to parse the huge display_line function and find the area of interest
> among the lines of wrap and hscroll code? where is the code where the matrix is filled?

display_line is the relatively low level which produces a single
screen line of a window, i.e. a single row of characters, images, and
other "display elements" that occupy a single visual line.

Before you modify display_line, you need to decide how will the
vertical-layout display produce glyphs.  The easiest would be to
produce them in column-wise order, because that basically processes
the buffer in the order of character positions, like Emacs does now.
This will allow you to leave the lower levels, which walk the buffer
and deliver characters, mostly intact.

So perhaps you should have a display_column function, which will look
like display_line, but instead of advancing the X coordinate, it will
advance the Y coordinate, and produce a single column, top to bottom.
Then the basic calculations should be the same as in display_line,
except the column ends when it reaches last_visible_y instead of
last_visible_x.  IOW, the glyph matrices will be made of glyph columns
instead of glyph rows.  Basically, the roles of the X and the Y
coordinates will be swapped.

I suggest to invest some time in reading the code of display_line and
studying its main ideas and tasks.  Because the hypothetical
display_column will have to do something very similar, just vertically
instead of horizontally.  Ask questions where you cannot figure stuff
out.  Then design your display_column in a similar way.

> Regrettably my inexperience  is causing hurdles therefore any guidance would be a huge help.

Don't give up.  And thanks for working on this.

Re: Implementing Vertical Text support in Emacs

[समीर सिंह Sameer Singh]

[-- Attachment #1.1: Type: text/plain, Size: 2410 bytes --]

In order to find the "real meat" of the display_line function I started to
slowly remove most of the lines of the
function as long as it does not break emacs and text can be typed.

In the end I was left with this a 20-25 lines function (excluding comments
and spaces):
(I have attached this function too)

static bool
display_line (struct it *it, int cursor_vpos)
{
  struct glyph_row *row = it->glyph_row;

  /* Clear the result glyph row and enable it.  */
  prepare_desired_row (it->w, row, false);

  row->y = it->current_y;

  /* Loop generating characters.  The loop is left with IT on the next
     character to display.  */
  while (true)
    {

      /* Retrieve the next thing to display.  Value is false if end of
buffer reached.  */
      if (!get_next_display_element (it))
{
 break;
}

      PRODUCE_GLYPHS (it);

    at_end_of_line:
      /* Is this a line end?  If yes, we're also done, after making
sure that a non-default face is extended up to the right
margin of the window.  */
      if (ITERATOR_AT_END_OF_LINE_P (it))
{
 /* Consume the line end.  This skips over invisible lines.  */
 set_iterator_to_next (it, true);
 break;
}

      set_iterator_to_next (it, true);
    }

  /* Compute pixel dimensions of this line.  */
  compute_line_metrics (it);

  /* Prepare for the next line.  This line starts horizontally at (X
     HPOS) = (0 0).  Vertical positions are incremented.  As a
     convenience for the caller, IT->glyph_row is set to the next
     row to be used.  */

  it->current_y += row->height;
  ++it->glyph_row;
  return MATRIX_ROW_DISPLAYS_TEXT_P (row);
}

Obviously most of the things do not work such as bidi, word wrapping,
displaying cursor and line numbers but still text is being shown
in rows one after the other and it can be scrolled, but now there are no
lines mentioning the hpos, the x-coordinate, first_visible_x or
last_visible_x, does that mean I do not understand the display_line
function? I thought its function was to fill a row with glyphs in the
desired matrix
to display it on the glass but now I cannot find a line which fills the
matrix. You had advised to swap the x and y coordinates, but here there
is no x coordinate present! How can the redisplay still work?


>> Regrettably my inexperience  is causing hurdles therefore any guidance
would be a huge help.

> Don't give up.  And thanks for working on this.

Thank you for bearing with me.

[-- Attachment #1.2: Type: text/html, Size: 3063 bytes --]

[-- Attachment #2: display.c --]
[-- Type: text/x-csrc, Size: 1331 bytes --]

static bool
display_line (struct it *it, int cursor_vpos)
{
  struct glyph_row *row = it->glyph_row;
 
  /* Clear the result glyph row and enable it.  */
  prepare_desired_row (it->w, row, false);

  row->y = it->current_y;
  
  /* Loop generating characters.  The loop is left with IT on the next
     character to display.  */
  while (true)
    {
            
      /* Retrieve the next thing to display.  Value is false if end of
	 buffer reached.  */
      if (!get_next_display_element (it))
	{
	  break;
	}

      PRODUCE_GLYPHS (it);
      
    at_end_of_line:
      /* Is this a line end?  If yes, we're also done, after making
	 sure that a non-default face is extended up to the right
	 margin of the window.  */
      if (ITERATOR_AT_END_OF_LINE_P (it))
	{	  
	  /* Consume the line end.  This skips over invisible lines.  */
	  set_iterator_to_next (it, true);
	  break;
	}

      set_iterator_to_next (it, true);
    }

  /* Compute pixel dimensions of this line.  */
  compute_line_metrics (it);

  /* Prepare for the next line.  This line starts horizontally at (X
     HPOS) = (0 0).  Vertical positions are incremented.  As a
     convenience for the caller, IT->glyph_row is set to the next
     row to be used.  */
  it->current_y += row->height;
  ++it->glyph_row;
  return MATRIX_ROW_DISPLAYS_TEXT_P (row);
}

Re: Implementing Vertical Text support in Emacs

[Eli Zaretskii]

> From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> Date: Sat, 10 Sep 2022 21:23:32 +0530
> Cc: emacs-devel@gnu.org
> 
> In order to find the "real meat" of the display_line function I started to slowly remove most of the lines of the
> function as long as it does not break emacs and text can be typed.
> 
> In the end I was left with this a 20-25 lines function (excluding comments and spaces):
> (I have attached this function too)
> 
> static bool
> display_line (struct it *it, int cursor_vpos)
> {
>   struct glyph_row *row = it->glyph_row;
>  
>   /* Clear the result glyph row and enable it.  */
>   prepare_desired_row (it->w, row, false);
> 
>   row->y = it->current_y;
>   
>   /* Loop generating characters.  The loop is left with IT on the next
>      character to display.  */
>   while (true)
>     {
>             
>       /* Retrieve the next thing to display.  Value is false if end of
> buffer reached.  */
>       if (!get_next_display_element (it))
> {
>  break;
> }
> 
>       PRODUCE_GLYPHS (it);
>       
>     at_end_of_line:
>       /* Is this a line end?  If yes, we're also done, after making
> sure that a non-default face is extended up to the right
> margin of the window.  */
>       if (ITERATOR_AT_END_OF_LINE_P (it))
> {  
>  /* Consume the line end.  This skips over invisible lines.  */
>  set_iterator_to_next (it, true);
>  break;
> }
> 
>       set_iterator_to_next (it, true);
>     }
> 
>   /* Compute pixel dimensions of this line.  */
>   compute_line_metrics (it);
> 
>   /* Prepare for the next line.  This line starts horizontally at (X
>      HPOS) = (0 0).  Vertical positions are incremented.  As a
>      convenience for the caller, IT->glyph_row is set to the next
>      row to be used.  */
> 
>   it->current_y += row->height;
>   ++it->glyph_row;
>   return MATRIX_ROW_DISPLAYS_TEXT_P (row);
> }
> 
> Obviously most of the things do not work such as bidi, word wrapping, displaying cursor and line numbers but
> still text is being shown
> in rows one after the other and it can be scrolled, but now there are no lines mentioning the hpos, the
> x-coordinate, first_visible_x or 
> last_visible_x, does that mean I do not understand the display_line function? I thought its function was to fill a
> row with glyphs in the desired matrix
> to display it on the glass but now I cannot find a line which fills the matrix.

That hides behind the PRODUCE_GLYPHS macro.

> You had advised to swap the x and y
> coordinates, but here there
> is no x coordinate present! How can the redisplay still work?

display_line also makes the decisions when the screen line is full and
the rest should be on the next screen line this is one of its most
important roles.  But you have removed that code, so you don't see it
in what's left.  The parts that deal with the X coordinate that
exceeds last_visible_x are those which make that decision.

Re: Implementing Vertical Text support in Emacs

[समीर सिंह Sameer Singh]

[-- Attachment #1: Type: text/plain, Size: 3671 bytes --]

>
> Before you modify display_line, you need to decide how will the
> vertical-layout display produce glyphs.


Can you please expand more on this?
How to produce glyphs in a column wise order?
I looked at gui_produce_glyphs and that led me to append_glyph, but I could
not figure out how the glyphs are produced.
There is also draw_glyphs but I think working with that will require
editing the low level *term files (if it even is the right function)

Thanks

On Sat, Sep 10, 2022 at 9:47 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> > Date: Sat, 10 Sep 2022 21:23:32 +0530
> > Cc: emacs-devel@gnu.org
> >
> > In order to find the "real meat" of the display_line function I started
> to slowly remove most of the lines of the
> > function as long as it does not break emacs and text can be typed.
> >
> > In the end I was left with this a 20-25 lines function (excluding
> comments and spaces):
> > (I have attached this function too)
> >
> > static bool
> > display_line (struct it *it, int cursor_vpos)
> > {
> >   struct glyph_row *row = it->glyph_row;
> >
> >   /* Clear the result glyph row and enable it.  */
> >   prepare_desired_row (it->w, row, false);
> >
> >   row->y = it->current_y;
> >
> >   /* Loop generating characters.  The loop is left with IT on the next
> >      character to display.  */
> >   while (true)
> >     {
> >
> >       /* Retrieve the next thing to display.  Value is false if end of
> > buffer reached.  */
> >       if (!get_next_display_element (it))
> > {
> >  break;
> > }
> >
> >       PRODUCE_GLYPHS (it);
> >
> >     at_end_of_line:
> >       /* Is this a line end?  If yes, we're also done, after making
> > sure that a non-default face is extended up to the right
> > margin of the window.  */
> >       if (ITERATOR_AT_END_OF_LINE_P (it))
> > {
> >  /* Consume the line end.  This skips over invisible lines.  */
> >  set_iterator_to_next (it, true);
> >  break;
> > }
> >
> >       set_iterator_to_next (it, true);
> >     }
> >
> >   /* Compute pixel dimensions of this line.  */
> >   compute_line_metrics (it);
> >
> >   /* Prepare for the next line.  This line starts horizontally at (X
> >      HPOS) = (0 0).  Vertical positions are incremented.  As a
> >      convenience for the caller, IT->glyph_row is set to the next
> >      row to be used.  */
> >
> >   it->current_y += row->height;
> >   ++it->glyph_row;
> >   return MATRIX_ROW_DISPLAYS_TEXT_P (row);
> > }
> >
> > Obviously most of the things do not work such as bidi, word wrapping,
> displaying cursor and line numbers but
> > still text is being shown
> > in rows one after the other and it can be scrolled, but now there are no
> lines mentioning the hpos, the
> > x-coordinate, first_visible_x or
> > last_visible_x, does that mean I do not understand the display_line
> function? I thought its function was to fill a
> > row with glyphs in the desired matrix
> > to display it on the glass but now I cannot find a line which fills the
> matrix.
>
> That hides behind the PRODUCE_GLYPHS macro.
>
> > You had advised to swap the x and y
> > coordinates, but here there
> > is no x coordinate present! How can the redisplay still work?
>
> display_line also makes the decisions when the screen line is full and
> the rest should be on the next screen line this is one of its most
> important roles.  But you have removed that code, so you don't see it
> in what's left.  The parts that deal with the X coordinate that
> exceeds last_visible_x are those which make that decision.
>

[-- Attachment #2: Type: text/html, Size: 4806 bytes --]

Re: Implementing Vertical Text support in Emacs

[Eli Zaretskii]

> From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> Date: Fri, 30 Sep 2022 01:27:01 +0530
> Cc: emacs-devel@gnu.org
> 
>  Before you modify display_line, you need to decide how will the
>  vertical-layout display produce glyphs.
> 
> Can you please expand more on this?
> How to produce glyphs in a column wise order?

I did that in the original message.  Quoting myself:

  Before you modify display_line, you need to decide how will the
  vertical-layout display produce glyphs.  The easiest would be to
  produce them in column-wise order, because that basically processes
  the buffer in the order of character positions, like Emacs does now.
  This will allow you to leave the lower levels, which walk the buffer
  and deliver characters, mostly intact.

  So perhaps you should have a display_column function, which will look
  like display_line, but instead of advancing the X coordinate, it will
  advance the Y coordinate, and produce a single column, top to bottom.
  Then the basic calculations should be the same as in display_line,
  except the column ends when it reaches last_visible_y instead of
  last_visible_x.  IOW, the glyph matrices will be made of glyph columns
  instead of glyph rows.  Basically, the roles of the X and the Y
  coordinates will be swapped.

> I looked at gui_produce_glyphs and that led me to append_glyph, but I could not figure out how the glyphs are
> produced.
> There is also draw_glyphs but I think working with that will require editing the low level *term files (if it even is
> the right function)

I didn't mean gui_produce_glyphs and its subroutines, I mean the logic
of how glyphs are produced, as described above, in display_line
itself.  The current code in display_line produces glyph of a single
screen line, in visual order, from left to right.  Above I suggested
to write a similar display_column function, which will produce glyphs
of a single column, top to bottom, and will make the decisions
regarding when the column is full and should be terminated, like
display_line does with respect to screen lines.  Does this sounds like
a good approach to you?

If the approach sounds good, but something in my description is
unclear, please ask more specific questions about those unclear parts
of what I wrote.

Re: Implementing Vertical Text support in Emacs

[समीर सिंह Sameer Singh]

[-- Attachment #1: Type: text/plain, Size: 3137 bytes --]

I tried writing a vertical version of the display_line function but could
not get it to work. Emacs does not display any character after these
changes.
(maybe I should have also rewrote move_it_in_display_line_to?)

Therefore forgive me for asking again, if for now I just want to make the
characters appear column wise instead of row wise, disregarding line
overflow, or termination of a line etc,
do I still have to look into the display_line function, because as I recall
I had once removed most of the lines of the display_line function and Emacs
was still displaying lines fines.

Again, thank you for your patience.

On Fri, Sep 30, 2022 at 12:00 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> > Date: Fri, 30 Sep 2022 01:27:01 +0530
> > Cc: emacs-devel@gnu.org
> >
> >  Before you modify display_line, you need to decide how will the
> >  vertical-layout display produce glyphs.
> >
> > Can you please expand more on this?
> > How to produce glyphs in a column wise order?
>
> I did that in the original message.  Quoting myself:
>
>   Before you modify display_line, you need to decide how will the
>   vertical-layout display produce glyphs.  The easiest would be to
>   produce them in column-wise order, because that basically processes
>   the buffer in the order of character positions, like Emacs does now.
>   This will allow you to leave the lower levels, which walk the buffer
>   and deliver characters, mostly intact.
>
>   So perhaps you should have a display_column function, which will look
>   like display_line, but instead of advancing the X coordinate, it will
>   advance the Y coordinate, and produce a single column, top to bottom.
>   Then the basic calculations should be the same as in display_line,
>   except the column ends when it reaches last_visible_y instead of
>   last_visible_x.  IOW, the glyph matrices will be made of glyph columns
>   instead of glyph rows.  Basically, the roles of the X and the Y
>   coordinates will be swapped.
>
> > I looked at gui_produce_glyphs and that led me to append_glyph, but I
> could not figure out how the glyphs are
> > produced.
> > There is also draw_glyphs but I think working with that will require
> editing the low level *term files (if it even is
> > the right function)
>
> I didn't mean gui_produce_glyphs and its subroutines, I mean the logic
> of how glyphs are produced, as described above, in display_line
> itself.  The current code in display_line produces glyph of a single
> screen line, in visual order, from left to right.  Above I suggested
> to write a similar display_column function, which will produce glyphs
> of a single column, top to bottom, and will make the decisions
> regarding when the column is full and should be terminated, like
> display_line does with respect to screen lines.  Does this sounds like
> a good approach to you?
>
> If the approach sounds good, but something in my description is
> unclear, please ask more specific questions about those unclear parts
> of what I wrote.
>

[-- Attachment #2: Type: text/html, Size: 3771 bytes --]

Re: Implementing Vertical Text support in Emacs

[Eli Zaretskii]

> From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> Date: Sun, 20 Nov 2022 16:48:31 +0530
> Cc: emacs-devel@gnu.org
> 
> I tried writing a vertical version of the display_line function but could not get it to work. Emacs does not
> display any character after these changes.
> (maybe I should have also rewrote move_it_in_display_line_to?)

I don't think move_it_in_display_line_to is relevant at this early stage of
your work.

> Therefore forgive me for asking again, if for now I just want to make the characters appear column wise
> instead of row wise, disregarding line overflow, or termination of a line etc,
> do I still have to look into the display_line function, because as I recall I had once removed most of the lines
> of the display_line function and Emacs was still displaying lines fines.

I'm not sure I understand the question.  Are you saying that if you ignore
the line overflow, termination of a line, etc., you can write code that does
display characters column-wise?

In any case, it is hard to discuss this without seeing the code which
doesn't work.  Is the code you wrote available anywhere for review?

Re: Implementing Vertical Text support in Emacs

[समीर सिंह Sameer Singh]

[-- Attachment #1: Type: text/plain, Size: 2196 bytes --]

>
> I'm not sure I understand the question.  Are you saying that if you ignore
> the line overflow, termination of a line, etc., you can write code that
> does
> display characters column-wise?


I thought that displaying line row wise is the "base" of the display_line
function with code for
line truncation, overflow, bidi etc added on top of it in "if" blocks, so
if I could just get the base to work (i.e. displaying lines in a column)
then I could work on the the finer details.
Am I wrong?

In any case, it is hard to discuss this without seeing the code which
> doesn't work.  Is the code you wrote available anywhere for review?
>

Unfortunately it is no longer available.
One line which I remember causing a problem though was line 24500 in
src/xdisp.c
row->y = it->current_y;
if I changed it to: row->x = it->current_x; Emacs no longer displays
characters.


On Sun, Nov 20, 2022 at 5:13 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> > Date: Sun, 20 Nov 2022 16:48:31 +0530
> > Cc: emacs-devel@gnu.org
> >
> > I tried writing a vertical version of the display_line function but
> could not get it to work. Emacs does not
> > display any character after these changes.
> > (maybe I should have also rewrote move_it_in_display_line_to?)
>
> I don't think move_it_in_display_line_to is relevant at this early stage of
> your work.
>
> > Therefore forgive me for asking again, if for now I just want to make
> the characters appear column wise
> > instead of row wise, disregarding line overflow, or termination of a
> line etc,
> > do I still have to look into the display_line function, because as I
> recall I had once removed most of the lines
> > of the display_line function and Emacs was still displaying lines fines.
>
> I'm not sure I understand the question.  Are you saying that if you ignore
> the line overflow, termination of a line, etc., you can write code that
> does
> display characters column-wise?
>
> In any case, it is hard to discuss this without seeing the code which
> doesn't work.  Is the code you wrote available anywhere for review?
>

[-- Attachment #2: Type: text/html, Size: 3106 bytes --]

Re: Implementing Vertical Text support in Emacs

[Eli Zaretskii]

> From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> Date: Sun, 20 Nov 2022 17:38:05 +0530
> Cc: emacs-devel@gnu.org
> 
>  I'm not sure I understand the question.  Are you saying that if you ignore
>  the line overflow, termination of a line, etc., you can write code that does
>  display characters column-wise?
> 
> I thought that displaying line row wise is the "base" of the display_line function with code for
> line truncation, overflow, bidi etc added on top of it in "if" blocks

That is basically correct.

> so if I could just get the base to work (i.e.  displaying lines in a
> column) then I could work on the the finer details.  Am I wrong?

No, you are not wrong.  But the base of display_line is very simple:

  prepare_desired_row (it->w, row, false);

  row->y = it->current_y;
  row->start = it->start;
  row->continuation_lines_width = it->continuation_lines_width;
  row->displays_text_p = true;
  row->starts_in_middle_of_char_p = it->starts_in_middle_of_char_p;
  it->starts_in_middle_of_char_p = false;
  it->stretch_adjust = 0;
  row->ascent = it->max_ascent;
  row->height = it->max_ascent + it->max_descent;
  row->phys_ascent = it->max_phys_ascent;
  row->phys_height = it->max_phys_ascent + it->max_phys_descent;
  row->extra_line_spacing = it->max_extra_line_spacing;
  while (true)
    {
      if (!get_next_display_element (it))
	{
	  row->ends_at_zv_p = true;
	  break;
	}

      PRODUCE_GLYPHS (it);
      if (/* Not a newline.  */
	  nglyphs > 0
	  /* Glyphs produced fit entirely in the line.  */
	  && it->current_x < it->last_visible_x)
	{
	  it->hpos += nglyphs;
	  row->ascent = max (row->ascent, it->max_ascent);
	  row->height = max (row->height, it->max_ascent + it->max_descent);
	  row->phys_ascent = max (row->phys_ascent, it->max_phys_ascent);
	  row->phys_height = max (row->phys_height,
				  it->max_phys_ascent + it->max_phys_descent);
	}
      else
	{
	  ++it->hpos;
	  row->ascent = max (row->ascent, it->max_ascent);
	  row->height = max (row->height, it->max_ascent + it->max_descent);
	  row->phys_ascent = max (row->phys_ascent, it->max_phys_ascent);
	  row->phys_height = max (row->phys_height,
				  it->max_phys_ascent + it->max_phys_descent);
	  row->extra_line_spacing = max (row->extra_line_spacing,
					 it->max_extra_line_spacing);
	}
      if (ITERATOR_AT_END_OF_LINE_P (it))
	{
	  /* Add a space at the end of the line that is used to
	     display the cursor there.  */
	  if (!IT_OVERFLOW_NEWLINE_INTO_FRINGE (it))
	    append_space_for_newline (it, false);

	  /* Extend the face to the end of the line.  */
	  extend_face_to_end_of_line (it);

	  /* Consume the line end.  This skips over invisible lines.  */
	  set_iterator_to_next (it, true);
	  it->continuation_lines_width = 0;
	  break;
	}
    }
  compute_line_metrics (it);

Basically, it loops calling get_next_display_element, then PRODUCE_GLYPHS,
and then set_iterator_to_next.