bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 5055 bytes --]

I can reproduce the bug in the following configuration:
- Large monitor or small font in terminal.
- Having 2 vertically splitted windows.
- Enable `display-line-number-mode` on the left pane.
- Terminal flickers on scrooling on some lines.

The main point is it flickers only when the right pane has a little
content. If the file contents fits into the whole right window, the it
doesn't flicker, and it flickers only on the lines which do not have
content to display. For example,
--------------------
|        |         |
|        |         |
|        |~        |
| *|*    |~        |
|        |~        |
|        |~        |
--------------------

"~" is the part where there is no content. When I use the left window
and scroll on the lines where the right windown doesn't have content,
the screen flickers. But, as long, as I open some large file in the
right pange it works as expected w/o any flickering.

I tried to find a possible way to put some content in there, but seems
like emacs supports frigne only on GUI.

I tried different terminal emulators (iTerm2, Alacritty), in additional
to w/ and w/o tmux.

The same configuration but with ssh to linux work perfectly well. So, I
assume, it excudes terminal emulator issues.

I also tried different emacs distributions:
 - https://emacsformacosx.com/
 - https://github.com/jimeh/emacs-builds
 - Some with come homebrew as well.

Do you have any possible ideas where I can look into it?



In GNU Emacs 28.1.91 (build 1, x86_64-apple-darwin20.6.0, NS appkit-2202.70
Version 11.6.8 (Build 20G730))
 of 2022-08-22 built on Mac-1661214538296.local
Repository revision: bfa5bcf79b5069126308664c1701f86d253df337
Repository branch: HEAD
System Description:  macOS 12.5.1

Configured using:
 'configure --with-ns --with-modules
 '--enable-locallisppath=/Library/Application
 Support/Emacs/${version}/site-lisp:/Library/Application
 Support/Emacs/site-lisp:/usr/local/share/emacs/site-lisp'
 --with-xwidgets --with-native-compilation
 'CFLAGS=-I/usr/local/opt/gcc/include -I/usr/local/opt/libgccjit/include
 -O2' 'LDFLAGS=-L/usr/local/opt/gcc/lib/gcc/12
 -L/usr/local/opt/gcc/lib/gcc/12/gcc/x86_64-apple-darwin20/12
 -L/usr/local/opt/libgccjit/lib/gcc/12 -I/usr/local/opt/gcc/include
 -I/usr/local/opt/libgccjit/include -Wl,-headerpad_max_install_names''

Configured features:
ACL DBUS GIF GLIB GMP GNUTLS JPEG JSON LCMS2 LIBXML2 MODULES NATIVE_COMP
NOTIFY KQUEUE NS PDUMPER PNG RSVG THREADS TIFF TOOLKIT_SCROLL_BARS XIM
XWIDGETS ZLIB

Important settings:
  value of $LC_ALL: en_US.UTF-8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: C++//l

Minor modes in effect:
  display-line-numbers-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  abbrev-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny rfc822 mml mml-sec epa
derived epg rfc6068 epg-config gnus-util rmail rmail-loaddefs
auth-source eieio eieio-core eieio-loaddefs password-cache json map
text-property-search time-date mm-decode mm-bodies mm-encode mail-parse
rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mail-utils comp comp-cstr warnings subr-x
rx cl-seq cl-macs cl-extra help-mode seq display-line-numbers cc-mode
cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars
cc-defs dired-aux cl-loaddefs cl-lib dired dired-loaddefs term/xterm
xterm byte-opt gv bytecomp byte-compile cconv iso-transl tooltip eldoc
paren electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode
mwheel term/ns-win ns-win ucs-normalize mule-util term/common-win
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget hashtable-print-readable backquote threads
xwidget-internal dbusbind kqueue cocoa ns lcms2 multi-tty
make-network-process native-compile emacs)

Memory information:
((conses 16 133662 7351)
 (symbols 48 9789 0)
 (strings 32 28712 1544)
 (string-bytes 1 1059484)
 (vectors 16 17862)
 (vector-slots 8 336624 10901)
 (floats 8 32 218)
 (intervals 56 2454 0)
 (buffers 992 16))

[-- Attachment #2: Type: text/html, Size: 5772 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Dmitrii Kuragin <kuragin@google.com> writes:

> I can reproduce the bug in the following configuration:
> - Large monitor or small font in terminal.
> - Having 2 vertically splitted windows.
> - Enable `display-line-number-mode` on the left pane.
> - Terminal flickers on scrooling on some lines.
>
> The main point is it flickers only when the right pane has a little
> content. If the file contents fits into the whole right window, the it
> doesn't flicker, and it flickers only on the lines which do not have
> content to display. For example,
> --------------------
> |        |         |
> |        |         |
> |        |~        |
> | *|*    |~        |
> |        |~        |
> |        |~        |
> --------------------
>
> "~" is the part where there is no content. When I use the left window
> and scroll on the lines where the right windown doesn't have content,
> the screen flickers. But, as long, as I open some large file in the
> right pange it works as expected w/o any flickering.
>
> I tried to find a possible way to put some content in there, but seems
> like emacs supports frigne only on GUI.
>
> I tried different terminal emulators (iTerm2, Alacritty), in additional
> to w/ and w/o tmux.
>
> The same configuration but with ssh to linux work perfectly well. So, I
> assume, it excudes terminal emulator issues.
>
> I also tried different emacs distributions:
>  - https://emacsformacosx.com/
>  - https://github.com/jimeh/emacs-builds
>  - Some with come homebrew as well.
>
> Do you have any possible ideas where I can look into it?

I tried your recipe here with emacs -Q (Emacs 28.1 from Homebrew) in a
maximaized Terminal.app window, with a font as tiny as I could get (with
Command +/-).  I could not reproduce the flickering.

Does this happen for you with emacs -Q in Terminal?

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> I tried your recipe here with emacs -Q (Emacs 28.1 from Homebrew) in a
> maximaized Terminal.app window, with a font as tiny as I could get (with
> Command +/-).  I could not reproduce the flickering.
>
> Does this happen for you with emacs -Q in Terminal?

BTW. this was

GNU Emacs 28.1 (build 1, aarch64-apple-darwin21.3.0) of 2022-04-30

and I'm running macOS 12.5.1.

Maybe someone else having access to maxOS 11 can reproduce this?

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1359 bytes --]

hm... I tried it in Terminal.app as well and it flickers less, likely
because it uses 256 colors, whereas in alacritty or iTerm2, I use 24 bit
colors.

I tried to record a video of the behavior:
https://drive.google.com/file/d/1nMf_3MxRk2cTdgF3tFmzAZoxcy3vQghc/view?usp=sharing

On Fri, Aug 26, 2022 at 10:41 PM Gerd Möllmann <gerd.moellmann@gmail.com>
wrote:

> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>
> > I tried your recipe here with emacs -Q (Emacs 28.1 from Homebrew) in a
> > maximaized Terminal.app window, with a font as tiny as I could get (with
> > Command +/-).  I could not reproduce the flickering.
> >
> > Does this happen for you with emacs -Q in Terminal?
>
> BTW. this was
>
> GNU Emacs 28.1 (build 1, aarch64-apple-darwin21.3.0) of 2022-04-30
>
> and I'm running macOS 12.5.1.
>
> Maybe someone else having access to maxOS 11 can reproduce this?
>


-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 2976 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1.1: Type: text/plain, Size: 2148 bytes --]

Here's a very interesting patch which fixes the flickering issue for me.

Maybe we do something inaccurate during the cost calculation? Or we use
some metric which is note representable on macos?

On Sat, Aug 27, 2022 at 8:03 AM Dmitrii Kuragin <kuragin@google.com> wrote:

> hm... I tried it in Terminal.app as well and it flickers less, likely
> because it uses 256 colors, whereas in alacritty or iTerm2, I use 24 bit
> colors.
>
> I tried to record a video of the behavior:
> https://drive.google.com/file/d/1nMf_3MxRk2cTdgF3tFmzAZoxcy3vQghc/view?usp=sharing
>
> On Fri, Aug 26, 2022 at 10:41 PM Gerd Möllmann <gerd.moellmann@gmail.com>
> wrote:
>
>> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>>
>> > I tried your recipe here with emacs -Q (Emacs 28.1 from Homebrew) in a
>> > maximaized Terminal.app window, with a font as tiny as I could get (with
>> > Command +/-).  I could not reproduce the flickering.
>> >
>> > Does this happen for you with emacs -Q in Terminal?
>>
>> BTW. this was
>>
>> GNU Emacs 28.1 (build 1, aarch64-apple-darwin21.3.0) of 2022-04-30
>>
>> and I'm running macOS 12.5.1.
>>
>> Maybe someone else having access to maxOS 11 can reproduce this?
>>
>
>
> --
> *If you get an email from me outside of the 9-5 it is *not* because I'm
> always on or expect an immediate response from you; it is because of work
> flexibility
> <http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
> .  Evening and weekend emails are a sign I allocated some regular working
> hours for other things (such as family, gym, friends,...).  And I encourage
> you to feel free to do the same.
>
>

-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #1.2: Type: text/html, Size: 5103 bytes --]

[-- Attachment #2: fix_flickering_on_macos.patch --]
[-- Type: application/octet-stream, Size: 2075 bytes --]

diff --git a/src/scroll.c b/src/scroll.c
index c643730965..fac29e67e8 100644
--- a/src/scroll.c
+++ b/src/scroll.c
@@ -687,30 +687,30 @@ do_direct_scrolling (struct frame *frame, struct glyph_matrix *current_matrix,
     {
       p = cost_matrix + i * (window_size + 1) + j;
 
-      if (p->insertcost < p->writecost
-	  && p->insertcost < p->deletecost
-	  && (write_follows_p || i < j))
-	{
-	  /* Insert is cheaper than deleting or writing lines.  Leave
-	     a hole in the result display that will be filled with
-	     empty lines when the queue is emptied.  */
-	  queue->count = 0;
-	  queue->window = i;
-	  queue->pos = i - p->insertcount;
-	  ++queue;
-
-	  i -= p->insertcount;
-	  write_follows_p = 0;
-	}
-      else if (p->deletecost < p->writecost
-	       && (write_follows_p || i > j))
-	{
-	  /* Deleting lines is cheaper.  By decrementing J, omit
-	     deletecount lines from the original.  */
-	  write_follows_p = 0;
-	  j -= p->deletecount;
-	}
-      else
+      /* if (p->insertcost < p->writecost */
+      /* 	  && p->insertcost < p->deletecost */
+      /* 	  && (write_follows_p || i < j)) */
+      /* 	{ */
+      /* 	  /\* Insert is cheaper than deleting or writing lines.  Leave */
+      /* 	     a hole in the result display that will be filled with */
+      /* 	     empty lines when the queue is emptied.  *\/ */
+      /* 	  queue->count = 0; */
+      /* 	  queue->window = i; */
+      /* 	  queue->pos = i - p->insertcount; */
+      /* 	  ++queue; */
+
+      /* 	  i -= p->insertcount; */
+      /* 	  write_follows_p = 0; */
+      /* 	} */
+      /* else if (p->deletecost < p->writecost */
+      /* 	       && (write_follows_p || i > j)) */
+      /* 	{ */
+      /* 	  /\* Deleting lines is cheaper.  By decrementing J, omit */
+      /* 	     deletecount lines from the original.  *\/ */
+      /* 	  write_follows_p = 0; */
+      /* 	  j -= p->deletecount; */
+      /* 	} */
+      /* else */
 	{
 	  /* One or more lines should be written.  In the direct
 	     scrolling method we do this by scrolling the lines to the

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> Cc: 57434@debbugs.gnu.org
> Date: Sat, 27 Aug 2022 08:46:44 -0700
> From:  Dmitrii Kuragin via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> Here's a very interesting patch which fixes the flickering issue for me.
> 
> Maybe we do something inaccurate during the cost calculation? Or we use some metric which is note
> representable on macos?

This has come up before, but we could never understand what's wrong
with that calculation.  Perhaps you could step in a debugger through
that code and tell what you see and why it flickers?

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1591 bytes --]

I think I can give it a try. I just need a bit more time to set up the
debugger workflow, since I run GDM only once in a few years :)

I also do not really understand the meaning of "cost" here and what metrics
we use to measure that. But, I'd assume it is something empirical.

Since we found the point which fixes it (cost calculation) I will try to
understand how it works better. and How having content on the right window
might affect the cost calculation.

On Sat, Aug 27, 2022 at 8:57 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > Cc: 57434@debbugs.gnu.org
> > Date: Sat, 27 Aug 2022 08:46:44 -0700
> > From:  Dmitrii Kuragin via "Bug reports for GNU Emacs,
> >  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> >
> > Here's a very interesting patch which fixes the flickering issue for me.
> >
> > Maybe we do something inaccurate during the cost calculation? Or we use
> some metric which is note
> > representable on macos?
>
> This has come up before, but we could never understand what's wrong
> with that calculation.  Perhaps you could step in a debugger through
> that code and tell what you see and why it flickers?
>


-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 3380 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Dmitrii Kuragin <kuragin@google.com>
> Date: Sat, 27 Aug 2022 09:01:22 -0700
> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>, 
> 	57434@debbugs.gnu.org
> 
> I think I can give it a try. I just need a bit more time to set up the debugger workflow, since I run GDM only once
> in a few years :) 

Thanks.

> I also do not really understand the meaning of "cost" here and what metrics we use to measure that. But, I'd
> assume it is something empirical.

It's supposed to measure the cost of moving the text-terminal cursor
from one point on the screen to another.  And yes, it's heuristics.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 2703 bytes --]

I am having difficulty running a debugger.

I tried gdb and signing, but it didn't work. I also tried lldb, but it
doesn't stop on a breakpoint for whatever reason.

I compiled with `-O0 -g3`, then
```
lldb
(lldb) file nextstep/Emacs.app/Contents/MacOS/Emacs
Current executable set to
'/Users/kuragin/Desktop/emacs/nextstep/Emacs.app/Contents/MacOS/Emacs'
(x86_64).
(lldb) breakpoint set -f scroll.c -l 270
Breakpoint 1: where = Emacs`do_scrolling + 485 at scroll.c:271:11, address
= 0x0000000100032da5
```

But, it doesn't stop there...

and I run Emacs like this:
```
arch --x86_64 make configure="CFLAGS='-O0 -g3'" -j 20 &&
nextstep/Emacs.app/Contents/MacOS/Emacs -nw
```

I can confirm that my patch fixes the problem for me, but I am not
confident that the issue is in the estimation cost.

When I have line numbers enabled, I assume, the scrolling logic would
always try to insert/delete/write lines. In my case it might be:
 - Writing (Is that writing on top of the current lines?) is cheaper.
 - Screen flickers because of the specific frequency of the terminal (or
the way we flush the buffer).
   For example, we insert empty lines and then the screen is updated, only
then we add content in there and redisplay again.

Potentially, some redrawing might happen inside of `ins_del_lines`? Instead
of redrawing the whole screen, we redraw it in the middle of modifying it?

Those are just my assumptions from reading the code.

I'd appreciate any help in debugging the issue.

On Sat, Aug 27, 2022 at 9:14 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Dmitrii Kuragin <kuragin@google.com>
> > Date: Sat, 27 Aug 2022 09:01:22 -0700
> > Cc: Gerd Möllmann <gerd.moellmann@gmail.com>,
> >       57434@debbugs.gnu.org
> >
> > I think I can give it a try. I just need a bit more time to set up the
> debugger workflow, since I run GDM only once
> > in a few years :)
>
> Thanks.
>
> > I also do not really understand the meaning of "cost" here and what
> metrics we use to measure that. But, I'd
> > assume it is something empirical.
>
> It's supposed to measure the cost of moving the text-terminal cursor
> from one point on the screen to another.  And yes, it's heuristics.
>


-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 6244 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 3696 bytes --]

I also tried vim in the similar configuration (display line numbers, 2
splits, etc).

I understand that it is unreasonable to compare 2 different things, but it
doesn't show any flickering issues there.

Do we actually need to redraw the whole line if we use relative numbers or
we can just redraw the portion with line numbers?

On Mon, Aug 29, 2022 at 7:18 AM Dmitrii Kuragin <kuragin@google.com> wrote:

> I am having difficulty running a debugger.
>
> I tried gdb and signing, but it didn't work. I also tried lldb, but it
> doesn't stop on a breakpoint for whatever reason.
>
> I compiled with `-O0 -g3`, then
> ```
> lldb
> (lldb) file nextstep/Emacs.app/Contents/MacOS/Emacs
> Current executable set to
> '/Users/kuragin/Desktop/emacs/nextstep/Emacs.app/Contents/MacOS/Emacs'
> (x86_64).
> (lldb) breakpoint set -f scroll.c -l 270
> Breakpoint 1: where = Emacs`do_scrolling + 485 at scroll.c:271:11, address
> = 0x0000000100032da5
> ```
>
> But, it doesn't stop there...
>
> and I run Emacs like this:
> ```
> arch --x86_64 make configure="CFLAGS='-O0 -g3'" -j 20 &&
> nextstep/Emacs.app/Contents/MacOS/Emacs -nw
> ```
>
> I can confirm that my patch fixes the problem for me, but I am not
> confident that the issue is in the estimation cost.
>
> When I have line numbers enabled, I assume, the scrolling logic would
> always try to insert/delete/write lines. In my case it might be:
>  - Writing (Is that writing on top of the current lines?) is cheaper.
>  - Screen flickers because of the specific frequency of the terminal (or
> the way we flush the buffer).
>    For example, we insert empty lines and then the screen is updated, only
> then we add content in there and redisplay again.
>
> Potentially, some redrawing might happen inside of `ins_del_lines`?
> Instead of redrawing the whole screen, we redraw it in the middle of
> modifying it?
>
> Those are just my assumptions from reading the code.
>
> I'd appreciate any help in debugging the issue.
>
> On Sat, Aug 27, 2022 at 9:14 AM Eli Zaretskii <eliz@gnu.org> wrote:
>
>> > From: Dmitrii Kuragin <kuragin@google.com>
>> > Date: Sat, 27 Aug 2022 09:01:22 -0700
>> > Cc: Gerd Möllmann <gerd.moellmann@gmail.com>,
>> >       57434@debbugs.gnu.org
>> >
>> > I think I can give it a try. I just need a bit more time to set up the
>> debugger workflow, since I run GDM only once
>> > in a few years :)
>>
>> Thanks.
>>
>> > I also do not really understand the meaning of "cost" here and what
>> metrics we use to measure that. But, I'd
>> > assume it is something empirical.
>>
>> It's supposed to measure the cost of moving the text-terminal cursor
>> from one point on the screen to another.  And yes, it's heuristics.
>>
>
>
> --
> *If you get an email from me outside of the 9-5 it is *not* because I'm
> always on or expect an immediate response from you; it is because of work
> flexibility
> <http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
> .  Evening and weekend emails are a sign I allocated some regular working
> hours for other things (such as family, gym, friends,...).  And I encourage
> you to feel free to do the same.
>
>

-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 8659 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Dmitrii Kuragin <kuragin@google.com> writes:

> I am having difficulty running a debugger.
>
> I tried gdb and signing, but it didn't work. I also tried lldb, but it doesn't stop on a breakpoint for whatever reason.
>
> I compiled with `-O0 -g3`, then
> ```
> lldb
> (lldb) file nextstep/Emacs.app/Contents/MacOS/Emacs
> Current executable set to '/Users/kuragin/Desktop/emacs/nextstep/Emacs.app/Contents/MacOS/Emacs' (x86_64).
> (lldb) breakpoint set -f scroll.c -l 270
> Breakpoint 1: where = Emacs`do_scrolling + 485 at scroll.c:271:11, address = 0x0000000100032da5
> ```
>
> But, it doesn't stop there...
>
> and I run Emacs like this:
> ```
> arch --x86_64 make configure="CFLAGS='-O0 -g3'" -j 20 && nextstep/Emacs.app/Contents/MacOS/Emacs -nw

I'm afraid I can't help here. because GDB doesn't support my platform.
There is something about using GDB with TTY Emacs in etc/DEBUG though.
Maybe that helps.

LLDB doesn't work for me, neither the one from Apple, nor from LLVM 14.
For some reason, SIGTTOU seems to be behave differently when running
Emacs -nw under LLDB, even when I tell LLDB to not stop or report it.
Or so I think, I'm not an LLDB expert.

Here is what I tried

cd src
lldb emacs
b main
run -Q -nw
process handle -s false -n false SIGTTOU
c

but then Emacs gets stuck.  Maybe it's a bug in LLDB.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Dmitrii Kuragin <kuragin@google.com>
> Date: Mon, 29 Aug 2022 07:18:43 -0700
> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>, 
> 	57434@debbugs.gnu.org
> 
> I compiled with `-O0 -g3`, then
> ```
> lldb
> (lldb) file nextstep/Emacs.app/Contents/MacOS/Emacs
> Current executable set to '/Users/kuragin/Desktop/emacs/nextstep/Emacs.app/Contents/MacOS/Emacs'
> (x86_64).
> (lldb) breakpoint set -f scroll.c -l 270
> Breakpoint 1: where = Emacs`do_scrolling + 485 at scroll.c:271:11, address = 0x0000000100032da5
> ```
> 
> But, it doesn't stop there...

Why scroll.c:271, when the code you patched begins on line 684?

> When I have line numbers enabled, I assume, the scrolling logic would always try to insert/delete/write lines. In
> my case it might be:
>  - Writing (Is that writing on top of the current lines?) is cheaper.
>  - Screen flickers because of the specific frequency of the terminal (or the way we flush the buffer).
>    For example, we insert empty lines and then the screen is updated, only then we add content in there and
> redisplay again.
> 
> Potentially, some redrawing might happen inside of `ins_del_lines`? Instead of redrawing the whole screen,
> we redraw it in the middle of modifying it?

There shouldn't be any redrawing when none of the shown buffers
changes in any way.  You see flickering when Emacs is completely idle,
yes?

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 2263 bytes --]

No, I see flickering during scrolling.

And the problem is much worse when I have line number mode enabled.

The buffers do not change, and the frame at the same position, but line
numbers change because they're in 'visual mode (relative numbers).

On Mon, Aug 29, 2022 at 9:01 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Dmitrii Kuragin <kuragin@google.com>
> > Date: Mon, 29 Aug 2022 07:18:43 -0700
> > Cc: Gerd Möllmann <gerd.moellmann@gmail.com>,
> >       57434@debbugs.gnu.org
> >
> > I compiled with `-O0 -g3`, then
> > ```
> > lldb
> > (lldb) file nextstep/Emacs.app/Contents/MacOS/Emacs
> > Current executable set to
> '/Users/kuragin/Desktop/emacs/nextstep/Emacs.app/Contents/MacOS/Emacs'
> > (x86_64).
> > (lldb) breakpoint set -f scroll.c -l 270
> > Breakpoint 1: where = Emacs`do_scrolling + 485 at scroll.c:271:11,
> address = 0x0000000100032da5
> > ```
> >
> > But, it doesn't stop there...
>
> Why scroll.c:271, when the code you patched begins on line 684?
>
> > When I have line numbers enabled, I assume, the scrolling logic would
> always try to insert/delete/write lines. In
> > my case it might be:
> >  - Writing (Is that writing on top of the current lines?) is cheaper.
> >  - Screen flickers because of the specific frequency of the terminal (or
> the way we flush the buffer).
> >    For example, we insert empty lines and then the screen is updated,
> only then we add content in there and
> > redisplay again.
> >
> > Potentially, some redrawing might happen inside of `ins_del_lines`?
> Instead of redrawing the whole screen,
> > we redraw it in the middle of modifying it?
>
> There shouldn't be any redrawing when none of the shown buffers
> changes in any way.  You see flickering when Emacs is completely idle,
> yes?
>


-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 4152 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Dmitrii Kuragin <kuragin@google.com>
> Date: Mon, 29 Aug 2022 07:38:48 -0700
> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>, 
> 	57434@debbugs.gnu.org
> 
> Do we actually need to redraw the whole line if we use relative numbers or we can just redraw the portion with
> line numbers?

We need to redraw only the parts that have actually changed.

But even with relative line numbers, the numbers only change if you
move the cursor vertically.  I thought you've seen flickering even
when cursor doesn't move at all, and Emacs is completely idle.  isn't
that so?

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1183 bytes --]

No, the problem is during scrolling.

On Mon, Aug 29, 2022 at 9:03 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Dmitrii Kuragin <kuragin@google.com>
> > Date: Mon, 29 Aug 2022 07:38:48 -0700
> > Cc: Gerd Möllmann <gerd.moellmann@gmail.com>,
> >       57434@debbugs.gnu.org
> >
> > Do we actually need to redraw the whole line if we use relative numbers
> or we can just redraw the portion with
> > line numbers?
>
> We need to redraw only the parts that have actually changed.
>
> But even with relative line numbers, the numbers only change if you
> move the cursor vertically.  I thought you've seen flickering even
> when cursor doesn't move at all, and Emacs is completely idle.  isn't
> that so?
>


-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 2681 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1850 bytes --]

See
https://drive.google.com/file/d/1nMf_3MxRk2cTdgF3tFmzAZoxcy3vQghc/view?usp=sharing

On Mon, Aug 29, 2022 at 9:05 AM Dmitrii Kuragin <kuragin@google.com> wrote:

> No, the problem is during scrolling.
>
> On Mon, Aug 29, 2022 at 9:03 AM Eli Zaretskii <eliz@gnu.org> wrote:
>
>> > From: Dmitrii Kuragin <kuragin@google.com>
>> > Date: Mon, 29 Aug 2022 07:38:48 -0700
>> > Cc: Gerd Möllmann <gerd.moellmann@gmail.com>,
>> >       57434@debbugs.gnu.org
>> >
>> > Do we actually need to redraw the whole line if we use relative numbers
>> or we can just redraw the portion with
>> > line numbers?
>>
>> We need to redraw only the parts that have actually changed.
>>
>> But even with relative line numbers, the numbers only change if you
>> move the cursor vertically.  I thought you've seen flickering even
>> when cursor doesn't move at all, and Emacs is completely idle.  isn't
>> that so?
>>
>
>
> --
> *If you get an email from me outside of the 9-5 it is *not* because I'm
> always on or expect an immediate response from you; it is because of work
> flexibility
> <http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
> .  Evening and weekend emails are a sign I allocated some regular working
> hours for other things (such as family, gym, friends,...).  And I encourage
> you to feel free to do the same.
>
>

-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 4639 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  57434@debbugs.gnu.org
> Date: Mon, 29 Aug 2022 17:15:41 +0200
> 
> LLDB doesn't work for me, neither the one from Apple, nor from LLVM 14.
> For some reason, SIGTTOU seems to be behave differently when running
> Emacs -nw under LLDB, even when I tell LLDB to not stop or report it.
> Or so I think, I'm not an LLDB expert.
> 
> Here is what I tried
> 
> cd src
> lldb emacs
> b main
> run -Q -nw
> process handle -s false -n false SIGTTOU
> c
> 
> but then Emacs gets stuck.  Maybe it's a bug in LLDB.

Is this specific to -nw sessions?  If so, maybe LLFB has a command
similar to GDB's "set new-console 1"?  That's what I do to make sure
the console used by GDB doesn't get messed up by the terminal setup
used by Emacs to prepare the terminal for itself.  Like this:

  $ gdb ./emacs
  ...
  (gdb) set new-console 1
  (gdb) r -Q -nw

Then Emacs gets a new console for its TTY frame, while GDB retains its
original console.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Dmitrii Kuragin <kuragin@google.com>
> Date: Mon, 29 Aug 2022 09:03:15 -0700
> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>, 
> 	57434@debbugs.gnu.org
> 
> No, I see flickering during scrolling.

What exactly do you mean by "scrolling"? which keys or commands do you
invoke?

> And the problem is much worse when I have line number mode enabled.
> 
> The buffers do not change, and the frame at the same position, but line numbers change because they're in
> 'visual mode (relative numbers).

This still leaves unanswered one question:

>  > I compiled with `-O0 -g3`, then
>  > ```
>  > lldb
>  > (lldb) file nextstep/Emacs.app/Contents/MacOS/Emacs
>  > Current executable set to
>  '/Users/kuragin/Desktop/emacs/nextstep/Emacs.app/Contents/MacOS/Emacs'
>  > (x86_64).
>  > (lldb) breakpoint set -f scroll.c -l 270
>  > Breakpoint 1: where = Emacs`do_scrolling + 485 at scroll.c:271:11, address = 0x0000000100032da5
>  > ```
>  > 
>  > But, it doesn't stop there...
> 
>  Why scroll.c:271, when the code you patched begins on line 684?
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Why are you setting the breakpoint on that line?

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Dmitrii Kuragin <kuragin@google.com>
> Date: Mon, 29 Aug 2022 09:07:32 -0700
> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>, 
> 	57434@debbugs.gnu.org
> 
> See https://drive.google.com/file/d/1nMf_3MxRk2cTdgF3tFmzAZoxcy3vQghc/view?usp=sharing

Sorry, my browser doesn't support viewing this.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> but then Emacs gets stuck.  Maybe it's a bug in LLDB.
>
> Is this specific to -nw sessions?  If so, maybe LLFB has a command
> similar to GDB's "set new-console 1"?  That's what I do to make sure
> the console used by GDB doesn't get messed up by the terminal setup
> used by Emacs to prepare the terminal for itself.  Like this:
>
>   $ gdb ./emacs
>   ...
>   (gdb) set new-console 1
>   (gdb) r -Q -nw
>
> Then Emacs gets a new console for its TTY frame, while GDB retains its
> original console.

That was an excellent hint, thanks!

The following seems to work:

cd src
lldb emacs
b main
process launch --tty -- -nw -Q

LLDB then opens a new terminal window with a working Emacs in it,
and stops at main in the original window.  One can interrupt the running
Emacs by issuing

process interrupt

in the LLDB terminal window and so on.  The process launch --tty instead
of run is key here.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 3256 bytes --]

Thank you Eli and Gerd!

Now I can run the debugger!

> What exactly do you mean by "scrolling"? which keys or commands do you
> invoke?

I use C-n/C-p (next-line).

> Why are you setting the breakpoint on that line?

I am trying to debug the current version w/o my patch, but probably I just
messed something up. Let's ignore this I got better results.

I am using `breakpoint set -f scroll.c -l 697`.

Currently, when I have 2 panes and I have the right pane with content on it
`do_direct_scrolling` doesn't go into the first condition in the loop. So,
It doesn't stop in the debugger. But, when I open the scratch buffer which
has only few lines and I try to scroll in the left pane on the lines where
the right buffer doesn't have any content because there are only few lines
in the buffer it stops:
```
(lldb) p *p
(matrix_elt) $3 = {
  writecost = 35356
  insertcost = 34958
  deletecost = 35357
  insertcount = 97
  deletecount = 1
  writecount = 1
}
```

... just because the cost of insertion is lower than the write cost. Then I
set the breakpoint into different line set the right window a buffer with
content:
```
(lldb) breakpoint set -f scroll.c -l 688
```
So, I see
```
(lldb) p *p
(matrix_elt) $4 = {
  writecost = 54426
  insertcost = 54996
  deletecost = 54735
  insertcount = 1
  deletecount = 8
  writecount = 148
}
```

Insert and delete cost are greater than write cost.

I see the behavior as a correct one (at least by design), but when we do
insert instead of writing the terminal flickers. Do we need to adjust some
numbers or do we have to understand the reason why we flush the content?



On Mon, Aug 29, 2022 at 10:14 AM Gerd Möllmann <gerd.moellmann@gmail.com>
wrote:

> Eli Zaretskii <eliz@gnu.org> writes:
>
> >> but then Emacs gets stuck.  Maybe it's a bug in LLDB.
> >
> > Is this specific to -nw sessions?  If so, maybe LLFB has a command
> > similar to GDB's "set new-console 1"?  That's what I do to make sure
> > the console used by GDB doesn't get messed up by the terminal setup
> > used by Emacs to prepare the terminal for itself.  Like this:
> >
> >   $ gdb ./emacs
> >   ...
> >   (gdb) set new-console 1
> >   (gdb) r -Q -nw
> >
> > Then Emacs gets a new console for its TTY frame, while GDB retains its
> > original console.
>
> That was an excellent hint, thanks!
>
> The following seems to work:
>
> cd src
> lldb emacs
> b main
> process launch --tty -- -nw -Q
>
> LLDB then opens a new terminal window with a working Emacs in it,
> and stops at main in the original window.  One can interrupt the running
> Emacs by issuing
>
> process interrupt
>
> in the LLDB terminal window and so on.  The process launch --tty instead
> of run is key here.
>
>
>

-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 5877 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Dmitrii Kuragin <kuragin@google.com>
> Date: Mon, 29 Aug 2022 11:24:45 -0700
> Cc: Eli Zaretskii <eliz@gnu.org>, 57434@debbugs.gnu.org
> 
> I see the behavior as a correct one (at least by design), but when we do insert instead of writing the terminal
> flickers. Do we need to adjust some numbers or do we have to understand the reason why we flush the
> content?

What do you mean by "flush the content"?  If this is what causes the
flickering, then please explain where does the code perform the
"flush".

More generally, we need to understand why insertion cause flickering
whereas writing to the terminal does not.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1820 bytes --]

On Mon, Aug 29, 2022 at 11:57 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Dmitrii Kuragin <kuragin@google.com>
> > Date: Mon, 29 Aug 2022 11:24:45 -0700
> > Cc: Eli Zaretskii <eliz@gnu.org>, 57434@debbugs.gnu.org
> >
> > I see the behavior as a correct one (at least by design), but when we do
> insert instead of writing the terminal
> > flickers. Do we need to adjust some numbers or do we have to understand
> the reason why we flush the
> > content?
>
> What do you mean by "flush the content"?  If this is what causes the
> flickering, then please explain where does the code perform the
> "flush".
>
> More generally, we need to understand why insertion cause flickering
> whereas writing to the terminal does not.
>

I agree.

Let's ignore what I said about flushing. My assumption was that we draw the
terminal content in multiple steps in different places. For example, we
remove some lines, then do some logic, then we draw chars on top of it. So,
if we have a lag between the operations and the terminal refreshes the
screen we see only part of the content. But, as I said. Let's ignore that
and if you have any guidance on how I can debug it further, it would be
awesome.

Flickering is consistent for some specific area. If I scroll between 2
lines, back-and-forth Emacs flickers consistently.

What would be my next steps to give more debug information?




-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 4094 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Dmitrii Kuragin <kuragin@google.com>
> Date: Mon, 29 Aug 2022 12:04:38 -0700
> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>, 
> 	57434@debbugs.gnu.org
> 
>  More generally, we need to understand why insertion cause flickering
>  whereas writing to the terminal does not.
> 
> I agree.
> 
> Let's ignore what I said about flushing. My assumption was that we draw the terminal content in multiple steps
> in different places. For example, we remove some lines, then do some logic, then we draw chars on top of it.
> So, if we have a lag between the operations and the terminal refreshes the screen we see only part of the
> content. But, as I said. Let's ignore that and if you have any guidance on how I can debug it further, it would be
> awesome. 
> 
> Flickering is consistent for some specific area. If I scroll between 2 lines, back-and-forth Emacs flickers
> consistently.
> 
> What would be my next steps to give more debug information?

Can you try some other terminal emulator?  I'm interested to know
whether all of them flicker, or just some.

Another idea is to disable the insert/delete optimization entirely, by
making sure the line_ins_del_ok flag of the terminal is reset.  The
question, of course, is what to base this on -- could be macOS or just
some specific terminal type.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 2868 bytes --]

I tried different terminal emulators, Alacritty, iTerm2, Terminal.app
(stock). All of them show the same issue. So, it is not a terminal emulator.

Additional info:

When the screen flickers I alway see only 2 rectangles w/o content and they
always are the same for the same scroll position.

Looking into the debugger, it corresponds to the same number of insertions
and insert positions. The more heavy content on the screen the more
frequently the regions w/o get visible.

As an option, we can add the optional flag where we can entirely disable
the optimization and probably let people test it on their setups.

One more possible cause is 24 bit colors in my terminal:
https://github.com/syl20bnr/spacemacs/wiki/Terminal

But, even when I run emacs using 256 colors:
```
echo $TERM
xterm-256color
```

But, It still shows the same issue.

I am open to any possible solution. What would you like me to do? :)

On Mon, Aug 29, 2022 at 12:16 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Dmitrii Kuragin <kuragin@google.com>
> > Date: Mon, 29 Aug 2022 12:04:38 -0700
> > Cc: Gerd Möllmann <gerd.moellmann@gmail.com>,
> >       57434@debbugs.gnu.org
> >
> >  More generally, we need to understand why insertion cause flickering
> >  whereas writing to the terminal does not.
> >
> > I agree.
> >
> > Let's ignore what I said about flushing. My assumption was that we draw
> the terminal content in multiple steps
> > in different places. For example, we remove some lines, then do some
> logic, then we draw chars on top of it.
> > So, if we have a lag between the operations and the terminal refreshes
> the screen we see only part of the
> > content. But, as I said. Let's ignore that and if you have any guidance
> on how I can debug it further, it would be
> > awesome.
> >
> > Flickering is consistent for some specific area. If I scroll between 2
> lines, back-and-forth Emacs flickers
> > consistently.
> >
> > What would be my next steps to give more debug information?
>
> Can you try some other terminal emulator?  I'm interested to know
> whether all of them flicker, or just some.
>
> Another idea is to disable the insert/delete optimization entirely, by
> making sure the line_ins_del_ok flag of the terminal is reset.  The
> question, of course, is what to base this on -- could be macOS or just
> some specific terminal type.
>


-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 5225 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Dmitrii Kuragin <kuragin@google.com>
> Date: Mon, 29 Aug 2022 12:26:07 -0700
> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>, 
> 	57434@debbugs.gnu.org
> 
> I tried different terminal emulators, Alacritty, iTerm2, Terminal.app (stock). All of them show the same issue.
> So, it is not a terminal emulator.

If this happens with all terminal emulators on macOS, we should reset
the line_ins_del_ok flag for macOS.  Look in term.c, where it is
initialized by consulting various terminfo features supported by the
terminal.  If all the features it consults indeed work on macOS, then
simply say something like

  #ifdef DARWIN_OS
    tty->line_ins_del_ok = 0;
  #else
  ... the current code...
  #endif

and see if the problem goes away.

Gerd, do you see the same on your system?

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1983 bytes --]

So far:
```
:~/Desktop% tput al; echo $?
0
:~/Desktop% tput AL; echo $?
1%dL0
:~/Desktop% tput dl; echo $?
0
:~/Desktop% tput DL; echo $?
1%dM0
:~/Desktop% tput sf; echo $?

0
0~/Desktop% tput sr; echo $?
:~/Desktop% tput wi; echo $?
tput: unknown terminfo capability 'wi'
4
:~/Desktop% tput cs; echo $?
%p1%d;%p2%dr0
:~/Desktop% tput cS; echo $?
tput: unknown terminfo capability 'cS'
4
```

Seems like Mac Os doesn't support "wi", but the condition is still going to
be true.

```
tty->line_ins_del_ok = 0;
```

make the emacs really smooth w/o any flickering.

On Mon, Aug 29, 2022 at 12:37 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Dmitrii Kuragin <kuragin@google.com>
> > Date: Mon, 29 Aug 2022 12:26:07 -0700
> > Cc: Gerd Möllmann <gerd.moellmann@gmail.com>,
> >       57434@debbugs.gnu.org
> >
> > I tried different terminal emulators, Alacritty, iTerm2, Terminal.app
> (stock). All of them show the same issue.
> > So, it is not a terminal emulator.
>
> If this happens with all terminal emulators on macOS, we should reset
> the line_ins_del_ok flag for macOS.  Look in term.c, where it is
> initialized by consulting various terminfo features supported by the
> terminal.  If all the features it consults indeed work on macOS, then
> simply say something like
>
>   #ifdef DARWIN_OS
>     tty->line_ins_del_ok = 0;
>   #else
>   ... the current code...
>   #endif
>
> and see if the problem goes away.
>
> Gerd, do you see the same on your system?
>


-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 4106 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 3313 bytes --]

On my remote linux machine `line_is_del_ok` should be false where
```
  tty->line_ins_del_ok
    = (((tty->TS_ins_line || tty->TS_ins_multi_lines)
        && (tty->TS_del_line || tty->TS_del_multi_lines))
       || (tty->scroll_region_ok
           && tty->TS_fwd_scroll && tty->TS_rev_scroll));
```
because `tput al` and `tput AL` are both false and `tput sr` and `tput sf`
are false ("unknown terminfo capability").

So, disabling `line_ins_del_ok` on mac os would lead to the same
configuration I have on the linux. On the linux machine scroll never caused
any troubles.

So, would it be ok to disable `line_ins_del_ok` for Mac OS? I would send a
patch if it's OK.

On Mon, Aug 29, 2022 at 1:25 PM Dmitrii Kuragin <kuragin@google.com> wrote:

> So far:
> ```
> :~/Desktop% tput al; echo $?
> 0
> :~/Desktop% tput AL; echo $?
> 1%dL0
> :~/Desktop% tput dl; echo $?
> 0
> :~/Desktop% tput DL; echo $?
> 1%dM0
> :~/Desktop% tput sf; echo $?
>
> 0
> 0~/Desktop% tput sr; echo $?
> :~/Desktop% tput wi; echo $?
> tput: unknown terminfo capability 'wi'
> 4
> :~/Desktop% tput cs; echo $?
> %p1%d;%p2%dr0
> :~/Desktop% tput cS; echo $?
> tput: unknown terminfo capability 'cS'
> 4
> ```
>
> Seems like Mac Os doesn't support "wi", but the condition is still going
> to be true.
>
> ```
> tty->line_ins_del_ok = 0;
> ```
>
> make the emacs really smooth w/o any flickering.
>
> On Mon, Aug 29, 2022 at 12:37 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
>> > From: Dmitrii Kuragin <kuragin@google.com>
>> > Date: Mon, 29 Aug 2022 12:26:07 -0700
>> > Cc: Gerd Möllmann <gerd.moellmann@gmail.com>,
>> >       57434@debbugs.gnu.org
>> >
>> > I tried different terminal emulators, Alacritty, iTerm2, Terminal.app
>> (stock). All of them show the same issue.
>> > So, it is not a terminal emulator.
>>
>> If this happens with all terminal emulators on macOS, we should reset
>> the line_ins_del_ok flag for macOS.  Look in term.c, where it is
>> initialized by consulting various terminfo features supported by the
>> terminal.  If all the features it consults indeed work on macOS, then
>> simply say something like
>>
>>   #ifdef DARWIN_OS
>>     tty->line_ins_del_ok = 0;
>>   #else
>>   ... the current code...
>>   #endif
>>
>> and see if the problem goes away.
>>
>> Gerd, do you see the same on your system?
>>
>
>
> --
> *If you get an email from me outside of the 9-5 it is *not* because I'm
> always on or expect an immediate response from you; it is because of work
> flexibility
> <http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
> .  Evening and weekend emails are a sign I allocated some regular working
> hours for other things (such as family, gym, friends,...).  And I encourage
> you to feel free to do the same.
>
>

-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 6812 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1.1: Type: text/plain, Size: 3998 bytes --]

Here's the patch.

On Mon, Aug 29, 2022 at 1:44 PM Dmitrii Kuragin <kuragin@google.com> wrote:

> On my remote linux machine `line_is_del_ok` should be false where
> ```
>   tty->line_ins_del_ok
>     = (((tty->TS_ins_line || tty->TS_ins_multi_lines)
>         && (tty->TS_del_line || tty->TS_del_multi_lines))
>        || (tty->scroll_region_ok
>            && tty->TS_fwd_scroll && tty->TS_rev_scroll));
> ```
> because `tput al` and `tput AL` are both false and `tput sr` and `tput sf`
> are false ("unknown terminfo capability").
>
> So, disabling `line_ins_del_ok` on mac os would lead to the same
> configuration I have on the linux. On the linux machine scroll never caused
> any troubles.
>
> So, would it be ok to disable `line_ins_del_ok` for Mac OS? I would send a
> patch if it's OK.
>
> On Mon, Aug 29, 2022 at 1:25 PM Dmitrii Kuragin <kuragin@google.com>
> wrote:
>
>> So far:
>> ```
>> :~/Desktop% tput al; echo $?
>> 0
>> :~/Desktop% tput AL; echo $?
>> 1%dL0
>> :~/Desktop% tput dl; echo $?
>> 0
>> :~/Desktop% tput DL; echo $?
>> 1%dM0
>> :~/Desktop% tput sf; echo $?
>>
>> 0
>> 0~/Desktop% tput sr; echo $?
>> :~/Desktop% tput wi; echo $?
>> tput: unknown terminfo capability 'wi'
>> 4
>> :~/Desktop% tput cs; echo $?
>> %p1%d;%p2%dr0
>> :~/Desktop% tput cS; echo $?
>> tput: unknown terminfo capability 'cS'
>> 4
>> ```
>>
>> Seems like Mac Os doesn't support "wi", but the condition is still going
>> to be true.
>>
>> ```
>> tty->line_ins_del_ok = 0;
>> ```
>>
>> make the emacs really smooth w/o any flickering.
>>
>> On Mon, Aug 29, 2022 at 12:37 PM Eli Zaretskii <eliz@gnu.org> wrote:
>>
>>> > From: Dmitrii Kuragin <kuragin@google.com>
>>> > Date: Mon, 29 Aug 2022 12:26:07 -0700
>>> > Cc: Gerd Möllmann <gerd.moellmann@gmail.com>,
>>> >       57434@debbugs.gnu.org
>>> >
>>> > I tried different terminal emulators, Alacritty, iTerm2, Terminal.app
>>> (stock). All of them show the same issue.
>>> > So, it is not a terminal emulator.
>>>
>>> If this happens with all terminal emulators on macOS, we should reset
>>> the line_ins_del_ok flag for macOS.  Look in term.c, where it is
>>> initialized by consulting various terminfo features supported by the
>>> terminal.  If all the features it consults indeed work on macOS, then
>>> simply say something like
>>>
>>>   #ifdef DARWIN_OS
>>>     tty->line_ins_del_ok = 0;
>>>   #else
>>>   ... the current code...
>>>   #endif
>>>
>>> and see if the problem goes away.
>>>
>>> Gerd, do you see the same on your system?
>>>
>>
>>
>> --
>> *If you get an email from me outside of the 9-5 it is *not* because I'm
>> always on or expect an immediate response from you; it is because of work
>> flexibility
>> <http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
>> .  Evening and weekend emails are a sign I allocated some regular
>> working hours for other things (such as family, gym, friends,...).  And I
>> encourage you to feel free to do the same.
>>
>>
>
> --
> *If you get an email from me outside of the 9-5 it is *not* because I'm
> always on or expect an immediate response from you; it is because of work
> flexibility
> <http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
> .  Evening and weekend emails are a sign I allocated some regular working
> hours for other things (such as family, gym, friends,...).  And I encourage
> you to feel free to do the same.
>
>

-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #1.2: Type: text/html, Size: 8591 bytes --]

[-- Attachment #2: 0001-Fix-terminal-Emacs-flickering-during-scrolling-on-Ma.patch --]
[-- Type: application/octet-stream, Size: 1243 bytes --]

From 219491518c4f6346f09b33c28c1b9b9215142f19 Mon Sep 17 00:00:00 2001
From: Dmitrii Kuragin <kuragin@chromium.org>
Date: Mon, 29 Aug 2022 13:58:53 -0700
Subject: [PATCH] Fix terminal Emacs flickering during scrolling on Mac OS.

* src/term.c (init_tty): Disable `line_ins_del_ok` when `DARWIN_OS` is
  defined because the optimization causes terminal flickering.
---
 src/term.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/src/term.c b/src/term.c
index 2e43d89232..e06f2bd9e0 100644
--- a/src/term.c
+++ b/src/term.c
@@ -4390,11 +4390,17 @@ init_tty (const char *name, const char *terminal_type, bool must_succeed)
     = (tty->Wcm->cm_abs
        && (tty->TS_set_window || tty->TS_set_scroll_region || tty->TS_set_scroll_region_1));
 
+#ifdef DARWIN_OS
+  /* Multiline optimization cause terminal flickering on Mac OS.
+     See (Bug#57434) */
+  tty->line_ins_del_ok = 0;
+#else
   tty->line_ins_del_ok
     = (((tty->TS_ins_line || tty->TS_ins_multi_lines)
         && (tty->TS_del_line || tty->TS_del_multi_lines))
        || (tty->scroll_region_ok
            && tty->TS_fwd_scroll && tty->TS_rev_scroll));
+#endif
 
   tty->char_ins_del_ok
     = ((tty->TS_ins_char || tty->TS_insert_mode
-- 
2.37.2.672.g94769d06f0-goog

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

> Gerd, do you see the same on your system?

No, I can't reproduce this.

After watching Dmitrii's video I thought it might be related to a light
terminal background color (I normally use a black background), so I
tried that, but no flickering.  I don't know what else to try now to
reproduce this.

Maybe it helps if I describe exactly what I'm doing?

~/ > uname -a
Darwin Mini.fritz.box 21.6.0 Darwin Kernel Version 21.6.0: Wed Aug 10
14:28:35 PDT 2022; root:xnu-8020.141.5~2/RELEASE_ARM64_T8101 arm64
~/ > infocmp -V       
ncurses 5.7.20081102
~/ > echo $TERM $LINES $COLUMNS
xterm-256color 95 364
~/ > cd emacs/master/src
[master] gerd@Mini 2022-08-30 7:50 
~/emacs/master/src/ > emacs -nw -Q xdisp.c
[master] gerd@Mini 2022-08-30 7:51 

C-x 3
C-x o
C-x b *scratch* RET
C-x o

Make the xdisp.c window scroll with C-n/C-p, C-v/M-v.

No flickering here.

Dmitrii, how do you set Emacs' background?  Is that the terminal's
background, or does it perhaps come from your Emacs config?  Or IOW, do
you test this with emacs -Q?

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Dmitrii Kuragin <kuragin@google.com> writes:

> So far:
> ```
> :~/Desktop% tput al; echo $?
> 0
> :~/Desktop% tput AL; echo $?
> 1%dL0
> :~/Desktop% tput dl; echo $?
> 0
> :~/Desktop% tput DL; echo $?
> 1%dM0
> :~/Desktop% tput sf; echo $?
>
> 0
> 0~/Desktop% tput sr; echo $?
> :~/Desktop% tput wi; echo $?
> tput: unknown terminfo capability 'wi'
> 4
> :~/Desktop% tput cs; echo $?
> %p1%d;%p2%dr0
> :~/Desktop% tput cS; echo $?
> tput: unknown terminfo capability 'cS'
> 4
> ```

Same here.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  57434@debbugs.gnu.org
> Date: Tue, 30 Aug 2022 08:09:33 +0200
> 
> Dmitrii Kuragin <kuragin@google.com> writes:
> 
> > So far:
> > ```
> > :~/Desktop% tput al; echo $?
> > 0
> > :~/Desktop% tput AL; echo $?
> > 1%dL0
> > :~/Desktop% tput dl; echo $?
> > 0
> > :~/Desktop% tput DL; echo $?
> > 1%dM0
> > :~/Desktop% tput sf; echo $?
> >
> > 0
> > 0~/Desktop% tput sr; echo $?
> > :~/Desktop% tput wi; echo $?
> > tput: unknown terminfo capability 'wi'
> > 4
> > :~/Desktop% tput cs; echo $?
> > %p1%d;%p2%dr0
> > :~/Desktop% tput cS; echo $?
> > tput: unknown terminfo capability 'cS'
> > 4
> > ```
> 
> Same here.

Thanks.

But I'm quite confused by all of this, because you don't show all the
relevant capabilities, AFAICT.

We have in term.c:

  tty->scroll_region_ok
    = (tty->Wcm->cm_abs
       && (tty->TS_set_window || tty->TS_set_scroll_region || tty->TS_set_scroll_region_1));

  tty->line_ins_del_ok
    = (((tty->TS_ins_line || tty->TS_ins_multi_lines)
        && (tty->TS_del_line || tty->TS_del_multi_lines))
       || (tty->scroll_region_ok
           && tty->TS_fwd_scroll && tty->TS_rev_scroll));

Please try all of the relevant capabilities and tell me which ones are
supported and which aren't.  (Please also mention both the capability
string and its term.c flag name, so that I shouldn't need to jump
back-and-forth in the source looking up each one to understand what it
means.)

Then we have in dispnew.c:

  /* If we cannot insert/delete lines, it's no use trying it.  */
  if (!FRAME_LINE_INS_DEL_OK (f))
    inhibit_id_p = 1;
  [...]
  /* Try doing i/d line, if not yet inhibited.  */
  if (!inhibit_id_p && i < desired_matrix->nrows)
    force_p |= scrolling (f);

Which means that 'scrolling', and thus 'scrolling_1' (where the
problem happens) will not be called if the line_ins_del_ok flag is not
set.

Furthermore, we have in scrolling_1:

  if (FRAME_SCROLL_REGION_OK (frame))
    {
      calculate_direct_scrolling (frame, matrix, window_size,
				  unchanged_at_bottom,
				  draw_cost, old_draw_cost,
				  old_hash, new_hash, free_at_end);
      do_direct_scrolling (frame, frame->current_matrix,
			   matrix, window_size, unchanged_at_top);
    }
  else
    {
      calculate_scrolling (frame, matrix, window_size, unchanged_at_bottom,
			   draw_cost, old_hash, new_hash,
			   free_at_end);
      do_scrolling (frame,
                    frame->current_matrix, matrix, window_size,
		    unchanged_at_top);
    }

which means do_direct_scrolling (which causes the problem) will not be
called if the terminal's scroll_region_ok flag is not set.

So given all of this, can you tell whether Emacs does TRT here?  That
is:

  . are all the capabilities that are supposed to be available for
    these two flags are indeed available?
  . do we need to check any additional capabilities, which are in fact
    used in the problematic scenario, but not tested as part of
    setting these two flags?

Assuming that Emacs does TRT, i.e. sets the flags correctly and uses
only the capabilities that are conditions for the flags set, then the
next important question is: why doesn't Gerd see the flickering on a
very similar system and the same terminal emulator?  Is it possible
that some other local software configuration on Dmitrii's machine
causes this, directly or indirectly?  E.g., Dmitrii, do you have some
display-related software/driver that has some "optimization" features
turned on?  If so, can you turn them off and try again?

Thanks.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Dmitrii Kuragin <kuragin@google.com>
> Date: Mon, 29 Aug 2022 13:25:06 -0700
> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>, 
> 	57434@debbugs.gnu.org
> 
> :~/Desktop% tput al; echo $?
> 0
> :~/Desktop% tput AL; echo $?
> 1%dL0
> :~/Desktop% tput dl; echo $?
> 0
> :~/Desktop% tput DL; echo $?
> 1%dM0
> :~/Desktop% tput sf; echo $?
> 
> 0
> 0~/Desktop% tput sr; echo $?
> :~/Desktop% tput wi; echo $?
> tput: unknown terminfo capability 'wi'
> 4
> :~/Desktop% tput cs; echo $?
> %p1%d;%p2%dr0
> :~/Desktop% tput cS; echo $?
> tput: unknown terminfo capability 'cS'
> 4
> ```
> 
> Seems like Mac Os doesn't support "wi", but the condition is still going to be true.

Emacs is supposed to use the alternative capabilities, as follows from
the condition for line_ins_del_ok flag to be set.  Does it indeed use
the other alternatives?

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

> But I'm quite confused by all of this, because you don't show all the
> relevant capabilities, AFAICT.

We could also compare terminfo capabilities by comparing the output of
'infocmp -1'.  This is what is prints on my system:

~/emacs/master/ > infocmp -1
#	Reconstructed via infocmp from file: /usr/share/terminfo/78/xterm-256color
xterm-256color|xterm with 256 colors,
	am,
	bce,
	ccc,
	km,
	mc5i,
	mir,
	msgr,
	npc,
	xenl,
	colors#256,
	cols#80,
	it#8,
	lines#24,
	pairs#32767,
	acsc=``aaffggiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~,
	bel=^G,
	blink=\E[5m,
	bold=\E[1m,
	cbt=\E[Z,
	civis=\E[?25l,
	clear=\E[H\E[2J,
	cnorm=\E[?12l\E[?25h,
	cr=^M,
	csr=\E[%i%p1%d;%p2%dr,
	cub=\E[%p1%dD,
	cub1=^H,
	cud=\E[%p1%dB,
	cud1=^J,
	cuf=\E[%p1%dC,
	cuf1=\E[C,
	cup=\E[%i%p1%d;%p2%dH,
	cuu=\E[%p1%dA,
	cuu1=\E[A,
	cvvis=\E[?12;25h,
	dch=\E[%p1%dP,
	dch1=\E[P,
	dl=\E[%p1%dM,
	dl1=\E[M,
	ech=\E[%p1%dX,
	ed=\E[J,
	el=\E[K,
	el1=\E[1K,
	flash=\E[?5h$<100/>\E[?5l,
	home=\E[H,
	hpa=\E[%i%p1%dG,
	ht=^I,
	hts=\EH,
	ich=\E[%p1%d@,
	il=\E[%p1%dL,
	il1=\E[L,
	ind=^J,
	indn=\E[%p1%dS,
	initc=\E]4;%p1%d;rgb\:%p2%{255}%*%{1000}%/%2.2X/%p3%{255}%*%{1000}%/%2.2X/%p4%{255}%*%{1000}%/%2.2X\E\\,
	invis=\E[8m,
	is2=\E[!p\E[?3;4l\E[4l\E>,
	kDC=\E[3;2~,
	kEND=\E[1;2F,
	kHOM=\E[1;2H,
	kIC=\E[2;2~,
	kLFT=\E[1;2D,
	kNXT=\E[6;2~,
	kPRV=\E[5;2~,
	kRIT=\E[1;2C,
	kb2=\EOE,
	kbs=^H,
	kcbt=\E[Z,
	kcub1=\EOD,
	kcud1=\EOB,
	kcuf1=\EOC,
	kcuu1=\EOA,
	kdch1=\E[3~,
	kend=\EOF,
	kent=\EOM,
	kf1=\EOP,
	kf10=\E[21~,
	kf11=\E[23~,
	kf12=\E[24~,
	kf13=\E[1;2P,
	kf14=\E[1;2Q,
	kf15=\E[1;2R,
	kf16=\E[1;2S,
	kf17=\E[15;2~,
	kf18=\E[17;2~,
	kf19=\E[18;2~,
	kf2=\EOQ,
	kf20=\E[19;2~,
	kf21=\E[20;2~,
	kf22=\E[21;2~,
	kf23=\E[23;2~,
	kf24=\E[24;2~,
	kf25=\E[1;5P,
	kf26=\E[1;5Q,
	kf27=\E[1;5R,
	kf28=\E[1;5S,
	kf29=\E[15;5~,
	kf3=\EOR,
	kf30=\E[17;5~,
	kf31=\E[18;5~,
	kf32=\E[19;5~,
	kf33=\E[20;5~,
	kf34=\E[21;5~,
	kf35=\E[23;5~,
	kf36=\E[24;5~,
	kf37=\E[1;6P,
	kf38=\E[1;6Q,
	kf39=\E[1;6R,
	kf4=\EOS,
	kf40=\E[1;6S,
	kf41=\E[15;6~,
	kf42=\E[17;6~,
	kf43=\E[18;6~,
	kf44=\E[19;6~,
	kf45=\E[20;6~,
	kf46=\E[21;6~,
	kf47=\E[23;6~,
	kf48=\E[24;6~,
	kf49=\E[1;3P,
	kf5=\E[15~,
	kf50=\E[1;3Q,
	kf51=\E[1;3R,
	kf52=\E[1;3S,
	kf53=\E[15;3~,
	kf54=\E[17;3~,
	kf55=\E[18;3~,
	kf56=\E[19;3~,
	kf57=\E[20;3~,
	kf58=\E[21;3~,
	kf59=\E[23;3~,
	kf6=\E[17~,
	kf60=\E[24;3~,
	kf61=\E[1;4P,
	kf62=\E[1;4Q,
	kf63=\E[1;4R,
	kf7=\E[18~,
	kf8=\E[19~,
	kf9=\E[20~,
	khome=\EOH,
	kich1=\E[2~,
	kind=\E[1;2B,
	kmous=\E[M,
	knp=\E[6~,
	kpp=\E[5~,
	kri=\E[1;2A,
	mc0=\E[i,
	mc4=\E[4i,
	mc5=\E[5i,
	meml=\El,
	memu=\Em,
	op=\E[39;49m,
	rc=\E8,
	rev=\E[7m,
	ri=\EM,
	rin=\E[%p1%dT,
	rmacs=\E(B,
	rmam=\E[?7l,
	rmcup=\E[?1049l,
	rmir=\E[4l,
	rmkx=\E[?1l\E>,
	rmm=\E[?1034l,
	rmso=\E[27m,
	rmul=\E[24m,
	rs1=\Ec,
	rs2=\E[!p\E[?3;4l\E[4l\E>,
	sc=\E7,
	setab=\E[%?%p1%{8}%<%t4%p1%d%e%p1%{16}%<%t10%p1%{8}%-%d%e48;5;%p1%d%;m,
	setaf=\E[%?%p1%{8}%<%t3%p1%d%e%p1%{16}%<%t9%p1%{8}%-%d%e38;5;%p1%d%;m,
	sgr=%?%p9%t\E(0%e\E(B%;\E[0%?%p6%t;1%;%?%p2%t;4%;%?%p1%p3%|%t;7%;%?%p4%t;5%;%?%p7%t;8%;m,
	sgr0=\E(B\E[m,
	smacs=\E(0,
	smam=\E[?7h,
	smcup=\E[?1049h,
	smir=\E[4h,
	smkx=\E[?1h\E=,
	smm=\E[?1034h,
	smso=\E[7m,
	smul=\E[4m,
	tbc=\E[3g,
	u6=\E[%i%d;%dR,
	u7=\E[6n,
	u8=\E[?1;2c,
	u9=\E[c,
	vpa=\E[%i%p1%dd,

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Dmitrii Kuragin <kuragin@google.com>
> Date: Mon, 29 Aug 2022 14:08:04 -0700
> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>, 
> 	57434@debbugs.gnu.org
> 
> Here's the patch.

Sorry, this is premature.  See the questions I asked in my other
message.  We need to understand what exactly is going on here and why.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: Dmitrii Kuragin <kuragin@google.com>,  57434@debbugs.gnu.org
> Date: Tue, 30 Aug 2022 08:04:12 +0200
> 
> Maybe it helps if I describe exactly what I'm doing?
> 
> ~/ > uname -a
> Darwin Mini.fritz.box 21.6.0 Darwin Kernel Version 21.6.0: Wed Aug 10
> 14:28:35 PDT 2022; root:xnu-8020.141.5~2/RELEASE_ARM64_T8101 arm64
> ~/ > infocmp -V       
> ncurses 5.7.20081102
> ~/ > echo $TERM $LINES $COLUMNS
> xterm-256color 95 364
> ~/ > cd emacs/master/src
> [master] gerd@Mini 2022-08-30 7:50 
> ~/emacs/master/src/ > emacs -nw -Q xdisp.c
> [master] gerd@Mini 2022-08-30 7:51 
> 
> C-x 3
> C-x o
> C-x b *scratch* RET
> C-x o
> 
> Make the xdisp.c window scroll with C-n/C-p, C-v/M-v.
> 
> No flickering here.

Try after enabling display-line-numbers-mode, I think Dmitrii said the
flickering is much more prominent in that case.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

> Try after enabling display-line-numbers-mode, I think Dmitrii said the
> flickering is much more prominent in that case.

Ah, I forgot that.  Now did:

M-x display-line-number-mode RET
>> C-x 3
>> C-x o
>> C-x b *scratch* RET
>> C-x o
>> 
>> Make the xdisp.c window scroll with C-n/C-p, C-v/M-v.

No flickering, neither with dark or light terminal bg.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: kuragin@google.com,  57434@debbugs.gnu.org
> Date: Tue, 30 Aug 2022 13:53:47 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Try after enabling display-line-numbers-mode, I think Dmitrii said the
> > flickering is much more prominent in that case.
> 
> Ah, I forgot that.  Now did:
> 
> M-x display-line-number-mode RET
> >> C-x 3
> >> C-x o
> >> C-x b *scratch* RET
> >> C-x o
> >> 
> >> Make the xdisp.c window scroll with C-n/C-p, C-v/M-v.
> 
> No flickering, neither with dark or light terminal bg.

And if you do

  M-: (setq display-line-numbers 'relative) RET

does anything change then?

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

> And if you do
>
>   M-: (setq display-line-numbers 'relative) RET
>
> does anything change then?

Nope.

And it also doesn't flicker with a black terminal bg and starting Emacs
with -bg <something>.  And it doesn't flicker in the same terminal when
I start Emacs normally, but set-face-background to something.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: kuragin@google.com,  57434@debbugs.gnu.org
> Date: Tue, 30 Aug 2022 14:15:30 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > And if you do
> >
> >   M-: (setq display-line-numbers 'relative) RET
> >
> > does anything change then?
> 
> Nope.
> 
> And it also doesn't flicker with a black terminal bg and starting Emacs
> with -bg <something>.  And it doesn't flicker in the same terminal when
> I start Emacs normally, but set-face-background to something.

OK, thanks.  So now the question of what's different on Dmitrii's
machine is the most relevant one.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1911 bytes --]

In my config, I just use different themes. But, when I run w/ "-Q" I have
the same color, but text flickers, not the background. It is less visible,
but it erases some lines of text and brings them back.

On Mon, Aug 29, 2022 at 11:04 PM Gerd Möllmann <gerd.moellmann@gmail.com>
wrote:

> Eli Zaretskii <eliz@gnu.org> writes:
>
> > Gerd, do you see the same on your system?
>
> No, I can't reproduce this.
>
> After watching Dmitrii's video I thought it might be related to a light
> terminal background color (I normally use a black background), so I
> tried that, but no flickering.  I don't know what else to try now to
> reproduce this.
>
> Maybe it helps if I describe exactly what I'm doing?
>
> ~/ > uname -a
> Darwin Mini.fritz.box 21.6.0 Darwin Kernel Version 21.6.0: Wed Aug 10
> 14:28:35 PDT 2022; root:xnu-8020.141.5~2/RELEASE_ARM64_T8101 arm64
> ~/ > infocmp -V
> ncurses 5.7.20081102
> ~/ > echo $TERM $LINES $COLUMNS
> xterm-256color 95 364
> ~/ > cd emacs/master/src
> [master] gerd@Mini 2022-08-30 7:50
> ~/emacs/master/src/ > emacs -nw -Q xdisp.c
> [master] gerd@Mini 2022-08-30 7:51
>
> C-x 3
> C-x o
> C-x b *scratch* RET
> C-x o
>
> Make the xdisp.c window scroll with C-n/C-p, C-v/M-v.
>
> No flickering here.
>
> Dmitrii, how do you set Emacs' background?  Is that the terminal's
> background, or does it perhaps come from your Emacs config?  Or IOW, do
> you test this with emacs -Q?
>
>

-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 3353 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1902 bytes --]

Could you please try these settings?
```
(setq display-line-numbers-type 'visual)
(global-display-line-numbers-mode)

(global-hl-line-mode)

(global-display-fill-column-indicator-mode)
```

On Mon, Aug 29, 2022 at 11:04 PM Gerd Möllmann <gerd.moellmann@gmail.com>
wrote:

> Eli Zaretskii <eliz@gnu.org> writes:
>
> > Gerd, do you see the same on your system?
>
> No, I can't reproduce this.
>
> After watching Dmitrii's video I thought it might be related to a light
> terminal background color (I normally use a black background), so I
> tried that, but no flickering.  I don't know what else to try now to
> reproduce this.
>
> Maybe it helps if I describe exactly what I'm doing?
>
> ~/ > uname -a
> Darwin Mini.fritz.box 21.6.0 Darwin Kernel Version 21.6.0: Wed Aug 10
> 14:28:35 PDT 2022; root:xnu-8020.141.5~2/RELEASE_ARM64_T8101 arm64
> ~/ > infocmp -V
> ncurses 5.7.20081102
> ~/ > echo $TERM $LINES $COLUMNS
> xterm-256color 95 364
> ~/ > cd emacs/master/src
> [master] gerd@Mini 2022-08-30 7:50
> ~/emacs/master/src/ > emacs -nw -Q xdisp.c
> [master] gerd@Mini 2022-08-30 7:51
>
> C-x 3
> C-x o
> C-x b *scratch* RET
> C-x o
>
> Make the xdisp.c window scroll with C-n/C-p, C-v/M-v.
>
> No flickering here.
>
> Dmitrii, how do you set Emacs' background?  Is that the terminal's
> background, or does it perhaps come from your Emacs config?  Or IOW, do
> you test this with emacs -Q?
>
>

-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 3357 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 5712 bytes --]

I have exactly the same output for `infocmp -1`.

On Tue, Aug 30, 2022 at 4:23 AM Gerd Möllmann <gerd.moellmann@gmail.com>
wrote:

> Eli Zaretskii <eliz@gnu.org> writes:
>
> > But I'm quite confused by all of this, because you don't show all the
> > relevant capabilities, AFAICT.
>
> We could also compare terminfo capabilities by comparing the output of
> 'infocmp -1'.  This is what is prints on my system:
>
> ~/emacs/master/ > infocmp -1
> #       Reconstructed via infocmp from file:
> /usr/share/terminfo/78/xterm-256color
> xterm-256color|xterm with 256 colors,
>         am,
>         bce,
>         ccc,
>         km,
>         mc5i,
>         mir,
>         msgr,
>         npc,
>         xenl,
>         colors#256,
>         cols#80,
>         it#8,
>         lines#24,
>         pairs#32767,
>         acsc=``aaffggiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~,
>         bel=^G,
>         blink=\E[5m,
>         bold=\E[1m,
>         cbt=\E[Z,
>         civis=\E[?25l,
>         clear=\E[H\E[2J,
>         cnorm=\E[?12l\E[?25h,
>         cr=^M,
>         csr=\E[%i%p1%d;%p2%dr,
>         cub=\E[%p1%dD,
>         cub1=^H,
>         cud=\E[%p1%dB,
>         cud1=^J,
>         cuf=\E[%p1%dC,
>         cuf1=\E[C,
>         cup=\E[%i%p1%d;%p2%dH,
>         cuu=\E[%p1%dA,
>         cuu1=\E[A,
>         cvvis=\E[?12;25h,
>         dch=\E[%p1%dP,
>         dch1=\E[P,
>         dl=\E[%p1%dM,
>         dl1=\E[M,
>         ech=\E[%p1%dX,
>         ed=\E[J,
>         el=\E[K,
>         el1=\E[1K,
>         flash=\E[?5h$<100/>\E[?5l,
>         home=\E[H,
>         hpa=\E[%i%p1%dG,
>         ht=^I,
>         hts=\EH,
>         ich=\E[%p1%d@,
>         il=\E[%p1%dL,
>         il1=\E[L,
>         ind=^J,
>         indn=\E[%p1%dS,
>
> initc=\E]4;%p1%d;rgb\:%p2%{255}%*%{1000}%/%2.2X/%p3%{255}%*%{1000}%/%2.2X/%p4%{255}%*%{1000}%/%2.2X\E\\,
>         invis=\E[8m,
>         is2=\E[!p\E[?3;4l\E[4l\E>,
>         kDC=\E[3;2~,
>         kEND=\E[1;2F,
>         kHOM=\E[1;2H,
>         kIC=\E[2;2~,
>         kLFT=\E[1;2D,
>         kNXT=\E[6;2~,
>         kPRV=\E[5;2~,
>         kRIT=\E[1;2C,
>         kb2=\EOE,
>         kbs=^H,
>         kcbt=\E[Z,
>         kcub1=\EOD,
>         kcud1=\EOB,
>         kcuf1=\EOC,
>         kcuu1=\EOA,
>         kdch1=\E[3~,
>         kend=\EOF,
>         kent=\EOM,
>         kf1=\EOP,
>         kf10=\E[21~,
>         kf11=\E[23~,
>         kf12=\E[24~,
>         kf13=\E[1;2P,
>         kf14=\E[1;2Q,
>         kf15=\E[1;2R,
>         kf16=\E[1;2S,
>         kf17=\E[15;2~,
>         kf18=\E[17;2~,
>         kf19=\E[18;2~,
>         kf2=\EOQ,
>         kf20=\E[19;2~,
>         kf21=\E[20;2~,
>         kf22=\E[21;2~,
>         kf23=\E[23;2~,
>         kf24=\E[24;2~,
>         kf25=\E[1;5P,
>         kf26=\E[1;5Q,
>         kf27=\E[1;5R,
>         kf28=\E[1;5S,
>         kf29=\E[15;5~,
>         kf3=\EOR,
>         kf30=\E[17;5~,
>         kf31=\E[18;5~,
>         kf32=\E[19;5~,
>         kf33=\E[20;5~,
>         kf34=\E[21;5~,
>         kf35=\E[23;5~,
>         kf36=\E[24;5~,
>         kf37=\E[1;6P,
>         kf38=\E[1;6Q,
>         kf39=\E[1;6R,
>         kf4=\EOS,
>         kf40=\E[1;6S,
>         kf41=\E[15;6~,
>         kf42=\E[17;6~,
>         kf43=\E[18;6~,
>         kf44=\E[19;6~,
>         kf45=\E[20;6~,
>         kf46=\E[21;6~,
>         kf47=\E[23;6~,
>         kf48=\E[24;6~,
>         kf49=\E[1;3P,
>         kf5=\E[15~,
>         kf50=\E[1;3Q,
>         kf51=\E[1;3R,
>         kf52=\E[1;3S,
>         kf53=\E[15;3~,
>         kf54=\E[17;3~,
>         kf55=\E[18;3~,
>         kf56=\E[19;3~,
>         kf57=\E[20;3~,
>         kf58=\E[21;3~,
>         kf59=\E[23;3~,
>         kf6=\E[17~,
>         kf60=\E[24;3~,
>         kf61=\E[1;4P,
>         kf62=\E[1;4Q,
>         kf63=\E[1;4R,
>         kf7=\E[18~,
>         kf8=\E[19~,
>         kf9=\E[20~,
>         khome=\EOH,
>         kich1=\E[2~,
>         kind=\E[1;2B,
>         kmous=\E[M,
>         knp=\E[6~,
>         kpp=\E[5~,
>         kri=\E[1;2A,
>         mc0=\E[i,
>         mc4=\E[4i,
>         mc5=\E[5i,
>         meml=\El,
>         memu=\Em,
>         op=\E[39;49m,
>         rc=\E8,
>         rev=\E[7m,
>         ri=\EM,
>         rin=\E[%p1%dT,
>         rmacs=\E(B,
>         rmam=\E[?7l,
>         rmcup=\E[?1049l,
>         rmir=\E[4l,
>         rmkx=\E[?1l\E>,
>         rmm=\E[?1034l,
>         rmso=\E[27m,
>         rmul=\E[24m,
>         rs1=\Ec,
>         rs2=\E[!p\E[?3;4l\E[4l\E>,
>         sc=\E7,
>
> setab=\E[%?%p1%{8}%<%t4%p1%d%e%p1%{16}%<%t10%p1%{8}%-%d%e48;5;%p1%d%;m,
>
> setaf=\E[%?%p1%{8}%<%t3%p1%d%e%p1%{16}%<%t9%p1%{8}%-%d%e38;5;%p1%d%;m,
>
> sgr=%?%p9%t\E(0%e\E(B%;\E[0%?%p6%t;1%;%?%p2%t;4%;%?%p1%p3%|%t;7%;%?%p4%t;5%;%?%p7%t;8%;m,
>         sgr0=\E(B\E[m,
>         smacs=\E(0,
>         smam=\E[?7h,
>         smcup=\E[?1049h,
>         smir=\E[4h,
>         smkx=\E[?1h\E=,
>         smm=\E[?1034h,
>         smso=\E[7m,
>         smul=\E[4m,
>         tbc=\E[3g,
>         u6=\E[%i%d;%dR,
>         u7=\E[6n,
>         u8=\E[?1;2c,
>         u9=\E[c,
>         vpa=\E[%i%p1%dd,
>


-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 8250 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 5720 bytes --]

`tty->TS_ins_line` (al) is supprted.
`tty->TS_ins_multi_lines` (AL) is supprted.
`tty->TS_del_line` (dl) is supprted.
`tty->TS_del_multi_lines` (DL) is supprted.

to verify that I used `tput <cap_name>`.

I think, that is sufficient for `tty->line_ins_del_ok` to be true, but fo
completeness:

`tty->TS_fwd_scroll` (sf) is supprted.
`tty->TS_rev_scroll` (sr) is supprted.


`tty->TS_set_window` (wi) is NOT supprted.
`tty->TS_set_scroll_region` (cs) is supprted.
`tty->TS_set_scroll_region_1` (cS) is NOT supprted.

BUt I do not know how to verify `tty->Wcm->cm_abs`.


```
tty->scroll_region_ok
    = (tty->Wcm->cm_abs
       && (tty->TS_set_window || tty->TS_set_scroll_region ||
tty->TS_set_scroll_region_1));
```


```
tty->line_ins_del_ok
    = (((tty->TS_ins_line || tty->TS_ins_multi_lines)
        && (tty->TS_del_line || tty->TS_del_multi_lines))
       || (tty->scroll_region_ok
           && tty->TS_fwd_scroll && tty->TS_rev_scroll));
```

BTW, here's a video with what I have with "-Q", it still flickers:
https://drive.google.com/file/d/1Yq2QFWHR6CHkoM4buEokV6p3a1uOI8ao/view?usp=sharing

On Tue, Aug 30, 2022 at 4:09 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Gerd Möllmann <gerd.moellmann@gmail.com>
> > Cc: Eli Zaretskii <eliz@gnu.org>,  57434@debbugs.gnu.org
> > Date: Tue, 30 Aug 2022 08:09:33 +0200
> >
> > Dmitrii Kuragin <kuragin@google.com> writes:
> >
> > > So far:
> > > ```
> > > :~/Desktop% tput al; echo $?
> > > 0
> > > :~/Desktop% tput AL; echo $?
> > > 1%dL0
> > > :~/Desktop% tput dl; echo $?
> > > 0
> > > :~/Desktop% tput DL; echo $?
> > > 1%dM0
> > > :~/Desktop% tput sf; echo $?
> > >
> > > 0
> > > 0~/Desktop% tput sr; echo $?
> > > :~/Desktop% tput wi; echo $?
> > > tput: unknown terminfo capability 'wi'
> > > 4
> > > :~/Desktop% tput cs; echo $?
> > > %p1%d;%p2%dr0
> > > :~/Desktop% tput cS; echo $?
> > > tput: unknown terminfo capability 'cS'
> > > 4
> > > ```
> >
> > Same here.
>
> Thanks.
>
> But I'm quite confused by all of this, because you don't show all the
> relevant capabilities, AFAICT.
>
> We have in term.c:
>
>   tty->scroll_region_ok
>     = (tty->Wcm->cm_abs
>        && (tty->TS_set_window || tty->TS_set_scroll_region ||
> tty->TS_set_scroll_region_1));
>
>   tty->line_ins_del_ok
>     = (((tty->TS_ins_line || tty->TS_ins_multi_lines)
>         && (tty->TS_del_line || tty->TS_del_multi_lines))
>        || (tty->scroll_region_ok
>            && tty->TS_fwd_scroll && tty->TS_rev_scroll));
>
> Please try all of the relevant capabilities and tell me which ones are
> supported and which aren't.  (Please also mention both the capability
> string and its term.c flag name, so that I shouldn't need to jump
> back-and-forth in the source looking up each one to understand what it
> means.)
>
> Then we have in dispnew.c:
>
>   /* If we cannot insert/delete lines, it's no use trying it.  */
>   if (!FRAME_LINE_INS_DEL_OK (f))
>     inhibit_id_p = 1;
>   [...]
>   /* Try doing i/d line, if not yet inhibited.  */
>   if (!inhibit_id_p && i < desired_matrix->nrows)
>     force_p |= scrolling (f);
>
> Which means that 'scrolling', and thus 'scrolling_1' (where the
> problem happens) will not be called if the line_ins_del_ok flag is not
> set.
>
> Furthermore, we have in scrolling_1:
>
>   if (FRAME_SCROLL_REGION_OK (frame))
>     {
>       calculate_direct_scrolling (frame, matrix, window_size,
>                                   unchanged_at_bottom,
>                                   draw_cost, old_draw_cost,
>                                   old_hash, new_hash, free_at_end);
>       do_direct_scrolling (frame, frame->current_matrix,
>                            matrix, window_size, unchanged_at_top);
>     }
>   else
>     {
>       calculate_scrolling (frame, matrix, window_size, unchanged_at_bottom,
>                            draw_cost, old_hash, new_hash,
>                            free_at_end);
>       do_scrolling (frame,
>                     frame->current_matrix, matrix, window_size,
>                     unchanged_at_top);
>     }
>
> which means do_direct_scrolling (which causes the problem) will not be
> called if the terminal's scroll_region_ok flag is not set.
>
> So given all of this, can you tell whether Emacs does TRT here?  That
> is:
>
>   . are all the capabilities that are supposed to be available for
>     these two flags are indeed available?
>   . do we need to check any additional capabilities, which are in fact
>     used in the problematic scenario, but not tested as part of
>     setting these two flags?
>
> Assuming that Emacs does TRT, i.e. sets the flags correctly and uses
> only the capabilities that are conditions for the flags set, then the
> next important question is: why doesn't Gerd see the flickering on a
> very similar system and the same terminal emulator?  Is it possible
> that some other local software configuration on Dmitrii's machine
> causes this, directly or indirectly?  E.g., Dmitrii, do you have some
> display-related software/driver that has some "optimization" features
> turned on?  If so, can you turn them off and try again?
>
> Thanks.
>


-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 8414 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 6664 bytes --]

On Tue, Aug 30, 2022 at 9:19 AM Dmitrii Kuragin <kuragin@google.com> wrote:

> `tty->TS_ins_line` (al) is supprted.
> `tty->TS_ins_multi_lines` (AL) is supprted.
> `tty->TS_del_line` (dl) is supprted.
> `tty->TS_del_multi_lines` (DL) is supprted.
>
> to verify that I used `tput <cap_name>`.
>
> I think, that is sufficient for `tty->line_ins_del_ok` to be true, but fo
> completeness:
>
> `tty->TS_fwd_scroll` (sf) is supprted.
> `tty->TS_rev_scroll` (sr) is supprted.
>
>
> `tty->TS_set_window` (wi) is NOT supprted.
> `tty->TS_set_scroll_region` (cs) is supprted.
> `tty->TS_set_scroll_region_1` (cS) is NOT supprted.
>
> BUt I do not know how to verify `tty->Wcm->cm_abs`.
>
>
> ```
> tty->scroll_region_ok
>     = (tty->Wcm->cm_abs
>        && (tty->TS_set_window || tty->TS_set_scroll_region ||
> tty->TS_set_scroll_region_1));
> ```
>
>
> ```
> tty->line_ins_del_ok
>     = (((tty->TS_ins_line || tty->TS_ins_multi_lines)
>         && (tty->TS_del_line || tty->TS_del_multi_lines))
>        || (tty->scroll_region_ok
>            && tty->TS_fwd_scroll && tty->TS_rev_scroll));
> ```
>
> BTW, here's a video with what I have with "-Q", it still flickers:
> https://drive.google.com/file/d/1Yq2QFWHR6CHkoM4buEokV6p3a1uOI8ao/view?usp=sharing
>
> On Tue, Aug 30, 2022 at 4:09 AM Eli Zaretskii <eliz@gnu.org> wrote:
>
>> > From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> > Cc: Eli Zaretskii <eliz@gnu.org>,  57434@debbugs.gnu.org
>> > Date: Tue, 30 Aug 2022 08:09:33 +0200
>> >
>> > Dmitrii Kuragin <kuragin@google.com> writes:
>> >
>> > > So far:
>> > > ```
>> > > :~/Desktop% tput al; echo $?
>> > > 0
>> > > :~/Desktop% tput AL; echo $?
>> > > 1%dL0
>> > > :~/Desktop% tput dl; echo $?
>> > > 0
>> > > :~/Desktop% tput DL; echo $?
>> > > 1%dM0
>> > > :~/Desktop% tput sf; echo $?
>> > >
>> > > 0
>> > > 0~/Desktop% tput sr; echo $?
>> > > :~/Desktop% tput wi; echo $?
>> > > tput: unknown terminfo capability 'wi'
>> > > 4
>> > > :~/Desktop% tput cs; echo $?
>> > > %p1%d;%p2%dr0
>> > > :~/Desktop% tput cS; echo $?
>> > > tput: unknown terminfo capability 'cS'
>> > > 4
>> > > ```
>> >
>> > Same here.
>>
>> Thanks.
>>
>> But I'm quite confused by all of this, because you don't show all the
>> relevant capabilities, AFAICT.
>>
>> We have in term.c:
>>
>>   tty->scroll_region_ok
>>     = (tty->Wcm->cm_abs
>>        && (tty->TS_set_window || tty->TS_set_scroll_region ||
>> tty->TS_set_scroll_region_1));
>>
>>   tty->line_ins_del_ok
>>     = (((tty->TS_ins_line || tty->TS_ins_multi_lines)
>>         && (tty->TS_del_line || tty->TS_del_multi_lines))
>>        || (tty->scroll_region_ok
>>            && tty->TS_fwd_scroll && tty->TS_rev_scroll));
>>
>> Please try all of the relevant capabilities and tell me which ones are
>> supported and which aren't.  (Please also mention both the capability
>> string and its term.c flag name, so that I shouldn't need to jump
>> back-and-forth in the source looking up each one to understand what it
>> means.)
>>
>> Then we have in dispnew.c:
>>
>>   /* If we cannot insert/delete lines, it's no use trying it.  */
>>   if (!FRAME_LINE_INS_DEL_OK (f))
>>     inhibit_id_p = 1;
>>   [...]
>>   /* Try doing i/d line, if not yet inhibited.  */
>>   if (!inhibit_id_p && i < desired_matrix->nrows)
>>     force_p |= scrolling (f);
>>
>> Which means that 'scrolling', and thus 'scrolling_1' (where the
>> problem happens) will not be called if the line_ins_del_ok flag is not
>> set.
>>
>> Furthermore, we have in scrolling_1:
>>
>>   if (FRAME_SCROLL_REGION_OK (frame))
>>     {
>>       calculate_direct_scrolling (frame, matrix, window_size,
>>                                   unchanged_at_bottom,
>>                                   draw_cost, old_draw_cost,
>>                                   old_hash, new_hash, free_at_end);
>>       do_direct_scrolling (frame, frame->current_matrix,
>>                            matrix, window_size, unchanged_at_top);
>>     }
>>   else
>>     {
>>       calculate_scrolling (frame, matrix, window_size,
>> unchanged_at_bottom,
>>                            draw_cost, old_hash, new_hash,
>>                            free_at_end);
>>       do_scrolling (frame,
>>                     frame->current_matrix, matrix, window_size,
>>                     unchanged_at_top);
>>     }
>>
>> which means do_direct_scrolling (which causes the problem) will not be
>> called if the terminal's scroll_region_ok flag is not set.
>>
>> So given all of this, can you tell whether Emacs does TRT here?  That
>> is:
>>
> Sorry, what does TRT mean?

>
>>   . are all the capabilities that are supposed to be available for
>>     these two flags are indeed available?
>>
> How can I verify it?

>   . do we need to check any additional capabilities, which are in fact
>>     used in the problematic scenario, but not tested as part of
>>     setting these two flags?
>>
> It makes sense to me, but since the output is still correct after the
glitch, doesn't it mean that capabilities work correctly?

>
>> Assuming that Emacs does TRT, i.e. sets the flags correctly and uses
>> only the capabilities that are conditions for the flags set, then the
>> next important question is: why doesn't Gerd see the flickering on a
>> very similar system and the same terminal emulator?  Is it possible
>> that some other local software configuration on Dmitrii's machine
>> causes this, directly or indirectly?  E.g., Dmitrii, do you have some
>> display-related software/driver that has some "optimization" features
>> turned on?  If so, can you turn them off and try again?
>>
>> Thanks.
>>
>
>
> --
> *If you get an email from me outside of the 9-5 it is *not* because I'm
> always on or expect an immediate response from you; it is because of work
> flexibility
> <http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
> .  Evening and weekend emails are a sign I allocated some regular working
> hours for other things (such as family, gym, friends,...).  And I encourage
> you to feel free to do the same.
>
>

-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 11506 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Dmitrii Kuragin <kuragin@google.com>
> Date: Tue, 30 Aug 2022 09:34:24 -0700
> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>, 
> 	57434@debbugs.gnu.org
> 
>  So given all of this, can you tell whether Emacs does TRT here?  That
>  is:
> 
> Sorry, what does TRT mean? 

The Right Thing

>    . are all the capabilities that are supposed to be available for
>      these two flags are indeed available?
> 
> How can I verify it? 

By looking at all the tput calls we emit during the problematic code,
I guess.

>    . do we need to check any additional capabilities, which are in fact
>      used in the problematic scenario, but not tested as part of
>      setting these two flags?
> 
> It makes sense to me, but since the output is still correct after the glitch, doesn't it mean that capabilities work
> correctly? 

The flickering isn't supposed to happen.

Btw, can you figure out which screen lines flicker?  Do all of them
flicker, or just some?  And if you disable relative line-numbers
(i.e. use absolute line-numbers), do the lines still flicker? all of
them?

>  E.g., Dmitrii, do you have some
>  display-related software/driver that has some "optimization" features
>  turned on?  If so, can you turn them off and try again?

Did you try looking for such features on your system?

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 3071 bytes --]

On Tue, Aug 30, 2022 at 9:59 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Dmitrii Kuragin <kuragin@google.com>
> > Date: Tue, 30 Aug 2022 09:34:24 -0700
> > Cc: Gerd Möllmann <gerd.moellmann@gmail.com>,
> >       57434@debbugs.gnu.org
> >
> >  So given all of this, can you tell whether Emacs does TRT here?  That
> >  is:
> >
> > Sorry, what does TRT mean?
>
> The Right Thing
>
Thanks.

>
> >    . are all the capabilities that are supposed to be available for
> >      these two flags are indeed available?
> >
> > How can I verify it?
>
> By looking at all the tput calls we emit during the problematic code,
> I guess.
>
I'd be more than happy to try it, but I am not sure I can do that w/o help.

>
> >    . do we need to check any additional capabilities, which are in fact
> >      used in the problematic scenario, but not tested as part of
> >      setting these two flags?
> >
> > It makes sense to me, but since the output is still correct after the
> glitch, doesn't it mean that capabilities work
> > correctly?
>
> The flickering isn't supposed to happen.


> Btw, can you figure out which screen lines flicker?  Do all of them
> flicker, or just some?  And if you disable relative line-numbers
> (i.e. use absolute line-numbers), do the lines still flicker? all of
> them?
>
w/o relative line number, the problematic code doesn't trigger and there is
no flickering.

Flickering appears once there is some load on redrawing (line numbers,
themes, lh mode, etc...). The line numbers expose that the most.

Flickering region is always the same for the same cursor position. I
previously messaged it in
>>> Flickering is consistent for some specific area. If I scroll between 2
lines, back-and-forth Emacs flickers consistently.

So, it flickers consistently at the same area and it correlates with the
queue values (window, pos) in scroll.c when the const estimation decides to
use insertion instead of writing.
See scroll.c:do_direct_scrolling:697.

Where can I upload video so you can see the behavior?


>
> >  E.g., Dmitrii, do you have some
> >  display-related software/driver that has some "optimization" features
> >  turned on?  If so, can you turn them off and try again?
>
> Did you try looking for such features on your system?
>
I was looking around and didn't find anything specific. The new M1 Pro has
some rendering optimizations, but I tried a different option and it didn't
change anything.

BTW, I experience quite the same thing as in
https://mail.gnu.org/archive/html/help-gnu-emacs/2018-04/msg00304.html
-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 7149 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

On 22-08-30 15:48 , Dmitrii Kuragin wrote:
> Could you please try these settings?
> ```
> (setq display-line-numbers-type 'visual)
> (global-display-line-numbers-mode)
> 
> (global-hl-line-mode)
> 
> (global-display-fill-column-indicator-mode)
> ```

Thanks.

I've tried that with and without an additional M-x load-theme ... RET, 
and no luck, it doesn't flicker here.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

On 22-08-30 15:48 , Dmitrii Kuragin wrote:
> Could you please try these settings?
> ```
> (setq display-line-numbers-type 'visual)
> (global-display-line-numbers-mode)
> 
> (global-hl-line-mode)
> 
> (global-display-fill-column-indicator-mode)
> ```

Thanks.

I've tried that with and without an additional M-x load-theme ... RET, 
and no luck, it doesn't flicker here.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> On 22-08-30 15:48 , Dmitrii Kuragin wrote:
>> Could you please try these settings?
>> ```
>> (setq display-line-numbers-type 'visual)
>> (global-display-line-numbers-mode)
>> (global-hl-line-mode)
>> (global-display-fill-column-indicator-mode)
>> ```
>
> Thanks.
>
> I've tried that with and without an additional M-x load-theme ... RET,
> and no luck, it doesn't flicker here.

"Good news":

I've now installed Alacritty, and with this file

;; 57434.el
(setq display-line-numbers-type 'visual)
(global-display-line-numbers-mode)
(global-hl-line-mode)
(global-display-fill-column-indicator-mode)

and, in an alacritty terminal window

./emacs -nw -Q -l ../../57434.el xdisp.c
C-x 3
C-x o
C-x b *scratch* RET
C-x o
C-n/C-p

it sometimes flickers.  A lot less than in your screencast showing emacs
-Q, but it's there.

The window of xdisp.c doesn't need to scroll for this to happen.  It
suffices to hold C-n with key repeat, and then reverse to C-p with
repeat, and then C-n again, and so forth.  I have to repeat that a dozen
times or so until it flickers.  Key repeat is the maximum my system
allows (which is not very fast, TBH).

$TERM seems to be "alacritty" by default, which has different
capabilities than xterm-256color.  But the flickering is also there with
xterm-256color.

And I double-checked with Terminal.app again: no flickering.

\o/

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  57434@debbugs.gnu.org
> Date: Wed, 31 Aug 2022 09:02:07 +0200
> 
> "Good news":

Indeed.

> $TERM seems to be "alacritty" by default, which has different
> capabilities than xterm-256color.  But the flickering is also there with
> xterm-256color.
> 
> And I double-checked with Terminal.app again: no flickering.

When you run with Terminal.app, does the code identified by Dmitrii as
responsible for the flickering (in scroll.c) get executed?

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> Cc: Eli Zaretskii <eliz@gnu.org>,  57434@debbugs.gnu.org
>> Date: Wed, 31 Aug 2022 09:02:07 +0200
>> 
>> "Good news":
>
> Indeed.
>
>> $TERM seems to be "alacritty" by default, which has different
>> capabilities than xterm-256color.  But the flickering is also there with
>> xterm-256color.
>> 
>> And I double-checked with Terminal.app again: no flickering.
>
> When you run with Terminal.app, does the code identified by Dmitrii as
> responsible for the flickering (in scroll.c) get executed?

I haven't checked yet.  But...

Dmitrii, could you please check if something changes when you set
baud-rate?  I can't see flickering in alacritty, when I

(setq baud-rate 1000000)

but it's kind of hard to reproduce anyway for me.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1627 bytes --]

On Wed, Aug 31, 2022 at 4:55 AM Gerd Möllmann <gerd.moellmann@gmail.com>
wrote:

> Eli Zaretskii <eliz@gnu.org> writes:
>
> >> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> >> Cc: Eli Zaretskii <eliz@gnu.org>,  57434@debbugs.gnu.org
> >> Date: Wed, 31 Aug 2022 09:02:07 +0200
> >>
> >> "Good news":
> >
> > Indeed.
> >
> >> $TERM seems to be "alacritty" by default, which has different
> >> capabilities than xterm-256color.  But the flickering is also there with
> >> xterm-256color.
> >>
> >> And I double-checked with Terminal.app again: no flickering.
> >
> > When you run with Terminal.app, does the code identified by Dmitrii as
> > responsible for the flickering (in scroll.c) get executed?
>
> I haven't checked yet.  But...
>
> Dmitrii, could you please check if something changes when you set
> baud-rate?  I can't see flickering in alacritty, when I
>
> (setq baud-rate 1000000)
>

I do not know what it does, but it does help.

I mean, it fixes the problem with flickering completely.

Could you please elaborate a bit more about possible consequences of that?


> but it's kind of hard to reproduce anyway for me.
>


-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 3793 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Dmitrii Kuragin <kuragin@google.com> writes:

> I do not know what it does, but it does help. 
>
> I mean, it fixes the problem with flickering completely.

3 thumbs up :-)

>
> Could you please elaborate a bit more about possible consequences of that?

Baud-rate is the basis for cost calculations.  It basically specifies
how fast the communication with the underlying terminal is.  On slow
terminals Emacs tries harder to minimize communication, IIRC at the
expense of using slower capabilities.  It's all heuristics, though.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1451 bytes --]

Sorry, but seems like it helped in some configuration, but when I tried it
on my second monitor, it didn't work: https://youtu.be/IHzJ0QtuTgs

`baud-rate` improves the situation somehow, so that some portion of
flickering disappears, but the issue is still there and looks the same that
insertion over writing causes the issue.

On Wed, Aug 31, 2022 at 7:38 AM Gerd Möllmann <gerd.moellmann@gmail.com>
wrote:

> Dmitrii Kuragin <kuragin@google.com> writes:
>
> > I do not know what it does, but it does help.
> >
> > I mean, it fixes the problem with flickering completely.
>
> 3 thumbs up :-)
>
> >
> > Could you please elaborate a bit more about possible consequences of
> that?
>
> Baud-rate is the basis for cost calculations.  It basically specifies
> how fast the communication with the underlying terminal is.  On slow
> terminals Emacs tries harder to minimize communication, IIRC at the
> expense of using slower capabilities.  It's all heuristics, though.
>
>

-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 2953 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 2736 bytes --]

And here's one more video of default `baud-rate` vs 1000000
https://youtu.be/51EbX6bNP0M
And here's a video how smooth vim works in the same setup:
https://youtu.be/newP7XEA610

So, it is definitely not the terminal or Mac OS problem.

And here's a video when I use patched emacs with disabled insertion for mac
os: https://youtu.be/_ZXpzF6KOEQ

Additionally, I want to say that now I see the problem even when I connect
to a remote linux machine using SSH.

Could it be that alacritty so fast that it gets into the state when the
emacs pointing is in an unsynced state with terminal frequency?

Like, the issues we might have with frame rate of monitors and FPS
within games.

On Wed, Aug 31, 2022 at 9:00 AM Dmitrii Kuragin <kuragin@google.com> wrote:

> Sorry, but seems like it helped in some configuration, but when I tried it
> on my second monitor, it didn't work: https://youtu.be/IHzJ0QtuTgs
>
> `baud-rate` improves the situation somehow, so that some portion of
> flickering disappears, but the issue is still there and looks the same that
> insertion over writing causes the issue.
>
> On Wed, Aug 31, 2022 at 7:38 AM Gerd Möllmann <gerd.moellmann@gmail.com>
> wrote:
>
>> Dmitrii Kuragin <kuragin@google.com> writes:
>>
>> > I do not know what it does, but it does help.
>> >
>> > I mean, it fixes the problem with flickering completely.
>>
>> 3 thumbs up :-)
>>
>> >
>> > Could you please elaborate a bit more about possible consequences of
>> that?
>>
>> Baud-rate is the basis for cost calculations.  It basically specifies
>> how fast the communication with the underlying terminal is.  On slow
>> terminals Emacs tries harder to minimize communication, IIRC at the
>> expense of using slower capabilities.  It's all heuristics, though.
>>
>>
>
> --
> *If you get an email from me outside of the 9-5 it is *not* because I'm
> always on or expect an immediate response from you; it is because of work
> flexibility
> <http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
> .  Evening and weekend emails are a sign I allocated some regular working
> hours for other things (such as family, gym, friends,...).  And I encourage
> you to feel free to do the same.
>
>

-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 6283 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Dmitrii Kuragin <kuragin@google.com>
> Date: Wed, 31 Aug 2022 09:00:51 -0700
> Cc: Eli Zaretskii <eliz@gnu.org>, 57434@debbugs.gnu.org
> 
> Sorry, but seems like it helped in some configuration, but when I tried it on my second monitor, it didn't work:
> https://youtu.be/IHzJ0QtuTgs
> 
> `baud-rate` improves the situation somehow, so that some portion of flickering disappears, but the issue is
> still there and looks the same that insertion over writing causes the issue.

Given our inability to pinpoint the root cause of the problem, and the
unsolved mystery why on some macOS systems the problem is much more
prominent than on other macOS systems, I tend to introduce a variable
that users could set from Lisp to tell Emacs not to use the optimized
insert/delete-lines algorithm in scroll.c.  Unless some significant
new ideas or facts emerge in the next day or two, that is.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Dmitrii Kuragin <kuragin@google.com>
> Date: Wed, 31 Aug 2022 09:21:00 -0700
> Cc: Eli Zaretskii <eliz@gnu.org>, 57434@debbugs.gnu.org
> 
> And here's one more video of default `baud-rate` vs 1000000 https://youtu.be/51EbX6bNP0M
> And here's a video how smooth vim works in the same setup: https://youtu.be/newP7XEA610
> 
> So, it is definitely not the terminal or Mac OS problem.

That's a wrong conclusion, AFAIU.  By "terminal or macOS problem" we
mean that some terminal commands, like those that insert and delete
lines in a region of a screen, cause flickering on those systems
and/or those terminals.  That vim doesn't show this flickering tells
us nothing: it might well be that vim doesn't use these terminal
commands.  After all, if we disable the use of those terminal commands
in Emacs, like you already tried, the flickering disappears.  So by
the same logic we could conclude that there's no problem at all.

> Additionally, I want to say that now I see the problem even when I connect to a remote linux machine using
> SSH.

Why did you expect the problem to disappear when you use Emacs via SSH
on the same terminal?  It's the same Emacs using the same algorithms
to decide which terminal commands to use in each case.

> Could it be that alacritty so fast that it gets into the state when the emacs pointing is in an unsynced state with
> terminal frequency? 

I don't see how this can happen.  Emacs outputs commands basically one
after the other, so their execution should be at the terminal's speed.

However, there's one place where we accumulate bytes before flushing
them: in update_frame_1:

	  if (FRAME_TERMCAP_P (f))
	    {
	      /* Flush out every so many lines.
		 Also flush out if likely to have more than 1k buffered
		 otherwise.   I'm told that some telnet connections get
		 really screwed by more than 1k output at once.  */
	      FILE *display_output = FRAME_TTY (f)->output;
	      if (display_output)
		{
		  ptrdiff_t outq = __fpending (display_output);
		  if (outq > 900
		      || (outq > 20 && ((i - 1) % preempt_count == 0)))
		    fflush (display_output);
		}
	    }

So maybe it's worthwhile to see if playing with the 900 figure here
helps in any way.  Or maybe __fpending doesn't work well on macOS?

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 4244 bytes --]

On Wed, Aug 31, 2022 at 9:34 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Dmitrii Kuragin <kuragin@google.com>
> > Date: Wed, 31 Aug 2022 09:21:00 -0700
> > Cc: Eli Zaretskii <eliz@gnu.org>, 57434@debbugs.gnu.org
> >
> > And here's one more video of default `baud-rate` vs 1000000
> https://youtu.be/51EbX6bNP0M
> > And here's a video how smooth vim works in the same setup:
> https://youtu.be/newP7XEA610
> >
> > So, it is definitely not the terminal or Mac OS problem.
>
> That's a wrong conclusion, AFAIU.  By "terminal or macOS problem" we
> mean that some terminal commands, like those that insert and delete
> lines in a region of a screen, cause flickering on those systems
> and/or those terminals.  That vim doesn't show this flickering tells
> us nothing: it might well be that vim doesn't use these terminal
> commands.  After all, if we disable the use of those terminal commands
> in Emacs, like you already tried, the flickering disappears.  So by
> the same logic we could conclude that there's no problem at all.
>
Exactly, but it basically works for me. Which is what I need :)


> > Additionally, I want to say that now I see the problem even when I
> connect to a remote linux machine using
> > SSH.
>
> Why did you expect the problem to disappear when you use Emacs via SSH
> on the same terminal?  It's the same Emacs using the same algorithms
> to decide which terminal commands to use in each case.
>
Probably, it is caused by slowness of SSH and/or and my misunderstanding of
the terminal capabilities, but I didn't see the problem when I was
connected to Linux using SSH.

>
> > Could it be that alacritty so fast that it gets into the state when the
> emacs pointing is in an unsynced state with
> > terminal frequency?
>
> I don't see how this can happen.  Emacs outputs commands basically one
> after the other, so their execution should be at the terminal's speed.
>
It also might be caused by my limited knowledge of TTY. But, I see that we
redraw linest not from top to bottom, but actually we try to redraw
different portions of the screen at different times. (I am looking into
`do_scrolling`).

And my assumption is that we cleared some portion of the screen and
prepared it for new contents, but due to unsynced frame rates, the terminal
redraws that partial state. And only then, we add some content in those
empty lines and then the terminal redraws it again and we see what we
needed to see.
Do we always draw frame as a single atomic operation or those things aren't
synchronized at all?


> However, there's one place where we accumulate bytes before flushing
> them: in update_frame_1:
>
>           if (FRAME_TERMCAP_P (f))
>             {
>               /* Flush out every so many lines.
>                  Also flush out if likely to have more than 1k buffered
>                  otherwise.   I'm told that some telnet connections get
>                  really screwed by more than 1k output at once.  */
>               FILE *display_output = FRAME_TTY (f)->output;
>               if (display_output)
>                 {
>                   ptrdiff_t outq = __fpending (display_output);
>                   if (outq > 900
>                       || (outq > 20 && ((i - 1) % preempt_count == 0)))
>                     fflush (display_output);
>                 }
>             }
>
> So maybe it's worthwhile to see if playing with the 900 figure here
> helps in any way.  Or maybe __fpending doesn't work well on macOS?

I tried bigger/smaller and delete the condition completely, it didn't
help....

>
>
Also, I want to point out that `baud-rate` works and reduces part of the
flickering and improves the situation overall. but `tty->line_ins_del_ok =
0;` fixes it completely.

I can try to add a new variable to configure/override the behavior.



-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 7936 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

> Given our inability to pinpoint the root cause of the problem, and the
> unsolved mystery why on some macOS systems the problem is much more
> prominent than on other macOS systems, I tend to introduce a variable
> that users could set from Lisp to tell Emacs not to use the optimized
> insert/delete-lines algorithm in scroll.c.  Unless some significant
> new ideas or facts emerge in the next day or two, that is.

I guess I've found these facts today, while rummaging in alacritty's
Github project.

Let me mention first that there's quite some flushing and flickering
going on in alacritty's Github issues with all sorts of terminal
applications (tmux, vim, neovim, ...), and on all platforms it runs on
(MS-Windows, GNU/Linux, macOS, ...).

Alacritty is one of a number of terminal emulators using OpenGL with GPU
accelerated framebuffer display.  I must admit that I didn't realize up
to now that such a thing exists.

Framebuffer updates depend on monitor refresh rates, among other things.
One can only update at some given points in time (vsync, vblank, and so
on, I'm not an expert).  That's the typical N frames/second thing one
can find in various contexts.  (And note the different behavior Dmitrii
mentioned on his second monitor.)

Very simplified, what happens is that Alacritty seems to put framebuffer
contents on the screen in the middle of updates sent from terminal
clients.  Which leads to flickering.

For a longer story, see for instance

https://github.com/karlstav/cava/issues/453
https://bytemeta.vip/repo/karlstav/cava/issues/453
https://github.com/alacritty/alacritty/issues/598
https://github.com/kovidgoyal/kitty/issues/4817

and also follow links there.  One screencast there shows vim flickering
in 100% the exact same way (even the colors) that Dmitrii showed with
Emacs :-).

The terminal emulators' proposed solution for this flickering seems to
be that clients support "synchronized updates" as they call it.  That
is, a client sends the terminal emulator begin-update/end-update control
sequences, which the emulator uses to avoid framebuffer updates in the
middle of the update.

Proposed specification:

https://gitlab.com/gnachman/iterm2/-/wikis/synchronized-updates-spec

A number of terminal clients seem to have added this.  Tmux seems to do
it.  Vim I don't know but I'd wager it does when I see the screencast
mentioned above and what Dmitrii showed.  I think other applications do
too, but I haven't digged deeper.

I couldn't find information which terminfo capabilties are supposed to
be used for this, and I don't think it's part of any official
specification.  And I'd like to add that alacritty.org says about itself
"The software is considered to be at a beta level of readiness", so I'm
a bit wary how stable this all is.

In summary, this is for me a very likely candidate for "the root cause".
Someone (tm) should probably take a deeper look at this.

Maybe Dmitrii is up for it?

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: Dmitrii Kuragin <kuragin@google.com>,  57434@debbugs.gnu.org
> Date: Thu, 01 Sep 2022 07:44:52 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Given our inability to pinpoint the root cause of the problem, and the
> > unsolved mystery why on some macOS systems the problem is much more
> > prominent than on other macOS systems, I tend to introduce a variable
> > that users could set from Lisp to tell Emacs not to use the optimized
> > insert/delete-lines algorithm in scroll.c.  Unless some significant
> > new ideas or facts emerge in the next day or two, that is.
> 
> I guess I've found these facts today, while rummaging in alacritty's
> Github project.

Thanks.

So we have 2 alternatives:

  . declare that flickering on alacritty is currently a known problem,
    and wait till Someone comes up with the proper solution for Emacs;
  . add that variable I mentioned above, and let users try to fix the
    problem with alacritty by flipping it.

TBH, given your description, I'm no longer sure a simple boolean
variable that disables insert/delete-line optimization will do, since
the issue seems to be a much more general one.  How sure are we that
some other scenario of redrawing a TTY frame won't cause similar
flickering regardless of the insert/delete-line feature?  So maybe
just document the issue in PROBLEMS and advise against using
alacritty?

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

> So we have 2 alternatives:
>
>   . declare that flickering on alacritty is currently a known problem,
>     and wait till Someone comes up with the proper solution for Emacs;
>   . add that variable I mentioned above, and let users try to fix the
>     problem with alacritty by flipping it.
>
> TBH, given your description, I'm no longer sure a simple boolean
> variable that disables insert/delete-line optimization will do, since
> the issue seems to be a much more general one.  How sure are we that
> some other scenario of redrawing a TTY frame won't cause similar
> flickering regardless of the insert/delete-line feature?

That's what I think, too.

Maybe Dmitrii is Someone :-).

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> So we have 2 alternatives:
>>
>>   . declare that flickering on alacritty is currently a known problem,
>>     and wait till Someone comes up with the proper solution for Emacs;
>>   . add that variable I mentioned above, and let users try to fix the
>>     problem with alacritty by flipping it.
>>
>> TBH, given your description, I'm no longer sure a simple boolean
>> variable that disables insert/delete-line optimization will do, since
>> the issue seems to be a much more general one.  How sure are we that
>> some other scenario of redrawing a TTY frame won't cause similar
>> flickering regardless of the insert/delete-line feature?
>
> That's what I think, too.
>
> Maybe Dmitrii is Someone :-).

I had an idea: Until this all is standardized/stable, how about adding
two hooks that are called when Emacs is beginning a terminal update and
when it is done?  The user could then send whatever he wants to the
terminal.  Or maybe beginnign an update/end an update on any type of
frame even?  Don't know if that's useful though.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: kuragin@google.com,  57434@debbugs.gnu.org
> Date: Thu, 01 Sep 2022 10:18:57 +0200
> 
> I had an idea: Until this all is standardized/stable, how about adding
> two hooks that are called when Emacs is beginning a terminal update and
> when it is done?  The user could then send whatever he wants to the
> terminal.  Or maybe beginnign an update/end an update on any type of
> frame even?  Don't know if that's useful though.

I'm not sure I understand: what would the user send to the terminal in
those hooks?

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

> I'm not sure I understand: what would the user send to the terminal in
> those hooks?

They would send the begin-update/end-update control sequences.  That
would be ESC P <something> in the one proposal I mentioned.

It would of course be better if we could do that automatically.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: kuragin@google.com,  57434@debbugs.gnu.org
> Date: Thu, 01 Sep 2022 10:56:35 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I'm not sure I understand: what would the user send to the terminal in
> > those hooks?
> 
> They would send the begin-update/end-update control sequences.  That
> would be ESC P <something> in the one proposal I mentioned.
> 
> It would of course be better if we could do that automatically.

Exactly.  I don't think it's reasonable to expect users to figure out
what and when to send if we cannot figure that out ourselves.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1725 bytes --]

Thank you for digging deeper into this! There is a lot of useful
information. But,
Here's a video of flickering https://youtu.be/Is2ebMXjhxg:
 - Different MacBook (2019 I believe).
 - Terminal.app (The one which comes as a standard).
 - Simplest Emacs from http://emacsformacosx.com/
 - No tmux.
 - No advanced user customizations (it is not my laptop).
 - Only display-line-numbers-mode is enabled with 'visual type.

It is definitely not attached to Alacritty, but probably the same issue
with refresh vs frame rate updates.

On Thu, Sep 1, 2022 at 4:30 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Gerd Möllmann <gerd.moellmann@gmail.com>
> > Cc: kuragin@google.com,  57434@debbugs.gnu.org
> > Date: Thu, 01 Sep 2022 10:56:35 +0200
> >
> > Eli Zaretskii <eliz@gnu.org> writes:
> >
> > > I'm not sure I understand: what would the user send to the terminal in
> > > those hooks?
> >
> > They would send the begin-update/end-update control sequences.  That
> > would be ESC P <something> in the one proposal I mentioned.
> >
> > It would of course be better if we could do that automatically.
>
> Exactly.  I don't think it's reasonable to expect users to figure out
> what and when to send if we cannot figure that out ourselves.
>


-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 4093 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 2479 bytes --]

Sorry, I completely misunderstood it.

I actually wasn't able to reproduce flickering on Terminal.app, only on
Alacritty.

I need a bit more rest :)

On Thu, Sep 1, 2022 at 5:27 AM Dmitrii Kuragin <kuragin@google.com> wrote:

> Thank you for digging deeper into this! There is a lot of useful
> information. But,
> Here's a video of flickering https://youtu.be/Is2ebMXjhxg:
>  - Different MacBook (2019 I believe).
>  - Terminal.app (The one which comes as a standard).
>  - Simplest Emacs from http://emacsformacosx.com/
>  - No tmux.
>  - No advanced user customizations (it is not my laptop).
>  - Only display-line-numbers-mode is enabled with 'visual type.
>
> It is definitely not attached to Alacritty, but probably the same issue
> with refresh vs frame rate updates.
>
> On Thu, Sep 1, 2022 at 4:30 AM Eli Zaretskii <eliz@gnu.org> wrote:
>
>> > From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> > Cc: kuragin@google.com,  57434@debbugs.gnu.org
>> > Date: Thu, 01 Sep 2022 10:56:35 +0200
>> >
>> > Eli Zaretskii <eliz@gnu.org> writes:
>> >
>> > > I'm not sure I understand: what would the user send to the terminal in
>> > > those hooks?
>> >
>> > They would send the begin-update/end-update control sequences.  That
>> > would be ESC P <something> in the one proposal I mentioned.
>> >
>> > It would of course be better if we could do that automatically.
>>
>> Exactly.  I don't think it's reasonable to expect users to figure out
>> what and when to send if we cannot figure that out ourselves.
>>
>
>
> --
> *If you get an email from me outside of the 9-5 it is *not* because I'm
> always on or expect an immediate response from you; it is because of work
> flexibility
> <http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
> .  Evening and weekend emails are a sign I allocated some regular working
> hours for other things (such as family, gym, friends,...).  And I encourage
> you to feel free to do the same.
>
>

-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 6362 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Dmitrii Kuragin <kuragin@google.com> writes:

> Sorry, I completely misunderstood it. 
>
> I actually wasn't able to reproduce flickering on Terminal.app, only
> on Alacritty.

No problem.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: Dmitrii Kuragin <kuragin@google.com>,  57434@debbugs.gnu.org
> Date: Thu, 01 Sep 2022 07:44:52 +0200
> 
> Proposed specification:
> 
> https://gitlab.com/gnachman/iterm2/-/wikis/synchronized-updates-spec

AFAIU, this defines just two special commands: Begin Synchronized
Update (BSU) and End Synchronized Update (ESU).

A natural place to emit these is in, respectively, update_begin_hook
and update_end_hook.  These two hooks are currently set to NULL (in
term.c) for TTY frames.  So as a first try, I suggest to define these
hooks for TTY frames, and make them emit these two commands.  If doing
so resolves the problem with flickering on alacritty, we can think how
to add that cleanly only for alacritty.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> Cc: kuragin@google.com, 57434@debbugs.gnu.org
> Date: Fri, 02 Sep 2022 10:16:35 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> 
> A natural place to emit these is in, respectively, update_begin_hook
> and update_end_hook.  These two hooks are currently set to NULL (in
> term.c) for TTY frames.  So as a first try, I suggest to define these
> hooks for TTY frames, and make them emit these two commands.  If doing
> so resolves the problem with flickering on alacritty, we can think how
> to add that cleanly only for alacritty.

Actually, I think we'll need one more small change.  These hooks are
called from update_begin and update_end like this:

      /* Update the display.  */
      if (FRAME_INITIAL_P (f))
        /* No actual display to update so the "update" is a nop and
           obviously isn't interrupted by pending input.  */
        paused_p = false;
      else
        {
          update_begin (f);
          paused_p = update_frame_1 (f, force_p, inhibit_hairy_id_p, 1, false);
          update_end (f);
        }

      if (FRAME_TERMCAP_P (f) || FRAME_MSDOS_P (f))
        {
          if (FRAME_TTY (f)->termscript)
	    fflush (FRAME_TTY (f)->termscript);
	  if (FRAME_TERMCAP_P (f))
	    fflush (FRAME_TTY (f)->output);
        }

I think the fact that we call fflush (FRAME_TTY (f)->output) after
update_end is a conceptual bug, which only goes unnoticed because
update_end is a no-op on TTY frames.  At least for the purpose of
testing the above possible fix, the order should be reversed: we
should fflush the terminal output _before_ we call update_end.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> Cc: Dmitrii Kuragin <kuragin@google.com>,  57434@debbugs.gnu.org
>> Date: Thu, 01 Sep 2022 07:44:52 +0200
>> 
>> Proposed specification:
>> 
>> https://gitlab.com/gnachman/iterm2/-/wikis/synchronized-updates-spec
>
> AFAIU, this defines just two special commands: Begin Synchronized
> Update (BSU) and End Synchronized Update (ESU).

Yes,

BTW, I think I forgot to add the linke yesterday, there's also) a
second specification here:

https://gist.github.com/christianparpart/d8a62cc1ab659194337d73e399004036

This page looks relatively new, so maybe it's a newer version of the
same proposal?  I don't know how this all fits together.

This one has also begin-update/end-update control sequences plus a
query.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

> Actually, I think we'll need one more small change.  These hooks are
> called from update_begin and update_end like this:

Dmitrii, do you want to try implementing that?

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> Actually, I think we'll need one more small change.  These hooks are
>> called from update_begin and update_end like this:
>
> Dmitrii, do you want to try implementing that?

Searching the web for "emacs synchronized updates" turned up this patch,
which you could try

https://gist.github.com/Patryk27/c7b9dac8113f4ccdb2ef74e0083d9d41

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: kuragin@google.com,  57434@debbugs.gnu.org
> Date: Sat, 03 Sep 2022 10:04:14 +0200
> 
> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
> 
> > Eli Zaretskii <eliz@gnu.org> writes:
> >
> >> Actually, I think we'll need one more small change.  These hooks are
> >> called from update_begin and update_end like this:
> >
> > Dmitrii, do you want to try implementing that?
> 
> Searching the web for "emacs synchronized updates" turned up this patch,
> which you could try
> 
> https://gist.github.com/Patryk27/c7b9dac8113f4ccdb2ef74e0083d9d41

Just without the explicit calls to fflush, please.  I think that's
there to cover for the bug in update_frame which I mentioned before.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1788 bytes --]

Thank you for all the findings, I was already working on an implementation
:)

I didn't dig too deep but it worked somehow.

I was stuck on the support of tmux, maybe I misconfigured the tmux or I
didn't have to use DCS escapes. I will try to figure this out.

BTW, the compilation on master fails w/:
```
debug-early(error (error "Keyword argument :inhibit-native-compile not one
of (:version :inhibit-provide :coding :autoloads :compile :provide)"))
```

On Sat, Sep 3, 2022 at 2:07 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Gerd Möllmann <gerd.moellmann@gmail.com>
> > Cc: kuragin@google.com,  57434@debbugs.gnu.org
> > Date: Sat, 03 Sep 2022 10:04:14 +0200
> >
> > Gerd Möllmann <gerd.moellmann@gmail.com> writes:
> >
> > > Eli Zaretskii <eliz@gnu.org> writes:
> > >
> > >> Actually, I think we'll need one more small change.  These hooks are
> > >> called from update_begin and update_end like this:
> > >
> > > Dmitrii, do you want to try implementing that?
> >
> > Searching the web for "emacs synchronized updates" turned up this patch,
> > which you could try
> >
> > https://gist.github.com/Patryk27/c7b9dac8113f4ccdb2ef74e0083d9d41
>
> Just without the explicit calls to fflush, please.  I think that's
> there to cover for the bug in update_frame which I mentioned before.
>


-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 4304 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Dmitrii Kuragin <kuragin@google.com>
> Date: Sat, 3 Sep 2022 09:35:09 -0700
> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>, 
> 	57434@debbugs.gnu.org
> 
> BTW, the compilation on master fails w/:
> ```
> debug-early(error (error "Keyword argument :inhibit-native-compile not one of (:version :inhibit-provide :coding
> :autoloads :compile :provide)"))
> ```

Remove lisp/emacs-lisp/generate-lisp-files.elc, then try again.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1496 bytes --]

Thank you.

I didn't find this file and it seems like `make clean` didn't do its thing.
So `find . -name "*.elc" -type f | xargs rm -f` did its thing.

Also, I tried to enable syncing and it works like a charm. The next problem
is I need to configure tmux to somehow respect it.

When I run Emacs in Terminal w/o tmux, syncing works well. But, within tmux
it just doesn't work. We can always wrap escape sequences (DCS?). But it
seems like there should be support from tmux on this.

On Sat, Sep 3, 2022 at 9:51 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Dmitrii Kuragin <kuragin@google.com>
> > Date: Sat, 3 Sep 2022 09:35:09 -0700
> > Cc: Gerd Möllmann <gerd.moellmann@gmail.com>,
> >       57434@debbugs.gnu.org
> >
> > BTW, the compilation on master fails w/:
> > ```
> > debug-early(error (error "Keyword argument :inhibit-native-compile not
> one of (:version :inhibit-provide :coding
> > :autoloads :compile :provide)"))
> > ```
>
> Remove lisp/emacs-lisp/generate-lisp-files.elc, then try again.
>


-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 3166 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Dmitrii Kuragin <kuragin@google.com>
> Date: Sat, 3 Sep 2022 10:14:06 -0700
> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>, 
> 	57434@debbugs.gnu.org
> 
> Also, I tried to enable syncing and it works like a charm. The next problem is I need to configure tmux to
> somehow respect it. 
> 
> When I run Emacs in Terminal w/o tmux, syncing works well. But, within tmux it just doesn't work. We can
> always wrap escape sequences (DCS?). But it seems like there should be support from tmux on this.

Not sure I understand what does this issue have to do with tmux.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Dmitrii Kuragin <kuragin@google.com> writes:

> Also, I tried to enable syncing and it works like a charm.

So I take it that you fixed the problem that you originally reported
with Alacritty.

Could you please show a diff of what you did?

Also, did you try both synchronized update proposols?

https://gitlab.com/gnachman/iterm2/-/wikis/synchronized-updates-spec
https://gist.github.com/christianparpart/d8a62cc1ab659194337d73e399004036

If you didn't try both, could you please do and report back?

> The next problem is I need to configure tmux to somehow respect it. 

Sorry, can't help with that, I don't know tmux.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Dmitrii Kuragin <kuragin@google.com> writes:
>
>> Also, I tried to enable syncing and it works like a charm.
>
> So I take it that you fixed the problem that you originally reported
> with Alacritty.
>
> Could you please show a diff of what you did?
>
> Also, did you try both synchronized update proposols?
>
> https://gitlab.com/gnachman/iterm2/-/wikis/synchronized-updates-spec
> https://gist.github.com/christianparpart/d8a62cc1ab659194337d73e399004036
>
> If you didn't try both, could you please do and report back?
>
>> The next problem is I need to configure tmux to somehow respect it. 
>
> Sorry, can't help with that, I don't know tmux.

Ping.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1.1: Type: text/plain, Size: 2167 bytes --]

Hello,

Sorry I made you wait. I tried the patch (see attachments) and everything
worked perfectly. No flickering.

I tried to implement the second spec, but it didn't work (at least in
Alacritty). But, I can try to add support for both.

BTW, tmux uses the sync update for their own TUI and they check "Sync"
terminal capability. Do we need to do the same or we can just send the
escapes and hope the unsupported terminal would just recover?

Side note: I still have the issue when I run emacs inside of tmux, but it
is due to lack of support from tmux side. I created a bug and will try to
implement the functionality in tmux sooner or later. See
https://github.com/tmux/tmux/issues/3325 foe details. I assume, in the case
of tmux, the multiplexer just consumes the escape codes and doesn't send
them to the parent terminal.


On Tue, Sep 6, 2022 at 9:59 PM Gerd Möllmann <gerd.moellmann@gmail.com>
wrote:

> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>
> > Dmitrii Kuragin <kuragin@google.com> writes:
> >
> >> Also, I tried to enable syncing and it works like a charm.
> >
> > So I take it that you fixed the problem that you originally reported
> > with Alacritty.
> >
> > Could you please show a diff of what you did?
> >
> > Also, did you try both synchronized update proposols?
> >
> > https://gitlab.com/gnachman/iterm2/-/wikis/synchronized-updates-spec
> >
> https://gist.github.com/christianparpart/d8a62cc1ab659194337d73e399004036
> >
> > If you didn't try both, could you please do and report back?
> >
> >> The next problem is I need to configure tmux to somehow respect it.
> >
> > Sorry, can't help with that, I don't know tmux.
>
> Ping.
>


-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #1.2: Type: text/html, Size: 4673 bytes --]

[-- Attachment #2: 0001-Implement-Synchrnized-Update-for-frame-rendering-in-.patch --]
[-- Type: application/octet-stream, Size: 1632 bytes --]

From df63c2616d99b8d9162c9c422c50dc3c8de65d25 Mon Sep 17 00:00:00 2001
From: Dmitrii Kuragin <kuragin@chromium.org>
Date: Sat, 3 Sep 2022 09:30:48 -0700
Subject: [PATCH] Implement Synchrnized Update for frame rendering in TTY.

Spec: https://gitlab.com/gnachman/iterm2/-/wikis/synchronized-updates-spec

* src/term.c (set_tty_hooks, tty_update_end, tty_update_begin): Put
  escape sequence at the beginning and end of the fram update in order
  to inform terminal about synchonization points.
---
 src/term.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/src/term.c b/src/term.c
index 2e43d89232..5f7c07b2cf 100644
--- a/src/term.c
+++ b/src/term.c
@@ -227,6 +227,14 @@ tty_reset_terminal_modes (struct terminal *terminal)
     }
 }
 
+static void
+tty_update_begin (struct frame *f)
+{
+  struct tty_display_info *tty = FRAME_TTY (f);
+
+  fputs ("\033P=1s\033\\", tty->output);
+}
+
 /* Flag the end of a display update on a termcap terminal. */
 
 static void
@@ -238,6 +246,7 @@ tty_update_end (struct frame *f)
     tty_show_cursor (tty);
   tty_turn_off_insert (tty);
   tty_background_highlight (tty);
+  fputs ("\033P=2s\033\\", tty->output);
   fflush (tty->output);
 }
 
@@ -3880,6 +3889,7 @@ set_tty_hooks (struct terminal *terminal)
   terminal->ring_bell_hook = &tty_ring_bell;
   terminal->reset_terminal_modes_hook = &tty_reset_terminal_modes;
   terminal->set_terminal_modes_hook = &tty_set_terminal_modes;
+  terminal->update_begin_hook = &tty_update_begin;
   terminal->update_end_hook = &tty_update_end;
 #ifdef MSDOS
   terminal->menu_show_hook = &x_menu_show;
-- 
2.37.2.789.g6183377224-goog

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Dmitrii Kuragin <kuragin@google.com>
> Date: Wed, 7 Sep 2022 09:11:32 -0700
> Cc: Eli Zaretskii <eliz@gnu.org>, 57434@debbugs.gnu.org
> 
> Sorry I made you wait. I tried the patch (see attachments) and everything worked perfectly. No flickering.

OK, thanks.

So now the next question is: on what should we base the activation of
these hooks?  There are several alternatives:

  . if alacritty produces a distinct value from tty-type, we could use
    that, or
  . if alacritty has a distinct terminfo capability that other
    terminals don't, we could use that, or
  . expose a variable to Lisp that users could set in order to turn
    this on and off, and tell users to turn it on if they see the
    issue

Any other ideas?

> BTW, tmux uses the sync update for their own TUI and they check "Sync" terminal capability. Do we need to
> do the same or we can just send the escapes and hope the unsupported terminal would just recover?

What is that Sync capability, what is its value for tmux, and how is
it supposed to be used?  Is there any documentation I could read about
that?

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Dmitrii Kuragin <kuragin@google.com>
>> Date: Wed, 7 Sep 2022 09:11:32 -0700
>> Cc: Eli Zaretskii <eliz@gnu.org>, 57434@debbugs.gnu.org
>> 
>> Sorry I made you wait. I tried the patch (see attachments) and everything worked perfectly. No flickering.
>
> OK, thanks.
>
> So now the next question is: on what should we base the activation of
> these hooks?  There are several alternatives:
>
>   . if alacritty produces a distinct value from tty-type, we could use
>     that, or
>   . if alacritty has a distinct terminfo capability that other
>     terminals don't, we could use that, or
>   . expose a variable to Lisp that users could set in order to turn
>     this on and off, and tell users to turn it on if they see the
>     issue
>
> Any other ideas?

I'd like to emphasize that this is not a problem limited to Alacritty.
There are a number of terminal emulators with GPU accelleration, which
all share the same cpmceptual problem. Alacritty is just the one with
the best marketing, riding the Rust wave.

AFAICT, we have the following situation:

 - We have two proposals P1 and P2 (that we know of).  Alacritty
   implements P1 only, says Dmitrii, and P2 has a table of emulators
   implementing P2, which may or may not implement P1.
   
 - Neither P1 nor P2 are detectable as a terminfo capability, so one has
   to match TERM or use a boolean switch.  I don't know if all emulators
   use a discernable TERM, or if they use standard TERM names.

>
>> BTW, tmux uses the sync update for their own TUI and they check "Sync" terminal capability. Do we need to
>> do the same or we can just send the escapes and hope the unsupported terminal would just recover?
>
> What is that Sync capability, what is its value for tmux, and how is
> it supposed to be used?  Is there any documentation I could read about
> that?

I know Sync as an xterm extension, although I don't remember what it is
for.  It's a non-standard terminfo capability which is only shown with
infocmp -x.

Neither P1 nor P2 mention Sync in any way.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: Dmitrii Kuragin <kuragin@google.com>,  57434@debbugs.gnu.org
> Date: Thu, 08 Sep 2022 07:31:17 +0200
> 
> > So now the next question is: on what should we base the activation of
> > these hooks?  There are several alternatives:
> >
> >   . if alacritty produces a distinct value from tty-type, we could use
> >     that, or
> >   . if alacritty has a distinct terminfo capability that other
> >     terminals don't, we could use that, or
> >   . expose a variable to Lisp that users could set in order to turn
> >     this on and off, and tell users to turn it on if they see the
> >     issue
> >
> > Any other ideas?
> 
> I'd like to emphasize that this is not a problem limited to Alacritty.
> There are a number of terminal emulators with GPU accelleration, which
> all share the same cpmceptual problem. Alacritty is just the one with
> the best marketing, riding the Rust wave.
> 
> AFAICT, we have the following situation:
> 
>  - We have two proposals P1 and P2 (that we know of).  Alacritty
>    implements P1 only, says Dmitrii, and P2 has a table of emulators
>    implementing P2, which may or may not implement P1.
>    
>  - Neither P1 nor P2 are detectable as a terminfo capability, so one has
>    to match TERM or use a boolean switch.  I don't know if all emulators
>    use a discernable TERM, or if they use standard TERM names.

So you are saying at this point only my last alternative, i.e. a
variable exposed to Lisp, is a reasonable way ahead?  Until, that is,
those emulators get their act together and agree on some standard way
of detecting the need for synchronized updates?

> >> BTW, tmux uses the sync update for their own TUI and they check "Sync" terminal capability. Do we need to
> >> do the same or we can just send the escapes and hope the unsupported terminal would just recover?
> >
> > What is that Sync capability, what is its value for tmux, and how is
> > it supposed to be used?  Is there any documentation I could read about
> > that?
> 
> I know Sync as an xterm extension, although I don't remember what it is
> for.  It's a non-standard terminfo capability which is only shown with
> infocmp -x.
> 
> Neither P1 nor P2 mention Sync in any way.

Does terminfo return the Sync capability on those terminals, as it
returns the standard ones?  That is, can we test for it inside
init_tty?

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

> So you are saying at this point only my last alternative, i.e. a
> variable exposed to Lisp, is a reasonable way ahead?  Until, that is,
> those emulators get their act together and agree on some standard way
> of detecting the need for synchronized updates?

Yes, I think an option is currently all we can do.  And somehow define
what to send to the terminal, P1 or P2.

I installed iTerm2 just now, which implements P2 at least.  iTerm2 has a
preference options to switch GPU acceleration on/off.  iTerm2 uses
TERM=xterm256-color, independent of the GPU setting.  So matching TERM
wouldn't work with iTerm2.  The query control sequence of P2 seems to
return if GPU accel is on or off on iTerm2.

>
>> >> BTW, tmux uses the sync update for their own TUI and they check "Sync" terminal capability. Do we need to
>> >> do the same or we can just send the escapes and hope the unsupported terminal would just recover?
>> >
>> > What is that Sync capability, what is its value for tmux, and how is
>> > it supposed to be used?  Is there any documentation I could read about
>> > that?
>> 
>> I know Sync as an xterm extension, although I don't remember what it is
>> for.  It's a non-standard terminfo capability which is only shown with
>> infocmp -x.
>> 
>> Neither P1 nor P2 mention Sync in any way.
>
> Does terminfo return the Sync capability on those terminals, as it
> returns the standard ones?  That is, can we test for it inside
> init_tty?

I think so.

BTW, iTerm2 doesn't seem to define Sync.  Kitty does. It's a mess.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: kuragin@google.com,  57434@debbugs.gnu.org
> Date: Thu, 08 Sep 2022 08:43:46 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > So you are saying at this point only my last alternative, i.e. a
> > variable exposed to Lisp, is a reasonable way ahead?  Until, that is,
> > those emulators get their act together and agree on some standard way
> > of detecting the need for synchronized updates?
> 
> Yes, I think an option is currently all we can do.  And somehow define
> what to send to the terminal, P1 or P2.

What is P1 and what is P2?

> I installed iTerm2 just now, which implements P2 at least.  iTerm2 has a
> preference options to switch GPU acceleration on/off.  iTerm2 uses
> TERM=xterm256-color, independent of the GPU setting.  So matching TERM
> wouldn't work with iTerm2.  The query control sequence of P2 seems to
> return if GPU accel is on or off on iTerm2.

What is the "query control sequence"?

> >> I know Sync as an xterm extension, although I don't remember what it is
> >> for.  It's a non-standard terminfo capability which is only shown with
> >> infocmp -x.
> >> 
> >> Neither P1 nor P2 mention Sync in any way.
> >
> > Does terminfo return the Sync capability on those terminals, as it
> > returns the standard ones?  That is, can we test for it inside
> > init_tty?
> 
> I think so.
> 
> BTW, iTerm2 doesn't seem to define Sync.  Kitty does. It's a mess.

We could at least detect that when terminfo returns non-nil for Sync.

Do you happen to know where is the definitive documentation of Sync?

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

> What is P1 and what is P2?

The two proposals that we know of.

>
>> I installed iTerm2 just now, which implements P2 at least.  iTerm2 has a
>> preference options to switch GPU acceleration on/off.  iTerm2 uses
>> TERM=xterm256-color, independent of the GPU setting.  So matching TERM
>> wouldn't work with iTerm2.  The query control sequence of P2 seems to
>> return if GPU accel is on or off on iTerm2.
>
> What is the "query control sequence"?

Proposal P2 defines 3 control sequences.  In addtion to begin/end, it
contains a control sequence to determine the status of synchronized
updates: on, off, permanently on, permanently off (AFAIU).

> Do you happen to know where is the definitive documentation of Sync?

Sorry, I don't know a definitive documentation.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: kuragin@google.com,  57434@debbugs.gnu.org
> Date: Thu, 08 Sep 2022 10:43:49 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > What is P1 and what is P2?
> 
> The two proposals that we know of.
> 
> >
> >> I installed iTerm2 just now, which implements P2 at least.  iTerm2 has a
> >> preference options to switch GPU acceleration on/off.  iTerm2 uses
> >> TERM=xterm256-color, independent of the GPU setting.  So matching TERM
> >> wouldn't work with iTerm2.  The query control sequence of P2 seems to
> >> return if GPU accel is on or off on iTerm2.
> >
> > What is the "query control sequence"?
> 
> Proposal P2 defines 3 control sequences.  In addtion to begin/end, it
> contains a control sequence to determine the status of synchronized
> updates: on, off, permanently on, permanently off (AFAIU).

I guess by P1 you mean these two sequences:

  BSU: ESC P = 1 s ESC \
  ESU: ESC P = 2 s ESC \

whereas by P2 you mean these:

  DECRQM: ESC [ ? 2026 $ p
  DECRPM: ESC [ ? 2026 ; n $ y  (with n = 0..4, and 0 or 4 means "no support")
  BSU:    ESC [ ? 2026 h
  ESU:    ESC [ ? 2026 l

> > Do you happen to know where is the definitive documentation of Sync?
> 
> Sorry, I don't know a definitive documentation.

And any documentation at all?

Anyway, given the problems usually related to querying terminals about
potentially unsupported features, and the general mess in this field,
I think our best bet is to have a function that could switch the
frame-update hooks between these 3 states:

  . unused
  . used with P1 BSU/ESU sequences
  . used with P2 BSU/ESU sequences

WDYT?

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

> I guess by P1 you mean these two sequences:
>
>   BSU: ESC P = 1 s ESC \
>   ESU: ESC P = 2 s ESC \
>
> whereas by P2 you mean these:
>
>   DECRQM: ESC [ ? 2026 $ p
>   DECRPM: ESC [ ? 2026 ; n $ y  (with n = 0..4, and 0 or 4 means "no support")
>   BSU:    ESC [ ? 2026 h
>   ESU:    ESC [ ? 2026 l

Yes, that looks like them.

>
>> > Do you happen to know where is the definitive documentation of Sync?
>> 
>> Sorry, I don't know a definitive documentation.
>
> And any documentation at all?

Not even that.  I tried to find something today, but nothing useful
turned up.

>
> Anyway, given the problems usually related to querying terminals about
> potentially unsupported features, and the general mess in this field,
> I think our best bet is to have a function that could switch the
> frame-update hooks between these 3 states:
>
>   . unused
>   . used with P1 BSU/ESU sequences
>   . used with P2 BSU/ESU sequences
>
> WDYT?

Yes, something like that, I guess.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 2241 bytes --]

I agree with Gerd on the discoveries.

Here's some code how tmux works with that [1], we can probably avoid it by
providing a frame-local flag which enables the functionality, so that
multiple emacs clients might be connected from potentially different
terminals.

We can improve the default value of the flag based on terminal capabilities
later once we have confidence in the way it works and probably extend it to
support different specifications of syncing.

[1]:
https://github.com/tmux/tmux/blob/9c34aad21c0837123a51a5a4233a016805d3e526/tty-features.c#L474

On Thu, Sep 8, 2022 at 2:35 AM Gerd Möllmann <gerd.moellmann@gmail.com>
wrote:

> Eli Zaretskii <eliz@gnu.org> writes:
>
> > I guess by P1 you mean these two sequences:
> >
> >   BSU: ESC P = 1 s ESC \
> >   ESU: ESC P = 2 s ESC \
> >
> > whereas by P2 you mean these:
> >
> >   DECRQM: ESC [ ? 2026 $ p
> >   DECRPM: ESC [ ? 2026 ; n $ y  (with n = 0..4, and 0 or 4 means "no
> support")
> >   BSU:    ESC [ ? 2026 h
> >   ESU:    ESC [ ? 2026 l
>
> Yes, that looks like them.
>
> >
> >> > Do you happen to know where is the definitive documentation of Sync?
> >>
> >> Sorry, I don't know a definitive documentation.
> >
> > And any documentation at all?
>
> Not even that.  I tried to find something today, but nothing useful
> turned up.
>
> >
> > Anyway, given the problems usually related to querying terminals about
> > potentially unsupported features, and the general mess in this field,
> > I think our best bet is to have a function that could switch the
> > frame-update hooks between these 3 states:
> >
> >   . unused
> >   . used with P1 BSU/ESU sequences
> >   . used with P2 BSU/ESU sequences
> >
> > WDYT?
>
> Yes, something like that, I guess.
>


-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 4332 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 2956 bytes --]

If everyone is OK with adding a new frame-local elisp flag, then I can
prepare a patch.

On Thu, Sep 8, 2022 at 8:59 AM Dmitrii Kuragin <kuragin@google.com> wrote:

> I agree with Gerd on the discoveries.
>
> Here's some code how tmux works with that [1], we can probably avoid it by
> providing a frame-local flag which enables the functionality, so that
> multiple emacs clients might be connected from potentially different
> terminals.
>
> We can improve the default value of the flag based on terminal
> capabilities later once we have confidence in the way it works and probably
> extend it to support different specifications of syncing.
>
> [1]:
> https://github.com/tmux/tmux/blob/9c34aad21c0837123a51a5a4233a016805d3e526/tty-features.c#L474
>
> On Thu, Sep 8, 2022 at 2:35 AM Gerd Möllmann <gerd.moellmann@gmail.com>
> wrote:
>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>> > I guess by P1 you mean these two sequences:
>> >
>> >   BSU: ESC P = 1 s ESC \
>> >   ESU: ESC P = 2 s ESC \
>> >
>> > whereas by P2 you mean these:
>> >
>> >   DECRQM: ESC [ ? 2026 $ p
>> >   DECRPM: ESC [ ? 2026 ; n $ y  (with n = 0..4, and 0 or 4 means "no
>> support")
>> >   BSU:    ESC [ ? 2026 h
>> >   ESU:    ESC [ ? 2026 l
>>
>> Yes, that looks like them.
>>
>> >
>> >> > Do you happen to know where is the definitive documentation of Sync?
>> >>
>> >> Sorry, I don't know a definitive documentation.
>> >
>> > And any documentation at all?
>>
>> Not even that.  I tried to find something today, but nothing useful
>> turned up.
>>
>> >
>> > Anyway, given the problems usually related to querying terminals about
>> > potentially unsupported features, and the general mess in this field,
>> > I think our best bet is to have a function that could switch the
>> > frame-update hooks between these 3 states:
>> >
>> >   . unused
>> >   . used with P1 BSU/ESU sequences
>> >   . used with P2 BSU/ESU sequences
>> >
>> > WDYT?
>>
>> Yes, something like that, I guess.
>>
>
>
> --
> *If you get an email from me outside of the 9-5 it is *not* because I'm
> always on or expect an immediate response from you; it is because of work
> flexibility
> <http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
> .  Evening and weekend emails are a sign I allocated some regular working
> hours for other things (such as family, gym, friends,...).  And I encourage
> you to feel free to do the same.
>
>

-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 6192 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Dmitrii Kuragin <kuragin@google.com>
> Date: Fri, 9 Sep 2022 08:48:57 -0700
> Cc: Eli Zaretskii <eliz@gnu.org>, 57434@debbugs.gnu.org
> 
> If everyone is OK with adding a new frame-local elisp flag, then I can prepare a patch.

I think the conclusion in the message you quoted was that we will need
a function, not just a variable.  That's because we need to allow two
different protocols for this, and we should allow switching between
the protocols.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Dmitrii Kuragin via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1469 bytes --]

Sorry for the late reply.

But could you please elaborate a bit more on what I can do now?

Do we want to add a hook like `begin_frame_update` or we need to add a
`sync_update_begin_escape_code` or we just say, sync_update_protocol (we
have 2 of those now).

Alternatively, we can allow users to specify it as a terminal capability
and let them override it. Like, `(setq xterm-extra-capabilities (quote
(modifyOtherKeys setSelection)))`

Which way would we like to proceed?

On Fri, Sep 9, 2022 at 9:00 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Dmitrii Kuragin <kuragin@google.com>
> > Date: Fri, 9 Sep 2022 08:48:57 -0700
> > Cc: Eli Zaretskii <eliz@gnu.org>, 57434@debbugs.gnu.org
> >
> > If everyone is OK with adding a new frame-local elisp flag, then I can
> prepare a patch.
>
> I think the conclusion in the message you quoted was that we will need
> a function, not just a variable.  That's because we need to allow two
> different protocols for this, and we should allow switching between
> the protocols.
>


-- 
*If you get an email from me outside of the 9-5 it is *not* because I'm
always on or expect an immediate response from you; it is because of work
flexibility
<http://www.inc.com/john-boitnott/how-flexible-hours-can-create-a-better-work-life-balance.html>
.  Evening and weekend emails are a sign I allocated some regular working
hours for other things (such as family, gym, friends,...).  And I encourage
you to feel free to do the same.

[-- Attachment #2: Type: text/html, Size: 3580 bytes --]

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Dmitrii Kuragin <kuragin@google.com>
> Date: Tue, 20 Sep 2022 09:35:41 -0700
> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>, 
> 	57434@debbugs.gnu.org
> 
> But could you please elaborate a bit more on what I can do now?
> 
> Do we want to add a hook like `begin_frame_update` or we need to add a
> `sync_update_begin_escape_code` or we just say, sync_update_protocol (we have 2 of those now).

I think we want to add begin/end_frame_update hooks, and we want them
to send the escape sequences that are determined by some state
variable which tells us which of the 2 protocols to use.  We then need
a function to allow changing that state variable, perhaps by an
explicit user command.

Bonus points for making the state variable be terminal-specific, so
that the same Emacs session could have TTY frames on several different
types of terminal, and use the correct protocol for each one of them.

I hope I explained this well enough; if not, feel free to ask specific
questions.

> Alternatively, we can allow users to specify it as a terminal capability and let them override it. Like, `(setq
> xterm-extra-capabilities (quote (modifyOtherKeys setSelection)))`

No, that's too dangerous, and not really needed.  It's not like there
are many different protocols for synchronized update.

Thanks.

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Dmitrii Kuragin <kuragin@google.com>
>> Date: Tue, 20 Sep 2022 09:35:41 -0700
>> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>, 
>> 	57434@debbugs.gnu.org
>> 
>> But could you please elaborate a bit more on what I can do now?
>> 
>> Do we want to add a hook like `begin_frame_update` or we need to add a
>> `sync_update_begin_escape_code` or we just say, sync_update_protocol (we have 2 of those now).
>
> I think we want to add begin/end_frame_update hooks, and we want them
> to send the escape sequences that are determined by some state
> variable which tells us which of the 2 protocols to use.  We then need
> a function to allow changing that state variable, perhaps by an
> explicit user command.
>
> Bonus points for making the state variable be terminal-specific, so
> that the same Emacs session could have TTY frames on several different
> types of terminal, and use the correct protocol for each one of them.

Yes.

I'm afraid I've lost the thread a bit, but ISTR that was what we arrived
at.  Plus something had to be changed with regard to fflush at the end
of the update (it was called too early, or too late).

bug#57434: 28.1.91; Terminal Emacs Mac OS flickering.

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: Dmitrii Kuragin <kuragin@google.com>,  57434@debbugs.gnu.org
> Date: Wed, 21 Sep 2022 08:10:02 +0200
> 
> Plus something had to be changed with regard to fflush at the end of
> the update (it was called too early, or too late).

Too late.  We should call it before the call to update_end.