bug#45042: 27.1; emacs -nw on Windows does not always update cursor position on screen

bug#45042: 27.1; emacs -nw on Windows does not always update cursor position on screen

[Ed Avis]

[-- Attachment #1: Type: text/plain, Size: 5854 bytes --]

I installed the GNU Emacs build for Microsoft Windows x86_64 from the
GNU FTP site.  Rather than the graphical interface I am running it in
text mode using 'emacs.exe -nw'.  If run from the ancient Windows
command prompt (cmd.exe) this works well.  But in other terminal
emulators there is a strange problem where the cursor position is not
always updated.

For example, run Emacs inside the Alacritty terminal emulator.  Moving
point up and down with the arrow keys (or C-p and C-n) works, but moving
left and right (with arrows, C-b, C-f) does not update the visible
cursor on screen.  Point did move, as you can see by typing some
characters, and the visible cursor will update when you move up or down,
but horizontal movement doesn't seem to display.

If this were just one terminal emulator I would not suggest it as a bug
in Emacs, but the same thing happens when running an ssh server on the
Windows machine and running emacs -nw over ssh from another Windows
machine.  The cursor movement problem happens whether the ssh client is
OpenSSH (ssh.exe) running in cmd.exe, or ssh.exe running in Alacritty,
or from PuTTY (which combines ssh client and terminal emulator).

Quick experiments with setting TERM to 'vt100' instead of 'xterm' did
not find a fix.  Is there something Emacs can do to make sure the cursor
position on screen is always updated?


In GNU Emacs 27.1 (build 1, x86_64-w64-mingw32)
 of 2020-08-21 built on CIRROCUMULUS
Repository revision: 86d8d76aa36037184db0b2897c434cdaab1a9ae8
Repository branch: HEAD
Windowing system distributor 'Microsoft Corp.', version 10.0.18363
System Description: Microsoft Windows 10 Pro (v10.0.1909.18363.1198)

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
 'configure --without-dbus --host=x86_64-w64-mingw32
 --without-compress-install 'CFLAGS=-O2 -static''

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND NOTIFY W32NOTIFY ACL GNUTLS LIBXML2
HARFBUZZ ZLIB TOOLKIT_SCROLL_BARS MODULES THREADS JSON PDUMPER LCMS2 GMP

Important settings:
  value of $LANG: ENG
  locale-coding-system: cp1252

Major mode: Fundamental

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs text-property-search time-date
subr-x seq byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs
cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
term/w32console tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel dos-w32 ls-lisp disp-table term/w32-win w32-win
w32-vars term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads w32notify w32 lcms2 multi-tty make-network-process
emacs)

Memory information:
((conses 16 43590 9530)
 (symbols 48 6020 1)
 (strings 32 15201 1632)
 (string-bytes 1 502900)
 (vectors 16 6718)
 (vector-slots 8 78458 6176)
 (floats 8 18 317)
 (intervals 56 187 0)
 (buffers 1000 12))


This email and any files transmitted with it are CONFIDENTIAL and are intended solely for the use of the individual(s) or entity to whom they are addressed. Any unauthorized copying, disclosure or distribution of the material within this email is strictly forbidden. Any views or opinions presented within this email are solely those of the author and do not necessarily represent those of QMA Wadhwani LLP (QMAW) unless otherwise specifically stated. An electronic message is not binding on its sender. Any message referring to a binding agreement must be confirmed in writing and duly signed. If you have received this email in error, please notify the sender immediately and delete the original. Telephone, electronic and other communications and conversations with QMAW and/or its associated persons may be recorded and retained. QMAW is authorized and regulated by the Financial Conduct Authority. QMAW (registered in England No. OC303168) has its registered office at 9th Floor Orion House, 5 Upper St Martin's Lane, London, WC2H 9EA.

Please note that your personal information may be stored and processed in any country where we have facilities or in which we engage service providers. If you provide personal information to us by email or otherwise, you consent to the transfer of that information to countries outside of your country of residence and these countries may have different data protection rules than your country.'

To learn about our privacy policies, please use this link<https://www.pgim.com/disclaimer/privacy-center> to read the PGIM Privacy Notice.

[-- Attachment #2: Type: text/html, Size: 7548 bytes --]

bug#45042: 27.1; emacs -nw on Windows does not always update cursor position on screen

[Eli Zaretskii]

> From: Ed Avis <ed.avis@qmaw.com>
> Date: Fri, 4 Dec 2020 18:53:28 +0000
> 
> Is there something Emacs can do to make sure the cursor
> position on screen is always updated?

Someone who has access to such systems will have debug the functions
in w32console.c and see what isn't working in those cases.  As you can
see, we just use SetConsoleCursorPosition, an API that is supposed to
be agnostic to the terminal emulator running, as long as it emulates
the w32 console well enough.

Or maybe these terminal emulators set TERM to some value that dupes
Emacs into incorrect operation?  If so, try unsetting TERM.

bug#45042: 27.1; emacs -nw on Windows does not always update cursor position on screen

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> Is there something Emacs can do to make sure the cursor
>> position on screen is always updated?
>
> Someone who has access to such systems will have debug the functions
> in w32console.c and see what isn't working in those cases.  As you can
> see, we just use SetConsoleCursorPosition, an API that is supposed to
> be agnostic to the terminal emulator running, as long as it emulates
> the w32 console well enough.
>
> Or maybe these terminal emulators set TERM to some value that dupes
> Emacs into incorrect operation?  If so, try unsetting TERM.

Ed, did you try unsetting TERM?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#45042: 27.1; emacs -nw on Windows does not always update cursor position on screen

[Ed Avis]

The problem still happens when I start the Alacritty terminal emulator on Windows, and

>set TERM=
>emacs -nw

(I think on Windows there is no difference between setting TERM to an empty string and unsetting it.)

--
Ed Avis <ed.avis@qmaw.com>
Please ignore confidentiality stuff below this point.

This email and any files transmitted with it are CONFIDENTIAL and are intended solely for the use of the individual(s) or entity to whom they are addressed. Any unauthorized copying, disclosure or distribution of the material within this email is strictly forbidden. Any views or opinions presented within this email are solely those of the author and do not necessarily represent those of QMA Wadhwani LLP (QMAW) unless otherwise specifically stated. An electronic message is not binding on its sender. Any message referring to a binding agreement must be confirmed in writing and duly signed. If you have received this email in error, please notify the sender immediately and delete the original. Telephone, electronic and other communications and conversations with QMAW and/or its associated persons may be recorded and retained. QMAW is authorized and regulated by the Financial Conduct Authority. QMAW (registered in England No. OC303168) has its registered office at 9th Floor Orion House, 5 Upper St Martin's Lane, London, WC2H 9EA.

Please note that your personal information may be stored and processed in any country where we have facilities or in which we engage service providers. If you provide personal information to us by email or otherwise, you consent to the transfer of that information to countries outside of your country of residence and these countries may have different data protection rules than your country.’

To learn about our privacy policies, please use this link<https://www.pgim.com/disclaimer/privacy-center> to read the PGIM Privacy Notice.

bug#45042: 27.1; emacs -nw on Windows does not always update cursor position on screen

[Eli Zaretskii]

> From: Ed Avis <ed.avis@qmaw.com>
> CC: "45042@debbugs.gnu.org" <45042@debbugs.gnu.org>
> Date: Sun, 24 Jan 2021 07:22:26 +0000
> 
> The problem still happens when I start the Alacritty terminal emulator on Windows, and
> 
> >set TERM=
> >emacs -nw

I see quite a few problems reported against Alacritty, so maybe this
is another one.

In any case, as I said: someone who has access to Alacritty will have
to debug this and find out what goes wrong.

> (I think on Windows there is no difference between setting TERM to an empty string and unsetting it.)

There is a difference, but cmd.exe doesn't let you do the former.

bug#45042: 27.1; emacs -nw on Windows does not always update cursor position on screen

[Ed Avis]

[-- Attachment #1: Type: text/plain, Size: 1957 bytes --]

The problem with the cursor movement not updating doesn't just happen under Alacritty.  I can also reproduce it using an ssh server on Windows, by sshing to the machine and running emacs -nw.  The ssh client can be PuTTY or the Windows port of OpenSSH running in the Windows command prompt.  Unsetting TERM does not appear to make a difference in any of these cases.



This email and any files transmitted with it are CONFIDENTIAL and are intended solely for the use of the individual(s) or entity to whom they are addressed. Any unauthorized copying, disclosure or distribution of the material within this email is strictly forbidden. Any views or opinions presented within this email are solely those of the author and do not necessarily represent those of QMA Wadhwani LLP (QMAW) unless otherwise specifically stated. An electronic message is not binding on its sender. Any message referring to a binding agreement must be confirmed in writing and duly signed. If you have received this email in error, please notify the sender immediately and delete the original. Telephone, electronic and other communications and conversations with QMAW and/or its associated persons may be recorded and retained. QMAW is authorized and regulated by the Financial Conduct Authority. QMAW (registered in England No. OC303168) has its registered office at 9th Floor Orion House, 5 Upper St Martin's Lane, London, WC2H 9EA.

Please note that your personal information may be stored and processed in any country where we have facilities or in which we engage service providers. If you provide personal information to us by email or otherwise, you consent to the transfer of that information to countries outside of your country of residence and these countries may have different data protection rules than your country.'

To learn about our privacy policies, please use this link<https://www.pgim.com/disclaimer/privacy-center> to read the PGIM Privacy Notice.

[-- Attachment #2: Type: text/html, Size: 2607 bytes --]

bug#45042: 27.1; emacs -nw on Windows does not always update cursor position on screen

[Eli Zaretskii]

> From: Ed Avis <ed.avis@qmaw.com>
> CC: "larsi@gnus.org" <larsi@gnus.org>,
>  "45042@debbugs.gnu.org"<45042@debbugs.gnu.org>
> Date: Mon, 25 Jan 2021 08:00:00 +0000
> 
> The problem with the cursor movement not updating doesn't just happen under Alacritty.  I can also
> reproduce it using an ssh server on Windows, by sshing to the machine and running emacs -nw.  The ssh
> client can be PuTTY or the Windows port of OpenSSH running in the Windows command prompt.  Unsetting
> TERM does not appear to make a difference in any of these cases.

PuTTY emulates a Unix terminal with terminfo database, and thus
expects the program to send escape sequences to move the cursor.  If
the OpenSSH client used by Windows does the same, then it means these
two emulators aren't currently supported by Emacs on Windows, because
Emacs assumes a native Windows console.

Patches to add support for curses-like escape sequences are welcome.

bug#45042: 27.1; emacs -nw on Windows does not always update cursor position on screen

[Ed Avis]

[-- Attachment #1: Type: text/plain, Size: 2093 bytes --]

Thanks for the explanation.  So although Emacs uses escape sequences to move the cursor on Unix-like platforms, the Windows build uses the native console API.  What surprised me was that the cursor position does update for up and down movements, and it does update horizontally at the same time.  It just doesn't move if you do a horizontal movement by itself.  If the cursor movement commands aren't supported by PuTTY or by OpenSSH server+client, I would expect to see neither kind of movement working.


This email and any files transmitted with it are CONFIDENTIAL and are intended solely for the use of the individual(s) or entity to whom they are addressed. Any unauthorized copying, disclosure or distribution of the material within this email is strictly forbidden. Any views or opinions presented within this email are solely those of the author and do not necessarily represent those of QMA Wadhwani LLP (QMAW) unless otherwise specifically stated. An electronic message is not binding on its sender. Any message referring to a binding agreement must be confirmed in writing and duly signed. If you have received this email in error, please notify the sender immediately and delete the original. Telephone, electronic and other communications and conversations with QMAW and/or its associated persons may be recorded and retained. QMAW is authorized and regulated by the Financial Conduct Authority. QMAW (registered in England No. OC303168) has its registered office at 9th Floor Orion House, 5 Upper St Martin's Lane, London, WC2H 9EA.

Please note that your personal information may be stored and processed in any country where we have facilities or in which we engage service providers. If you provide personal information to us by email or otherwise, you consent to the transfer of that information to countries outside of your country of residence and these countries may have different data protection rules than your country.'

To learn about our privacy policies, please use this link<https://www.pgim.com/disclaimer/privacy-center> to read the PGIM Privacy Notice.

[-- Attachment #2: Type: text/html, Size: 2606 bytes --]

bug#45042: 27.1; emacs -nw on Windows does not always update cursor position on screen

[Eli Zaretskii]

> From: Ed Avis <ed.avis@qmaw.com>
> CC: "larsi@gnus.org" <larsi@gnus.org>,
>  "45042@debbugs.gnu.org"<45042@debbugs.gnu.org>
> Date: Tue, 26 Jan 2021 07:21:27 +0000
> 
> Thanks for the explanation.  So although Emacs uses escape sequences to move the cursor on Unix-like
> platforms, the Windows build uses the native console API.  What surprised me was that the cursor position
> does update for up and down movements, and it does update horizontally at the same time.  It just doesn't
> move if you do a horizontal movement by itself.  If the cursor movement commands aren't supported by
> PuTTY or by OpenSSH server+client, I would expect to see neither kind of movement working.

I don't know the answer, but I'm quite sure that if you look at the
sources of the respective packages, you will see the reason.

It is also possible that in cases where it worked Emacs decided to
move the cursor by redrawing the following or the preceding character,
instead of calling the console API functions.  (These are redisplay
optimizations Emacs sometimes does.)