Emacs in a terminal via qrsh on Sun Grid Engine - text rendering

Emacs in a terminal via qrsh on Sun Grid Engine - text rendering

[Seb James]

Hello Emacs folks,

I have a problem with emacs used within a terminal on my university's
high performance computing (HPC) system, which runs Scientific Linux
with GNU Emacs 23.1.1(*)

The problem is incorrectly rendered text, such as might be seen if the
terminal settings were incorrectly matched between the client terminal
and the user's environment on the server. The text can be corrected with
a C-l, as reported by other users who have had mis-rendered text
problems. The sort of misrendering is very similar to that reported
here, although the cause is different there:

http://emacs.stackexchange.com/questions/9512/why-does-the-buffer-get-garbled

Our HPC system runs Sun's Grid Engine to manage all the worker nodes.
Here is the workflow: I ssh on to the head node, then to do any work, I
am required to request a session on one of the worker nodes. To do this
I use the Sun Grid Engine (SGE) binary "qrsh" which opens a remote shell
on a host of SGE's choosing. If I now emacs -nw somefile.cpp, I quicly
see mis-rendering of the text.

Recently emacs became available on the head node, and I see no such
problem when using emacs in the terminal on this system. Therefore, the
problem seems to occur only when using qrsh to get to a worker node.

I have studied my TERM environment variable. On my Linux client system
it is xterm-256color, on the head node it is also xterm-256color and on
the worker node the same.

I've looked at stty output, here are the changed values on the worker
node:

[me@hpcnode03 ~]$ stty 
speed 38400 baud; line = 0;
-brkint -imaxbel
-onlcr

And on the head node:

[me@myhpc-login1 ~]$ stty
speed 38400 baud; line = 0;
eol = M-^?; eol2 = M-^?;
-brkint ixany

I forget the details, but I did tinker with stty at one point, changing
numerous settings without finding one that fixed the problem.

I encounter the problem regardless of which client terminal I use to
connect to the HPC system (via ssh on Debian or Ubuntu or putty on
Windows and I think I tried ssh from Mac, too).

Does anyone have any suggestions about how to debug this? Of course, it
may be a bug in Sun Grid Engine's qrsh program. Does anyone else on the
list use emacs via Sun Grid Engine? Seen this issue?

Any help much appreciated, even if it's just a confirmation that I'd
have to take this up with the maintainers of Sun Grid Engine (I have
tried our HPC system support team, but they've not been able to figure
out a solution).

best regards,

Seb James

(*) GNU Emacs 23.1.1 (x86_64-redhat-linux-gnu, GTK+ Version 2.18.9) of
2013-07-17 on sl6.fnal.gov.

Re: Emacs in a terminal via qrsh on Sun Grid Engine - text rendering

[Javier]

Seb James <seb@esfnet.co.uk> wrote:
> I have a problem with emacs used within a terminal on my university's
> high performance computing (HPC) system, which runs Scientific Linux
> with GNU Emacs 23.1.1(*)

It might be a bug in emacs 23, which is very old.  Scientific
linux installations in clusters always have prehistoric versions
of software.  The best is that you get a modern emacs and
build it as a user

wget http://ftp.gnu.org/gnu/emacs/emacs-24.5.tar.gz
tar xzvf emacs-24.5.tar.gz
cd emacs-24.5
./configure --without-x  --without-sound  --prefix=~/local ; make ; make install

If you will always be running it inside a terminal emulator you can
use --without-x and have a smaller executable

Then you can run your up to date emacs with

~/local/bin/emacs

Re: Emacs in a terminal via qrsh on Sun Grid Engine - text rendering

[Seb James]

On Tue, 2015-10-13 at 19:22 +0000, Javier wrote:
> Seb James <seb@esfnet.co.uk> wrote:
> > I have a problem with emacs used within a terminal on my university's
> > high performance computing (HPC) system, which runs Scientific Linux
> > with GNU Emacs 23.1.1(*)
> 
> It might be a bug in emacs 23, which is very old.  Scientific
> linux installations in clusters always have prehistoric versions
> of software.  The best is that you get a modern emacs and
> build it as a user

Thanks Javier,

I've done that and the problem persists :(

Seb

Re: Emacs in a terminal via qrsh on Sun Grid Engine - text rendering

[Javier]

> I've done that and the problem persists :(

It might be that the colors of the emacs font faces in the buffer and
statusbar are the problem.  Try

TERM=xterm emacs -nw

TERM=xterm instead of TERM=xterm-256color will do the colorizing of
the screen slightly differently.

If that doesn't work you can try to remove color completely

emacs -nw --color=no

or this in your ~/.emacs

(add-to-list 'default-frame-alist '(tty-color-mode  . -1))

Re: Emacs in a terminal via qrsh on Sun Grid Engine - text rendering

[Eli Zaretskii]

> From: Javier <nospam@nospam.com>
> Date: Wed, 14 Oct 2015 17:47:36 +0000 (UTC)
> 
> > I've done that and the problem persists :(
> 
> It might be that the colors of the emacs font faces in the buffer and
> statusbar are the problem.  Try
> 
> TERM=xterm emacs -nw

Emacs on a TTY doesn't use any fonts.

If you mean the colors are the problem, then I don't understand how
could that be.  Are you saying that sending a command to the terminal
to switch on or off some color causes the cursor to move?  That can
happen only if the terminal driver is buggy, or if the value of TERM
lies to Emacs wrt the real features of the terminal (i.e. some problem
in the termcap/terminfo database).

Re: Emacs in a terminal via qrsh on Sun Grid Engine - text rendering

[Javier]

Eli Zaretskii <eliz@gnu.org> wrote:
> Are you saying that sending a command to the terminal
> to switch on or off some color causes the cursor to move?  That can
> happen only if the terminal driver is buggy, or if the value of TERM
> lies to Emacs wrt the real features of the terminal (i.e. some problem
> in the termcap/terminfo database).

To be honest I don't know where the problem can be.  To isolate the
source of the problem I would try

TERM=vt100 emacs -nw

If that is still faulty I would run vttest (option 2 and keep pressing Enter)
and compare it with what xterm gives in the local computer.

https://en.wikipedia.org/wiki/Vttest
http://invisible-island.net/vttest/

Also, which terminal application the OP is using?  konsole?
gnome-terminal?  Did he try running xterm?

Re: Emacs in a terminal via qrsh on Sun Grid Engine - text rendering

[Bob Proulx]

Seb James wrote:
> The problem is incorrectly rendered text, such as might be seen if the
> terminal settings were incorrectly matched between the client terminal
> and the user's environment on the server. The text can be corrected with
> a C-l, as reported by other users who have had mis-rendered text
> problems. The sort of misrendering is very similar to that reported
> here, although the cause is different there:
> 
> http://emacs.stackexchange.com/questions/9512/why-does-the-buffer-get-garbled

Garbled text on a text terminal today using TCP/IP connections I think
is mostly likely caused by a mismatched termcap / terminfo database.
Perhaps even an incorrect and buggy termcap / terminfo database.  The
side running emacs thinks the terminal is one way and the terminal
emulator thinks it is another way.

The garbled output tends to appear when incremental updates are
applied to the terminal.  The cursor is told to move to an x,y
location on the screen.  Something is diddled there.  Move.  Diddle.
Move diddle.  But the screen gets garbled when the terminal does not
actually do what the update intended.

This gets fixed with a C-l because C-l was designed in the days of
modem noise which would cause the screen to be modified in ways
unknown to the program.  It is designed to clean up after this type of
problem.  It will clear the screen to get to a known empty state and
paint top to bottom a clean display output of what should be there.
That cleans up all of the garbled output.

> Our HPC system runs Sun's Grid Engine to manage all the worker nodes.
> Here is the workflow: I ssh on to the head node, then to do any work, I
> am required to request a session on one of the worker nodes. To do this
> I use the Sun Grid Engine (SGE) binary "qrsh" which opens a remote shell
> on a host of SGE's choosing. If I now emacs -nw somefile.cpp, I quicly
> see mis-rendering of the text.

After you have grabbed a node can you then ssh directly to that node
for a test?  After grabbing a node with qrsh in parallel with it ssh
into the node using a normal ssh login outside of SGE.  Does the
problem exist that way too?

If the problem does not exist then the issue is something in the sge
qrsh environment.  Having a working environment should allow you to do
A-B comparisons between one and the other and quickly figure out what
is different.

If the problem exists with the direct connection then the issue is not
related to SGE qrsh and is simply an environment issue.  (I still
suspect terminfo / termcap problems.)

> Recently emacs became available on the head node, and I see no such
> problem when using emacs in the terminal on this system. Therefore, the
> problem seems to occur only when using qrsh to get to a worker node.
>
> I have studied my TERM environment variable. On my Linux client system
> it is xterm-256color, on the head node it is also xterm-256color and on
> the worker node the same.

I would be inclined to examine closely any differences between
versions on head and versions in the cluster.  Specifically terminfo
down the /usr/share/terminfo tree.  I am really suspecting that
terminfo is different between the systems.

> I've looked at stty output, here are the changed values on the worker
> node:
> 
> [me@hpcnode03 ~]$ stty 
> speed 38400 baud; line = 0;
> -brkint -imaxbel
> -onlcr
> 
> And on the head node:
> 
> [me@myhpc-login1 ~]$ stty
> speed 38400 baud; line = 0;
> eol = M-^?; eol2 = M-^?;
> -brkint ixany

This leads me to think that the software versions are not identical
between the head node and the rest of the cluster.

You can make the tty settings identical manually by cutting and
pasting the output from one into the other.

Dump it from one:

  $ stty -g
    4500:5:bf:8a3b:3:1c:7f:15:4:0:1:0:11:13:1a:0:12:f:17:16:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0

And past it into the other.

  $ stty 4500:5:bf:8a3b:3:1c:7f:15:4:0:1:0:11:13:1a:0:12:f:17:16:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0

That will make them identical among compatible systems.

> I encounter the problem regardless of which client terminal I use to
> connect to the HPC system (via ssh on Debian or Ubuntu or putty on
> Windows and I think I tried ssh from Mac, too).
> 
> Does anyone have any suggestions about how to debug this? Of course, it
> may be a bug in Sun Grid Engine's qrsh program. Does anyone else on the
> list use emacs via Sun Grid Engine? Seen this issue?

It has been several years since I have used SGE.  But I am familiar
with it and some other queue systems.

Do you have any TERMCAP or TERMINFO variables set in your environment?
https://en.wikipedia.org/wiki/Termcap#Environment_variables

  env | grep TERM

An environment variable would (could if set) override the system
definition.  This could be a cause of a problem.  It might also allow
you to easily override a system if the system were incorrect.  I have
often over the years run into incorrect terminfo databases.

Bob