GDB over TRAMP: problem with user I/O

GDB over TRAMP: problem with user I/O

[Vladilen Kozin]

Ran into problem with GDB over TRAMP. To me it looks like a bug in TRAMP
but could be that I've missed some settings required for such setup. Hence
posting here instead of bugs for now.

Setup is described in more details below, but in a nutshell:
- target machine with code to debug runs a vanilla Linux,
- Emacs on the host machine starts multi window GDB session over TRAMP,
works fine,
- excep I/O buffer gets intercepted by (probably) remote shell, so user
input can't be passed to the process being debugged (see below).

Source code being debugged
--------------------------
#include <stdio.h>
#include <stdlib.h>

int main()
{
     int num;
     printf ("Enter the number: ");
     scanf ("%d", &num );

     printf("Your number %d\n", num);
     exit(0);
}

Makefile executed on Target
---------------------------
CFLAGS = -std=c99 -pedantic -Wall -O2 -fPIC
DEBUG = -g -std=c99 -pedantic -Wall -O0 -fPIC

debug: fact.c
$(CC) $(DEBUG) $(LDFLAGS) -o $@ $< $(LDLIBS)

clean:
$(RM) -rf *.o *.so fact debug debug.dSYM


Remote debug session started from Host
--------------------------------------
M-x gdb
gdb -i=mi /fact:/vagrant/debug

Relevant GDB windows:

*gud-debug*
-----------
Reading symbols from /vagrant/debug...done.
(gdb) run
Starting program: /vagrant/debug

*input/output of debug*
-----------------------
&"warning: GDB: Failed to set controlling terminal: Operation not
permitted\n"
Enter the number: 3
/bin/sh: 22: 3: not found
///fe8a2e50a45c66d598f9ae7a276764a2#$

Notice how the input prompt in IO buffer appears fine and awaits for user
entry, but said entry gets snatched by /bin/sh and wouldn't let me send it
to the process debugged.

Note that running the same debug session locally works out fine, user
input doesn't get stolen.

I managed to reproduce this with absolutely barebones setup.

Emacs Host
----------
GNU Emacs 25.3.1 (x86_64-apple-darwin17.2.0, NS appkit-1561.10
Version 10.13.1 (Build 17B48)) of 2017-11-12

Debug Target
------------
vagrant@precise64:/vagrant$ uname -a
Linux precise64 3.2.0-23-generic #36-Ubuntu SMP Tue Apr 10
20:39:51 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

init.el
-------
(require 'gdb-mi)
(require 'tramp)

(setq gdb-many-windows t
      gdb-show-main t)

(setq tramp-default-method "ssh"
      tramp-default-user "vagrant")

(require 'package)
(add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/") t)
(package-initialize)
(unless (file-exists-p (concat user-emacs-directory "elpa/archives/melpa"))
  (package-refresh-contents))

(defvar required-packages '(exec-path-from-shell))

(let ((install #'(lambda (package)
                   (unless (package-installed-p package)
                     (package-install package))
                   (require package))))
  (message "Installing required packages %s" required-packages)
  (mapc install required-packages)
  (delete-other-windows))

(exec-path-from-shell-initialize)


Setting tramp-debug-on-error to true, doesn't show anything.

Could anyone help me figure out how to fix this? Or should I post to the
bug list?

Thanks

-- 
Best regards
Vlad Kozin

Re: GDB over TRAMP: problem with user I/O

[Michael Albinus]

Vladilen Kozin <vladilen.kozin@gmail.com> writes:

Hi Vladilen,

> Ran into problem with GDB over TRAMP. To me it looks like a bug in TRAMP
> but could be that I've missed some settings required for such setup. Hence
> posting here instead of bugs for now.

First of all, I could reproduce it. But I'm not sure it is a Tramp
error. Rather a missing feature.

I've tried to understand how gdb is working. Took a while, I'm not
familiar with that code. After gdb has been called, it starts to
asynchronous processes: gud-<program> (bound to buffer *gud-<program>*),
which is the gdb process itself, and gdb-inferior (bound to buffer
*input/output of <program>*), which does not run any program, but
allocates a tty only. You'll see this when calling "M-x list-processes".

--8<---------------cut here---------------start------------->8---
gdb-inferior    -2      run     *input/outpu... /dev/pts/4   
gud-debug       30367   run     *gud-debug*     /dev/pts/3   gdb -i=mi debug
--8<---------------cut here---------------end--------------->8---

Look at the local case: In the gdb-debug buffer, I've checked the used tty:

--8<---------------cut here---------------start------------->8---
(gdb) show inferior-tty
Terminal for future runs of program being debugged is "/dev/pts/4".
--8<---------------cut here---------------end--------------->8---

Starting the test program, and checking outside Emacs for pts/4, you'll
see that it is used by the gdb process:

--8<---------------cut here---------------start------------->8---
$ ps -eaf | grep pts/4
albinus  30405 30367  0 11:07 pts/4    00:00:00 /home/albinus/tmp/fact/debug
--8<---------------cut here---------------end--------------->8---

Now I have performed the remote case. For simplicity, I have just
prepended "/ssh::" to the test files, that's sufficient. The process
list looks different:

--8<---------------cut here---------------start------------->8---
*tramp/ssh d... 30451   run     *tramp/ssh d... /dev/pts/3   /bin/sh -i
gdb-inferior    30764   run     *input/outpu... /dev/pts/7   /bin/sh -i
gud-debug       30740   run     *gud-debug*     /dev/pts/5   /bin/sh -i
--8<---------------cut here---------------end--------------->8---

You see, that the gdb-inferior does not return just a tty. This is not
possible for remote processes. Instead, it is bound to /dev/pts/7, the
tty of the running remote shell.

In the gdb buffer, I've checked the used tty, again:

--8<---------------cut here---------------start------------->8---
(gdb) show inferior-tty
Terminal for future runs of program being debugged is "/dev/pts/8".
--8<---------------cut here---------------end--------------->8---

Starting the test program, you'll see a different situation:

--8<---------------cut here---------------start------------->8---
$ ps -eaf | grep pts/8
albinus  30567 30452  0 11:09 ?        00:00:00 sshd: albinus@pts/4,pts/6,pts/8
albinus  30765 30567  0 11:09 pts/8    00:00:00 /bin/sh
albinus  30927 26061  0 11:14 pts/9    00:00:00 grep pts/8
$ ps -eaf | grep fact
albinus  22347 22258  0 11:21 ?        00:00:00 /home/albinus/tmp/fact/debug
albinus  22377 31587  0 11:22 pts/0    00:00:00 grep fact
--8<---------------cut here---------------end--------------->8---

The test program, debug, has no tty. And gdb tries to communicate with
tty pts/8, which is the remote shell. Well, this cannot work. gdb tells
you this with the message &"warning: GDB: Failed to set controlling
terminal: Operation not permitted\n". Instead, your program is
communicating with the shell which owns /dev/pts/8. You will see, that
you could use the buffer *input/output of <program>* interactively.

I have no idea how to fix this. Handling remote ttys simply does not
work as gdb expects.

Best regards, Michael.

Re: GDB over TRAMP: problem with user I/O

[Michael Albinus]

Live System User <nyc4bos@aol.com> writes:

> Hi Michael,

Hi,

>> I have no idea how to fix this. Handling remote ttys simply does not
>> work as gdb expects.
>
>   Take a look at gdbserver  (part of the GDB gdb-gdbserver package
>   on Gnu/Linux).

Thanks for the hint. Unfortunately, while scanning its doc, I cannot see
how gdbserver could solve the problem at hand: redirecting the i/o of
the remote program being debugged to emacs.

Best regards, Michael.

Re: GDB over TRAMP: problem with user I/O

[Vladilen Kozin]

Hi Michael,

thank you very much for digging into it. I played with ttys, too, and
had somewhat similar experience perhaps with less understanding on my
part. Similar bug plagues a ton of external debugging tools that
interface with GDB: google spits out a ton of complaints in code
blocks, netbeans, etc that's been popping up at least since 2009. All
suggested "solutions" amount to cargoculting some nonsense that on
occasion works.

There's one bug reported to GDB bugtracker ages ago that remains open
and might be of note:
https://sourceware.org/bugzilla/show_bug.cgi?id=11403

I've also updated to the latest version of everything just to confirm
the issue is present in the latest GDB - it is. I'll see if I really
want to dig deeper there. The only reason I went with this remote
debugging setup is cause OSX has become notoriously hostile to GDB, so
thought I might spin a Linux box with Vagrant and run all my probes
this way with Emacs acting a thin client. Would've been cool. Still,
Tramp is incredible piece of software - I'm in awe.

Will report back if discover smth noteworthy.

Best


On 10 December 2017 at 10:19, Michael Albinus <michael.albinus@gmx.de> wrote:
> Vladilen Kozin <vladilen.kozin@gmail.com> writes:
>
> Hi Vladilen,
>
>> Ran into problem with GDB over TRAMP. To me it looks like a bug in TRAMP
>> but could be that I've missed some settings required for such setup. Hence
>> posting here instead of bugs for now.
>
> First of all, I could reproduce it. But I'm not sure it is a Tramp
> error. Rather a missing feature.
>
> I've tried to understand how gdb is working. Took a while, I'm not
> familiar with that code. After gdb has been called, it starts to
> asynchronous processes: gud-<program> (bound to buffer *gud-<program>*),
> which is the gdb process itself, and gdb-inferior (bound to buffer
> *input/output of <program>*), which does not run any program, but
> allocates a tty only. You'll see this when calling "M-x list-processes".
>
> --8<---------------cut here---------------start------------->8---
> gdb-inferior    -2      run     *input/outpu... /dev/pts/4
> gud-debug       30367   run     *gud-debug*     /dev/pts/3   gdb -i=mi debug
> --8<---------------cut here---------------end--------------->8---
>
> Look at the local case: In the gdb-debug buffer, I've checked the used tty:
>
> --8<---------------cut here---------------start------------->8---
> (gdb) show inferior-tty
> Terminal for future runs of program being debugged is "/dev/pts/4".
> --8<---------------cut here---------------end--------------->8---
>
> Starting the test program, and checking outside Emacs for pts/4, you'll
> see that it is used by the gdb process:
>
> --8<---------------cut here---------------start------------->8---
> $ ps -eaf | grep pts/4
> albinus  30405 30367  0 11:07 pts/4    00:00:00 /home/albinus/tmp/fact/debug
> --8<---------------cut here---------------end--------------->8---
>
> Now I have performed the remote case. For simplicity, I have just
> prepended "/ssh::" to the test files, that's sufficient. The process
> list looks different:
>
> --8<---------------cut here---------------start------------->8---
> *tramp/ssh d... 30451   run     *tramp/ssh d... /dev/pts/3   /bin/sh -i
> gdb-inferior    30764   run     *input/outpu... /dev/pts/7   /bin/sh -i
> gud-debug       30740   run     *gud-debug*     /dev/pts/5   /bin/sh -i
> --8<---------------cut here---------------end--------------->8---
>
> You see, that the gdb-inferior does not return just a tty. This is not
> possible for remote processes. Instead, it is bound to /dev/pts/7, the
> tty of the running remote shell.
>
> In the gdb buffer, I've checked the used tty, again:
>
> --8<---------------cut here---------------start------------->8---
> (gdb) show inferior-tty
> Terminal for future runs of program being debugged is "/dev/pts/8".
> --8<---------------cut here---------------end--------------->8---
>
> Starting the test program, you'll see a different situation:
>
> --8<---------------cut here---------------start------------->8---
> $ ps -eaf | grep pts/8
> albinus  30567 30452  0 11:09 ?        00:00:00 sshd: albinus@pts/4,pts/6,pts/8
> albinus  30765 30567  0 11:09 pts/8    00:00:00 /bin/sh
> albinus  30927 26061  0 11:14 pts/9    00:00:00 grep pts/8
> $ ps -eaf | grep fact
> albinus  22347 22258  0 11:21 ?        00:00:00 /home/albinus/tmp/fact/debug
> albinus  22377 31587  0 11:22 pts/0    00:00:00 grep fact
> --8<---------------cut here---------------end--------------->8---
>
> The test program, debug, has no tty. And gdb tries to communicate with
> tty pts/8, which is the remote shell. Well, this cannot work. gdb tells
> you this with the message &"warning: GDB: Failed to set controlling
> terminal: Operation not permitted\n". Instead, your program is
> communicating with the shell which owns /dev/pts/8. You will see, that
> you could use the buffer *input/output of <program>* interactively.
>
> I have no idea how to fix this. Handling remote ttys simply does not
> work as gdb expects.
>
> Best regards, Michael.



-- 
Best regards
Vlad Kozin