bug#13677: 24.2.50; workng on remote hosts with eshell is not very enjoyable

bug#13677: 24.2.50; workng on remote hosts with eshell is not very enjoyable

[Andreas Politz]

emacs -Q
M-x eshell
cd /ssh:somehost.com:
stat .
stat .
...

Every stat command starts a new tramp process, which, in my case, takes
about 3-4 seconds to get going and renders eshell fairly useless for
this kind of work. 

Can this be changed such that eshell uses start-file-process only if the
command results in a background process and use process-file otherwise ?

This seems to be a follow up to bug #3957.

Andreas




In GNU Emacs 24.2.50.1 (x86_64-pc-linux-gnu, GTK+ Version 2.20.1)
 of 2012-10-13 on cw-bkp0, modified by Debian
 (emacs-snapshot package, version 2:20121009-1+squeeze)
Windowing system distributor `The X.Org Foundation', version 11.0.10707000
System Description:	Debian GNU/Linux 6.0.6 (squeeze)

Configured using:
 `configure '--build' 'x86_64-linux-gnu' '--host' 'x86_64-linux-gnu'
 '--prefix=/usr' '--sharedstatedir=/var/lib' '--libexecdir=/usr/lib'
 '--localstatedir=/var' '--infodir=/usr/share/info'
 '--mandir=/usr/share/man' '--with-pop=yes'
 '--enable-locallisppath=/etc/emacs-snapshot:/etc/emacs:/usr/local/share/emacs/24.2.50/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/24.2.50/site-lisp:/usr/share/emacs/site-lisp'
 '--without-compress-info' '--with-crt-dir=/usr/lib//' '--with-x=yes'
 '--with-x-toolkit=gtk' '--with-imagemagick=yes'
 'build_alias=x86_64-linux-gnu' 'host_alias=x86_64-linux-gnu'
 'CFLAGS=-DDEBIAN -DSITELOAD_PURESIZE_EXTRA=5000 -g -O2' 'LDFLAGS=-g
 -Wl,--as-needed -znocombreloc' 'CPPFLAGS=''

Important settings:
  value of $LC_COLLATE: C
  value of $LC_MESSAGES: C
  value of $LANG: de_DE.UTF-8
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: EShell

Minor modes in effect:
  ispell-track-input-method: t
  idle-help-selwin-mode: t
  idle-help-mode: t
  tblc-mode: t
  emacs-interactive-mode: t
  show-paren-mode: t
  recentf-mode: t
  yas/global-mode: t
  window-numbering-mode: t
  shell-dirtrack-mode: t
  scroll-other-window-mode: t
  savehist-mode: t
  iswitchb-mode: t
  TeX-PDF-mode: t
  ekey-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ 
C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ 
C-/ C-/ C-M-x C-x o M-p <return> C-M-x g g <return> 
M-p <return> <return> M-p DEL <return> M-p <return> 
M-p M-h s l e e p SPC 1 m & <return> C-x m C-y C-/ 
C-n C-e C-y C-p C-y M-b M-b M-b DEL DEL DEL DEL @ C-e 
C-n C-n C-n C-n C-n <return> T h i s SPC s e e m s 
SPC t o SPC b e SPC M-! b z M-p <return> C-x o M-! 
M-p <return> C-x o M-h h a v e SPC b e e n SPC a d 
d r e s s e d SPC i n SPC M-b C-k f o r SPC s o m e 
M-b M-b M-b C-e SPC t i m e SPC n o w . M-q C-p C-p 
C-p C-p C-p C-p C-M-b C-M-b C-M-b C-M-b C-k C-n C-n 
C-n C-n C-n C-n C-a C-k C-x k <return> y M-x r e p 
o r t <return> b u g <return> e s h e l l SPC o <backspace> 
c o m m a n d s SPC o n SPC r e m o t e SPC h o s t 
s C-M-b C-M-b C-M-b C-M-b C-e SPC M-b M-b M-b M-b M-d 
M-b M-d w o r k n g M-SPC C-e w i t h SPC e s h e l 
l SPC i s SPC n o t SPC v e r y SPC e n j o y a b l 
e <return> C-x k <return> y M-x c d <return> C-g c 
d SPC / <return> M-x M-x M-x <return>

Recent messages:
Tramp: Opening connection for ssh.fh-trier.de using ssh...
Tramp: Sending command `exec ssh   -e none ssh.fh-trier.de'

Tramp: Waiting for prompts from remote shell
Tramp: Sending command `exec ssh   -e none ssh.fh-trier.de'
Tramp: Found remote shell prompt on `ssh.fh-trier.de'
Tramp: Opening connection for ssh.fh-trier.de using ssh...done
Quit
Buffer *unsent mail to bug-gnu-emacs@gnu.org* modified; kill anyway? (y or n)  y
Quit

Load-path shadows:
/home/politza/.emacs.d/scratch hides /home/politza/.emacs.d/plugins/jedi/scratch
/home/politza/.emacs.d/plugins/haskell-mode/haskell-cabal hides /home/politza/.emacs.d/plugins/haskell/haskell-cabal
/home/politza/.emacs.d/plugins/haskell-mode/haskell-decl-scan hides /home/politza/.emacs.d/plugins/haskell/haskell-decl-scan
/home/politza/.emacs.d/plugins/haskell-mode/haskell-doc hides /home/politza/.emacs.d/plugins/haskell/haskell-doc
/home/politza/.emacs.d/plugins/haskell-mode/ghc-core hides /home/politza/.emacs.d/plugins/haskell/ghc-core
/home/politza/.emacs.d/plugins/haskell-mode/haskell-mode hides /home/politza/.emacs.d/plugins/haskell/haskell-mode
/home/politza/.emacs.d/plugins/haskell-mode/haskell-c hides /home/politza/.emacs.d/plugins/haskell/haskell-c
/home/politza/.emacs.d/plugins/haskell-mode/haskell-indentation hides /home/politza/.emacs.d/plugins/haskell/haskell-indentation
/home/politza/.emacs.d/plugins/haskell-mode/haskell-site-file hides /home/politza/.emacs.d/plugins/haskell/haskell-site-file
/home/politza/.emacs.d/plugins/haskell-mode/haskell-ghci hides /home/politza/.emacs.d/plugins/haskell/haskell-ghci
/home/politza/.emacs.d/plugins/haskell-mode/inf-haskell hides /home/politza/.emacs.d/plugins/haskell/inf-haskell
/home/politza/.emacs.d/plugins/haskell-mode/haskell-indent hides /home/politza/.emacs.d/plugins/haskell/haskell-indent
/home/politza/.emacs.d/plugins/haskell-mode/haskell-hugs hides /home/politza/.emacs.d/plugins/haskell/haskell-hugs
/home/politza/.emacs.d/plugins/haskell-mode/haskell-font-lock hides /home/politza/.emacs.d/plugins/haskell/haskell-font-lock
/home/politza/.emacs.d/plugins/haskell-mode/haskell-simple-indent hides /home/politza/.emacs.d/plugins/haskell/haskell-simple-indent
/home/politza/.emacs.d/plugins/tblc/tblc hides /home/politza/.emacs.d/plugins/tblc
/home/politza/.emacs.d/plugins/haskell-mode/haskell-cabal hides /usr/share/emacs/site-lisp/haskell-mode/haskell-cabal
/home/politza/.emacs.d/plugins/haskell-mode/haskell-doc hides /usr/share/emacs/site-lisp/haskell-mode/haskell-doc
/home/politza/.emacs.d/plugins/haskell-mode/haskell-mode hides /usr/share/emacs/site-lisp/haskell-mode/haskell-mode
/home/politza/.emacs.d/plugins/haskell-mode/haskell-c hides /usr/share/emacs/site-lisp/haskell-mode/haskell-c
/home/politza/.emacs.d/plugins/haskell-mode/haskell-indentation hides /usr/share/emacs/site-lisp/haskell-mode/haskell-indentation
/home/politza/.emacs.d/plugins/haskell-mode/haskell-site-file hides /usr/share/emacs/site-lisp/haskell-mode/haskell-site-file
/home/politza/.emacs.d/plugins/haskell-mode/haskell-ghci hides /usr/share/emacs/site-lisp/haskell-mode/haskell-ghci
/home/politza/.emacs.d/plugins/haskell-mode/inf-haskell hides /usr/share/emacs/site-lisp/haskell-mode/inf-haskell
/home/politza/.emacs.d/plugins/haskell-mode/haskell-hugs hides /usr/share/emacs/site-lisp/haskell-mode/haskell-hugs
/home/politza/.emacs.d/plugins/haskell-mode/haskell-font-lock hides /usr/share/emacs/site-lisp/haskell-mode/haskell-font-lock
/home/politza/.emacs.d/plugins/haskell-mode/haskell-simple-indent hides /usr/share/emacs/site-lisp/haskell-mode/haskell-simple-indent
/home/politza/.emacs.d/plugins/haskell-mode/haskell-decl-scan hides /usr/share/emacs/site-lisp/haskell-mode/haskell-decl-scan
/home/politza/.emacs.d/plugins/haskell-mode/haskell-indent hides /usr/share/emacs/site-lisp/haskell-mode/haskell-indent
/home/politza/.emacs.d/plugins/haskell-mode/haskell-decl-scan hides /usr/share/emacs/24.2.50/site-lisp/haskell-mode/haskell-decl-scan
/home/politza/.emacs.d/plugins/haskell-mode/haskell-c hides /usr/share/emacs/24.2.50/site-lisp/haskell-mode/haskell-c
/home/politza/.emacs.d/plugins/haskell-mode/haskell-ghci hides /usr/share/emacs/24.2.50/site-lisp/haskell-mode/haskell-ghci
/home/politza/.emacs.d/plugins/haskell-mode/haskell-doc hides /usr/share/emacs/24.2.50/site-lisp/haskell-mode/haskell-doc
/home/politza/.emacs.d/plugins/haskell-mode/haskell-indent hides /usr/share/emacs/24.2.50/site-lisp/haskell-mode/haskell-indent
/home/politza/.emacs.d/plugins/haskell-mode/haskell-mode hides /usr/share/emacs/24.2.50/site-lisp/haskell-mode/haskell-mode
/home/politza/.emacs.d/plugins/haskell-mode/haskell-hugs hides /usr/share/emacs/24.2.50/site-lisp/haskell-mode/haskell-hugs
/home/politza/.emacs.d/plugins/haskell-mode/haskell-site-file hides /usr/share/emacs/24.2.50/site-lisp/haskell-mode/haskell-site-file
/home/politza/.emacs.d/plugins/haskell-mode/haskell-cabal hides /usr/share/emacs/24.2.50/site-lisp/haskell-mode/haskell-cabal
/home/politza/.emacs.d/plugins/haskell-mode/inf-haskell hides /usr/share/emacs/24.2.50/site-lisp/haskell-mode/inf-haskell
/home/politza/.emacs.d/plugins/haskell-mode/haskell-font-lock hides /usr/share/emacs/24.2.50/site-lisp/haskell-mode/haskell-font-lock
/home/politza/.emacs.d/plugins/haskell-mode/haskell-simple-indent hides /usr/share/emacs/24.2.50/site-lisp/haskell-mode/haskell-simple-indent
/home/politza/.emacs.d/plugins/haskell-mode/haskell-indentation hides /usr/share/emacs/24.2.50/site-lisp/haskell-mode/haskell-indentation
/usr/share/emacs/24.2.50/site-lisp/cscope/xcscope hides /usr/share/emacs/site-lisp/xcscope
/usr/share/emacs/24.2.50/site-lisp/debian-startup hides /usr/share/emacs/site-lisp/debian-startup
/home/politza/.emacs.d/plugins/term hides /usr/share/emacs/24.2.50/lisp/term
/home/politza/.emacs.d/plugins/pdf-isearch/startup hides /usr/share/emacs/24.2.50/lisp/startup
/usr/share/emacs/site-lisp/rst hides /usr/share/emacs/24.2.50/lisp/textmodes/rst
/usr/share/emacs/site-lisp/remember-el/remember hides /usr/share/emacs/24.2.50/lisp/textmodes/remember
/home/politza/.emacs.d/plugins/python/python/python hides /usr/share/emacs/24.2.50/lisp/progmodes/python
/home/politza/.emacs.d/plugins/el-get/.dir-locals hides /usr/share/emacs/24.2.50/lisp/gnus/.dir-locals
/home/politza/.emacs.d/calc hides /usr/share/emacs/24.2.50/lisp/calc/calc
/home/politza/.emacs.d/plugins/python/python/sym-comp hides /usr/share/emacs/24.2.50/lisp/obsolete/sym-comp

Features:
(shadow emacsbug vc-bzr cc-langs cc-mode cc-fonts cc-guess cc-menus
cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs mule-util
reposition tramp-cmds tramp-cache tramp-sh misearch multi-isearch
dired-aux scroll-hide-empty-lines jka-compr pp em-unix em-term term
ehelp electric ange-ftp em-script em-prompt em-ls em-hist em-pred
em-glob em-dirs em-cmpl em-basic em-banner em-alias esh-var esh-io
esh-cmd esh-opt esh-ext esh-proc esh-arg esh-groups eshell esh-module
esh-mode esh-util mailalias smtpmail sendmail qp ispell quail sort
smiley gnus-cite mail-extr gnus-async gnus-bcklg gnus-ml vc-dispatcher
vc-svn disp-table two-column iso-transl nndraft nnmh utf-7 gnutls
network-stream starttls nnimap parse-time tls utf7 netrc nnfolder nnnil
gnus-agent gnus-srvr gnus-score score-mode nnvirtual gnus-msg gnus-art
mm-uu mml2015 epg-config mm-view mml-smime smime dig mailcap nntp
gnus-cache gnus-sum nnoo gnus-group gnus-undo nnmail mail-source
gnus-start gnus-spec gnus-int gnus-range message rfc822 mml mml-sec
mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045
ietf-drums mailabbrev gmm-utils mailheader gnus-win gnus gnus-ems
nnheader mail-utils imenu plugin-autoloads emacs-customizations
nogroup-customizations wp-customizations view-customizations
tex-customizations reftex-customizations
reftex-miscellaneous-configurations-customizations
reftex-label-support-customizations
reftex-referencing-labels-customizations
reftex-defining-label-environments-customizations AUCTeX-customizations
preview-customizations preview-latex-customizations
preview-appearance-customizations TeX-parse-customizations
TeX-file-customizations TeX-command-customizations
TeX-view-customizations LaTeX-customizations LaTeX-macro-customizations
LaTeX-math-customizations LaTeX-indentation-customizations
table-customizations table-hooks-customizations outlines-customizations
programming-customizations tools-customizations vc-customizations
semantic-customizations makefile-customizations etags-customizations
ediff-customizations diff-mode-customizations languages-customizations
matlab-customizations sh-customizations python-customizations rx
haskell-customizations c-customizations multimedia-customizations
image-customizations help-customizations ekey-customizations
info-lookup-customizations info-customizations customize-customizations
custom-buffer-customizations apropos-customizations help-mode
files-customizations uniquify-customizations uniquify
sunrise-customizations recentf-customizations find-file-customizations
backup-customizations faces-customizations
highlight-symbol-customizations font-lock-customizations
external-customizations server-customizations processes-customizations
shell-customizations gud-customizations tooltip-customizations
grep-customizations compilation-customizations next-error-customizations
comint-customizations SQL-customizations man-customizations
environment-customizations windows-customizations
minibuffer-customizations savehist-customizations completion-spelling
menu-customizations keyboard-customizations chistory-customizations
initialization-customizations frames-customizations
ediff-window-customizations dired-customizations dired-x-customizations
dired-x dired-details-customizations editing-customizations
yasnippet-customizations matching-customizations
paren-matching-customizations paren-showing-customizations
isearch-customizations bookmark-customizations killing-customizations
indent-customizations fill-customizations emulations-customizations
editing-basics-customizations development-customizations
lisp-customizations re-builder-customizations
inferior-lisp-customizations ielm-customizations ert-customizations
edebug-customizations bytecomp-customizations advice-customizations
internal-customizations alloc-customizations extensions-customizations
eldoc-customizations cust-print-customizations data-customizations
save-place-customizations convenience-customizations
imenu-tree-customizations tags-tree-customizations
company-customizations window-numbering-customizations
pabbrev-customizations kmacro-customizations imenu-customizations
ibuffer-customizations ibuf-macs hl-line-customizations
hippie-expand-customizations file-cache-customizations
ffap-customizations completion-customizations iswitchb-customizations
auto-complete-customizations browse-kill-ring-customizations
auto-revert-customizations auto-insert-customizations
comm-customizations tramp-customizations browse-url-customizations
applications-customizations mediawiki-customizations w3m-customizations
package-customizations mail-customizations smtpmail-customizations
sendmail-customizations gnus-customizations gnus-summary-customizations
gnus-summary-various-customizations gnus-summary-exit-customizations
gnus-start-customizations gnus-server-customizations
gnus-message-customizations message-customizations
gnus-exit-customizations ispell-customizations eshell-customizations
edebug doc-view-customizations calendar-customizations
org-customizations org-structure-customizations
org-plain-lists-customizations org-edit-structure-customizations
org-startup-customizations org-link-customizations
org-latex-customizations org-appearance-customizations
calculator-customizations calc-customizations server idle-help tblc
pcase hl-line read-interactively paren recentf tree-widget
.plugin-autoloads yasnippet dropdown-list window-numbering w3m
browse-url timezone w3m-hist w3m-e23 w3m-ccl ccl w3m-fsf w3m-favicon
w3m-image w3m-proc w3m-util view tramp tramp-compat tramp-loaddefs shell
track-last-window scroll-other-window saveplace savehist reftex
reftex-vars pabbrev thingatpt org ob-tangle ob-ref ob-lob ob-table
org-footnote org-src ob-comint ob-keys org-pcomplete pcomplete org-list
org-faces org-entities noutline outline org-version ob-emacs-lisp ob
org-compat org-macs ob-eval format-spec find-func cal-menu calendar
cal-loaddefs lib-edit lib-window lib-isearch lib-buffer reveal iswitchb
lib-basic lib-lispext latex easy-mmode tex-style tex info-look ibuffer
hippie-exp grep compile filecache edit-minibuffer eldoc-eval ekey assoc
dired-details+ warnings advice advice-preload dired dired-details
cool-prefix-bindings do-text lib-kbd comint-history comint ansi-color
ring anticus edmacro kmacro derived cl-macs gv ffap url-parse
auth-source eieio byte-opt bytecomp byte-compile cconv gnus-util mm-util
mail-prsvr password-cache url-vars eldoc help-fns cus-edit cus-start
cus-load wid-edit cl cl-lib auctex-autoloads tex-site info easymenu
browse-kill-ring-autoloads c-eldoc-autoloads cl-format-autoloads
company-autoloads eldoc-eval-autoloads highlight-symbol-autoloads
trie-autoloads heap-autoloads tNFA-autoloads queue-autoloads
undo-tree-autoloads package remember-autoloads planner-autoloads
w3m-load muse-autoloads time-date tooltip ediff-hook vc-hooks
lisp-float-type mwheel x-win x-dnd tool-bar dnd fontset image regexp-opt
fringe tabulated-list newcomment lisp-mode register page menu-bar
rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax
facemenu font-core frame cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese hebrew greek romanian slovak
czech european ethiopic indian cyrillic chinese case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer loaddefs button faces
cus-face macroexp files text-properties overlay sha1 md5 base64 format
env code-pages mule custom widget hashtable-print-readable backquote
make-network-process dbusbind dynamic-setting font-render-setting
move-toolbar gtk x-toolkit x multi-tty emacs)

bug#13677: 24.2.50; workng on remote hosts with eshell is not very enjoyable

[Michael Albinus]

Andreas Politz <politza@fh-trier.de> writes:

> emacs -Q
> M-x eshell
> cd /ssh:somehost.com:
> stat .
> stat .
> ...
>
> Every stat command starts a new tramp process, which, in my case, takes
> about 3-4 seconds to get going and renders eshell fairly useless for
> this kind of work. 
>
> Can this be changed such that eshell uses start-file-process only if the
> command results in a background process and use process-file otherwise ?

I fear that is not possible in general. The command could be
interactive, requiring user input. This would not be possible with
process-file.

And eshell does not know in advance, whether an external command runs
interactively.

> Andreas

Best regards, Michael.

bug#13677: 24.2.50; workng on remote hosts with eshell is not very enjoyable

[Stefan Monnier]

>> Every stat command starts a new tramp process, which, in my case, takes
>> about 3-4 seconds to get going and renders eshell fairly useless for
>> this kind of work. 

I think you want to enable "ControlPersist" and/or related options in
your ~/.ssh/config, so that only the first ssh needs to authenticate and
subsequent ones just reuse the connection already setup by the first.


        Stefan

bug#13677: 24.2.50; workng on remote hosts with eshell is not very enjoyable

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> Every stat command starts a new tramp process, which, in my case, takes
>>> about 3-4 seconds to get going and renders eshell fairly useless for
>>> this kind of work.
>
> I think you want to enable "ControlPersist" and/or related options in
> your ~/.ssh/config, so that only the first ssh needs to authenticate and
> subsequent ones just reuse the connection already setup by the first.

I've committed a patch to Tramp to use ControlMaster=auto whenever
possible. This might speed up remote processes a little bit, but most of
the startup time is still spent in Tramp's hand-shaking.

It would be good if we could have a mechanism to say eshell, whether an
external command shall run synchronously or asynchronously. I doubt,
that there will be a clever automatic detection. Eshell cannot know in
advance, wheter an external command is interactive, or not.

>         Stefan

Best regards, Michael.

bug#13677: 24.2.50; workng on remote hosts with eshell is not very enjoyable

[Andreas Politz]

Michael Albinus <michael.albinus@gmx.de> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> I think you want to enable "ControlPersist" and/or related options [...]

> I've committed a patch to Tramp to use ControlMaster=auto whenever
> possible. This might speed up remote processes a little bit, but most of
> the startup time is still spent in Tramp's hand-shaking.

Yes, it does help a little.  (Though ControlPersist is not available on
this system.)

>
> It would be good if we could have a mechanism to say eshell, whether an
> external command shall run synchronously or asynchronously. 

What about letting the user decide, by having a second command like
eshell-send-input-synchronously ?

Andreas

bug#13677: have a second connection for interactive eshell cmds (to avoid long handshaking)

[Ivan Zakharyaschev]

(This issue was brought up at
http://emacs.stackexchange.com/a/4035/5165 , and there I had some
discussion which led to an idea.)

Another idea has just come to me: What if eshell would open a second
persistent connection for interactive commands, and would reuse it?
Then we could avoid the long waiting for remote prompt etc. every time
(the long TRAMP's handshaking), isnt' that true?

Any eshell hackers?

-- 
Ivan