[Eshell patch] Visual subcommands and options

[Eshell patch] Visual subcommands and options

[Tassilo Horn]

[-- Attachment #1: Type: text/plain, Size: 894 bytes --]

Hi all,

eshell shows the output of commands in `eshell-visual-commands' in a
term buffer.  Those are typically interactive commands that show their
results in a pager or other interactive facility.  Typical commands like
that are top, less, more, etc.

However, until now you could only say "this command is visual".  But
there are many commands I use regularly that usually are not visual
except when they are called with some special option or subcommand.
Typical examples are several VCS commands such as "git log", "git show",
"git diff", or "hg log" which show their output with a pager by default
and thus are then visual in eshell's sense.

This patch adds two new defcustoms `eshell-visual-subcommands' and
`eshell-visual-options' that lets you also specifiy commands with
certain subcommands and options that make them visual.

Any objections to installing this patch?

Bye,
Tassilo


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: eshell-visual.patch --]
[-- Type: text/x-diff, Size: 6973 bytes --]

=== modified file 'lisp/ChangeLog'
--- lisp/ChangeLog	2013-05-31 07:34:56 +0000
+++ lisp/ChangeLog	2013-05-31 10:30:34 +0000
@@ -1,3 +1,22 @@
+2013-05-31  Tassilo Horn  <tsdh@gnu.org>
+
+	* eshell/esh-ext.el (eshell-external-command): Pass args to
+	`eshell-find-interpreter'.
+	(eshell-find-interpreter): Add new second parameter ARGS.
+
+	* eshell/em-script.el (eshell-script-initialize): Add second arg
+	to the function added as MATCH to `eshell-interpreter-alist'
+
+	* eshell/em-dirs.el (eshell-dirs-initialize): Add second arg to
+	the function added as MATCH to `eshell-interpreter-alist'
+
+	* eshell/em-term.el (eshell-visual-subcommands): New defcustom.
+	(eshell-visual-options): New defcustom.
+	(eshell-escape-control-x): Adapt docstring.
+	(eshell-term-initialize): Test `eshell-visual-subcommands' and
+	`eshell-visual-options' in addition to `eshell-visual-commands'.
+	(eshell-exec-visual): Pass args to `eshell-find-interpreter'.
+
 2013-05-31  Glenn Morris  <rgm@gnu.org>
 
 	* obsolete/sym-comp.el (symbol-complete):

=== modified file 'lisp/eshell/em-dirs.el'
--- lisp/eshell/em-dirs.el	2013-01-01 09:11:05 +0000
+++ lisp/eshell/em-dirs.el	2013-05-31 10:29:59 +0000
@@ -207,7 +207,8 @@
   (when eshell-cd-on-directory
     (make-local-variable 'eshell-interpreter-alist)
     (setq eshell-interpreter-alist
-	  (cons (cons 'eshell-lone-directory-p
+	  (cons (cons (lambda (file args)
+			(eshell-lone-directory-p file))
 		      'eshell-dirs-substitute-cd)
 		eshell-interpreter-alist)))
 

=== modified file 'lisp/eshell/em-script.el'
--- lisp/eshell/em-script.el	2013-01-01 09:11:05 +0000
+++ lisp/eshell/em-script.el	2013-05-31 10:27:19 +0000
@@ -61,7 +61,7 @@
   "Initialize the script parsing code."
   (make-local-variable 'eshell-interpreter-alist)
   (setq eshell-interpreter-alist
-	(cons '((lambda (file)
+	(cons '((lambda (file ignore-args)
 		  (string= (file-name-nondirectory file)
 			   "eshell")) . eshell/source)
 	      eshell-interpreter-alist))

=== modified file 'lisp/eshell/em-term.el'
--- lisp/eshell/em-term.el	2013-05-23 04:57:27 +0000
+++ lisp/eshell/em-term.el	2013-05-31 10:56:17 +0000
@@ -65,6 +65,39 @@
   :type '(repeat string)
   :group 'eshell-term)
 
+(defcustom eshell-visual-subcommands
+  nil
+  "An alist of the form
+
+  ((COMMAND1 (SUBCOMMAND1 SUBCOMMAND2...))
+   (COMMAND2 (SUBCOMMAND1 ...)))
+
+of commands with subcommands that present their output in a
+visual fashion.  A likely entry is
+
+  (\"git\" (\"log\" \"diff\" \"show\"))
+
+because git shows logs and diffs using a pager by default."
+  :type '(repeat (list (string :tag "Command")
+		       (repeat (string :tag "Subcommand")))))
+
+(defcustom eshell-visual-options
+  nil
+  "An alist of the form
+
+  ((COMMAND1 (OPTION1 OPTION2...))
+   (COMMAND2 (OPTION1 ...)))
+
+of commands with options that present their output in a visual
+fashion.  For example, an entry for a hypothetical foo command
+
+  (\"foo\" (\"--with-more\" \"--with-less\"))
+
+would show the foo output visually if it was called with the
+--with-more or --with-less option."
+  :type '(repeat (list (string :tag "Command")
+		       (repeat (string :tag "Option")))))
+
 ;; If you change this from term-term-name, you need to ensure that the
 ;; value you choose exists in the system's terminfo database.  (Bug#12485)
 (defcustom eshell-term-name term-term-name
@@ -77,8 +110,10 @@
 
 (defcustom eshell-escape-control-x t
   "If non-nil, allow <C-x> to be handled by Emacs key in visual buffers.
-See the variable `eshell-visual-commands'.  If this variable is set to
-nil, <C-x> will send that control character to the invoked process."
+See the variables `eshell-visual-commands',
+`eshell-visual-subcommands', and `eshell-visual-options'.  If
+this variable is set to nil, <C-x> will send that control
+character to the invoked process."
   :type 'boolean
   :group 'eshell-term)
 
@@ -93,9 +128,14 @@
   (make-local-variable 'eshell-interpreter-alist)
   (setq eshell-interpreter-alist
 	(cons (cons (function
-		     (lambda (command)
-		       (member (file-name-nondirectory command)
-			       eshell-visual-commands)))
+		     (lambda (command args)
+		       (or (member (file-name-nondirectory command)
+				   eshell-visual-commands)
+			   (member (car args)
+				   (cadr (assoc command eshell-visual-subcommands)))
+			   (intersection args
+					 (cadr (assoc command eshell-visual-options))
+					 :test 'string=))))
 		    'eshell-exec-visual)
 	      eshell-interpreter-alist)))
 
@@ -104,7 +144,7 @@
 ARGS are passed to the program.  At the moment, no piping of input is
 allowed."
   (let* (eshell-interpreter-alist
-	 (interp (eshell-find-interpreter (car args)))
+	 (interp (eshell-find-interpreter (car args) (cdr args)))
 	 (program (car interp))
 	 (args (eshell-flatten-list
 		(eshell-stringify-list (append (cdr interp)

=== modified file 'lisp/eshell/esh-ext.el'
--- lisp/eshell/esh-ext.el	2013-05-23 04:57:27 +0000
+++ lisp/eshell/esh-ext.el	2013-05-31 10:26:23 +0000
@@ -125,9 +125,10 @@
 
   (MATCH . INTERPRETER)
 
-MATCH should be a regexp, which is matched against the command name,
-or a function.  If either returns a non-nil value, then INTERPRETER
-will be used for that command.
+MATCH should be a regexp, which is matched against the command
+name, or a function receiving a command and its args.  If either
+returns a non-nil value, then INTERPRETER will be used for that
+command.
 
 If INTERPRETER is a string, it will be called as the command name,
 with the original command name passed as the first argument, with all
@@ -215,6 +216,7 @@
   (setq args (eshell-stringify-list (eshell-flatten-list args)))
   (let ((interp (eshell-find-interpreter
 		 command
+		 args
 		 ;; `eshell-find-interpreter' does not work correctly
 		 ;; for Tramp file name syntax.  But we don't need to
 		 ;; know the interpreter in that case, therefore the
@@ -267,7 +269,7 @@
 		(list (match-string 1)
 		      file)))))))
 
-(defun eshell-find-interpreter (file &optional no-examine-p)
+(defun eshell-find-interpreter (file args &optional no-examine-p)
   "Find the command interpreter with which to execute FILE.
 If NO-EXAMINE-P is non-nil, FILE will not be inspected for a script
 line of the form #!<interp>."
@@ -277,8 +279,9 @@
 	    (dolist (possible eshell-interpreter-alist)
 	      (cond
 	       ((functionp (car possible))
-		(and (funcall (car possible) file)
-		     (throw 'found (cdr possible))))
+		(let ((fn (car possible)))
+		  (and (funcall fn file args)
+		       (throw 'found (cdr possible)))))
 	       ((stringp (car possible))
 		(and (string-match (car possible) file)
 		     (throw 'found (cdr possible))))
@@ -312,7 +315,7 @@
 	    (setq interp (eshell-script-interpreter fullname))
 	    (if interp
 		(setq interp
-		      (cons (car (eshell-find-interpreter (car interp) t))
+		      (cons (car (eshell-find-interpreter (car interp) args t))
 			    (cdr interp)))))
 	  (or interp (list fullname)))))))
 

Re: [Eshell patch] Visual subcommands and options

[Tassilo Horn]

[-- Attachment #1: Type: text/plain, Size: 349 bytes --]

Hi,

here's a slightly updated version.  The most prominent change is that
the new variables are now usual alists with entries (COMMAND
SUBCOMMAND...) and (COMMAND OPTION...) instead of the subcommands and
options wrapped into yet another list.  The reason it was that way
previously was that I didn't find the `cons' custom type. ;-)

Bye,
Tassilo

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: eshell-visual.patch --]
[-- Type: text/x-diff, Size: 6975 bytes --]

=== modified file 'lisp/ChangeLog'
--- lisp/ChangeLog	2013-06-01 03:38:36 +0000
+++ lisp/ChangeLog	2013-06-01 09:07:09 +0000
@@ -1,3 +1,22 @@
+2013-06-01  Tassilo Horn  <tsdh@gnu.org>
+
+	* eshell/esh-ext.el (eshell-external-command): Pass args to
+	`eshell-find-interpreter'.
+	(eshell-find-interpreter): Add new second parameter ARGS.
+
+	* eshell/em-script.el (eshell-script-initialize): Add second arg
+	to the function added as MATCH to `eshell-interpreter-alist'
+
+	* eshell/em-dirs.el (eshell-dirs-initialize): Add second arg to
+	the function added as MATCH to `eshell-interpreter-alist'
+
+	* eshell/em-term.el (eshell-visual-subcommands): New defcustom.
+	(eshell-visual-options): New defcustom.
+	(eshell-escape-control-x): Adapt docstring.
+	(eshell-term-initialize): Test `eshell-visual-subcommands' and
+	`eshell-visual-options' in addition to `eshell-visual-commands'.
+	(eshell-exec-visual): Pass args to `eshell-find-interpreter'.
+
 2013-06-01  Leo Liu  <sdl.web@gmail.com>
 
 	* progmodes/octave.el (octave-mode-syntax-table): Give `.'

=== modified file 'lisp/eshell/em-dirs.el'
--- lisp/eshell/em-dirs.el	2013-01-01 09:11:05 +0000
+++ lisp/eshell/em-dirs.el	2013-05-31 10:29:59 +0000
@@ -207,7 +207,8 @@
   (when eshell-cd-on-directory
     (make-local-variable 'eshell-interpreter-alist)
     (setq eshell-interpreter-alist
-	  (cons (cons 'eshell-lone-directory-p
+	  (cons (cons (lambda (file args)
+			(eshell-lone-directory-p file))
 		      'eshell-dirs-substitute-cd)
 		eshell-interpreter-alist)))
 

=== modified file 'lisp/eshell/em-script.el'
--- lisp/eshell/em-script.el	2013-01-01 09:11:05 +0000
+++ lisp/eshell/em-script.el	2013-05-31 11:02:17 +0000
@@ -61,7 +61,7 @@
   "Initialize the script parsing code."
   (make-local-variable 'eshell-interpreter-alist)
   (setq eshell-interpreter-alist
-	(cons '((lambda (file)
+	(cons '((lambda (file args)
 		  (string= (file-name-nondirectory file)
 			   "eshell")) . eshell/source)
 	      eshell-interpreter-alist))

=== modified file 'lisp/eshell/em-term.el'
--- lisp/eshell/em-term.el	2013-05-23 04:57:27 +0000
+++ lisp/eshell/em-term.el	2013-06-01 09:16:43 +0000
@@ -65,6 +65,39 @@
   :type '(repeat string)
   :group 'eshell-term)
 
+(defcustom eshell-visual-subcommands
+  nil
+  "An alist of the form
+
+  ((COMMAND1 SUBCOMMAND1 SUBCOMMAND2...)
+   (COMMAND2 SUBCOMMAND1 ...))
+
+of commands with subcommands that present their output in a
+visual fashion.  A likely entry is
+
+  (\"git\" \"log\" \"diff\" \"show\")
+
+because git shows logs and diffs using a pager by default."
+  :type '(repeat (cons (string :tag "Command")
+		       (repeat (string :tag "Subcommand")))))
+
+(defcustom eshell-visual-options
+  nil
+  "An alist of the form
+
+  ((COMMAND1 OPTION1 OPTION2...)
+   (COMMAND2 OPTION1 ...))
+
+of commands with options that present their output in a visual
+fashion.  For example, a sensible entry would be
+
+  (\"git\" \"--help\")
+
+because \"git <command> --help\" shows the command's
+documentation with a pager."
+  :type '(repeat (cons (string :tag "Command")
+		       (repeat (string :tag "Option")))))
+
 ;; If you change this from term-term-name, you need to ensure that the
 ;; value you choose exists in the system's terminfo database.  (Bug#12485)
 (defcustom eshell-term-name term-term-name
@@ -77,8 +110,10 @@
 
 (defcustom eshell-escape-control-x t
   "If non-nil, allow <C-x> to be handled by Emacs key in visual buffers.
-See the variable `eshell-visual-commands'.  If this variable is set to
-nil, <C-x> will send that control character to the invoked process."
+See the variables `eshell-visual-commands',
+`eshell-visual-subcommands', and `eshell-visual-options'.  If
+this variable is set to nil, <C-x> will send that control
+character to the invoked process."
   :type 'boolean
   :group 'eshell-term)
 
@@ -93,9 +128,14 @@
   (make-local-variable 'eshell-interpreter-alist)
   (setq eshell-interpreter-alist
 	(cons (cons (function
-		     (lambda (command)
-		       (member (file-name-nondirectory command)
-			       eshell-visual-commands)))
+		     (lambda (command args)
+		       (let ((command (file-name-nondirectory command)))
+			 (or (member command eshell-visual-commands)
+			     (member (car args)
+				     (cdr (assoc command eshell-visual-subcommands)))
+			     (intersection args
+					   (cdr (assoc command eshell-visual-options))
+					   :test 'string=)))))
 		    'eshell-exec-visual)
 	      eshell-interpreter-alist)))
 
@@ -104,7 +144,7 @@
 ARGS are passed to the program.  At the moment, no piping of input is
 allowed."
   (let* (eshell-interpreter-alist
-	 (interp (eshell-find-interpreter (car args)))
+	 (interp (eshell-find-interpreter (car args) (cdr args)))
 	 (program (car interp))
 	 (args (eshell-flatten-list
 		(eshell-stringify-list (append (cdr interp)

=== modified file 'lisp/eshell/esh-ext.el'
--- lisp/eshell/esh-ext.el	2013-05-23 04:57:27 +0000
+++ lisp/eshell/esh-ext.el	2013-05-31 14:48:30 +0000
@@ -125,9 +125,10 @@
 
   (MATCH . INTERPRETER)
 
-MATCH should be a regexp, which is matched against the command name,
-or a function.  If either returns a non-nil value, then INTERPRETER
-will be used for that command.
+MATCH should be a regexp, which is matched against the command
+name, or a function of arity 2 receiving the COMMAND and its
+ARGS (a list).  If either returns a non-nil value, then
+INTERPRETER will be used for that command.
 
 If INTERPRETER is a string, it will be called as the command name,
 with the original command name passed as the first argument, with all
@@ -215,6 +216,7 @@
   (setq args (eshell-stringify-list (eshell-flatten-list args)))
   (let ((interp (eshell-find-interpreter
 		 command
+		 args
 		 ;; `eshell-find-interpreter' does not work correctly
 		 ;; for Tramp file name syntax.  But we don't need to
 		 ;; know the interpreter in that case, therefore the
@@ -267,7 +269,7 @@
 		(list (match-string 1)
 		      file)))))))
 
-(defun eshell-find-interpreter (file &optional no-examine-p)
+(defun eshell-find-interpreter (file args &optional no-examine-p)
   "Find the command interpreter with which to execute FILE.
 If NO-EXAMINE-P is non-nil, FILE will not be inspected for a script
 line of the form #!<interp>."
@@ -277,8 +279,9 @@
 	    (dolist (possible eshell-interpreter-alist)
 	      (cond
 	       ((functionp (car possible))
-		(and (funcall (car possible) file)
-		     (throw 'found (cdr possible))))
+		(let ((fn (car possible)))
+		  (and (funcall fn file args)
+		       (throw 'found (cdr possible)))))
 	       ((stringp (car possible))
 		(and (string-match (car possible) file)
 		     (throw 'found (cdr possible))))
@@ -312,7 +315,7 @@
 	    (setq interp (eshell-script-interpreter fullname))
 	    (if interp
 		(setq interp
-		      (cons (car (eshell-find-interpreter (car interp) t))
+		      (cons (car (eshell-find-interpreter (car interp) args t))
 			    (cdr interp)))))
 	  (or interp (list fullname)))))))
 

Re: [Eshell patch] Visual subcommands and options

[Aidan Gauland]

Tassilo Horn <tsdh@gnu.org> writes:

> eshell shows the output of commands in `eshell-visual-commands' in a
> term buffer.  Those are typically interactive commands that show their
> results in a pager or other interactive facility.  Typical commands like
> that are top, less, more, etc.
>
> However, until now you could only say "this command is visual".  But
> there are many commands I use regularly that usually are not visual
> except when they are called with some special option or subcommand.
> Typical examples are several VCS commands such as "git log", "git show",
> "git diff", or "hg log" which show their output with a pager by default
> and thus are then visual in eshell's sense.
>
> This patch adds two new defcustoms `eshell-visual-subcommands' and
> `eshell-visual-options' that lets you also specifiy commands with
> certain subcommands and options that make them visual.
>
> Any objections to installing this patch?

Sounds good.  I wouldn't use this myself, since I just set the PAGER
environment variable to "cat", so the output is just dumped to the
*eshell* buffer, but I would use this if any programs go into an ncurses
mode only for some subcommands.  I don't have anything against this
being added, though, so go ahead! :) (And the patch looks sane enough.)

--Aidan

Re: [Eshell patch] Visual subcommands and options

[Tassilo Horn]

Aidan Gauland <aidalgol@no8wireless.co.nz> writes:

Hi Aidan,

>> This patch adds two new defcustoms `eshell-visual-subcommands' and
>> `eshell-visual-options' that lets you also specifiy commands with
>> certain subcommands and options that make them visual.
>>
>> Any objections to installing this patch?
>
> Sounds good.  I wouldn't use this myself, since I just set the PAGER
> environment variable to "cat", so the output is just dumped to the
> *eshell* buffer,

If the output is large as with "git/bzr log", you have wait until it's
all there before starting to, say, search.  And since the most recent
commits are on top of the output, I prefer starting there than below the
initial commit.

Well, but your approach has the benefit that you can use standard emacs
commands whereas I have to fiddle with what the pager provides.  Hm, is
there something that lets me do isearch in a term buffer and gives me a
point so that I can copy and yank?  Currently, I'm mousing my way
there...

> but I would use this if any programs go into an ncurses mode only for
> some subcommands.  I don't have anything against this being added,
> though, so go ahead! :) (And the patch looks sane enough.)

Great, thanks.  Installed as revno 112822.

Bye,
Tassilo

Re: [Eshell patch] Visual subcommands and options

[Aidan Gauland]

Tassilo Horn <tsdh@gnu.org> writes:

> If the output is large as with "git/bzr log", you have wait until it's
> all there before starting to, say, search.  And since the most recent
> commits are on top of the output, I prefer starting there than below the
> initial commit.

Oh yeah, good point.  I should see if there's some other way that's
non-blocking.  Maybe there's a value for PAGER that tells the programs
to just dump to stdout.

> Well, but your approach has the benefit that you can use standard emacs
> commands whereas I have to fiddle with what the pager provides.  Hm, is
> there something that lets me do isearch in a term buffer and gives me a
> point so that I can copy and yank?  Currently, I'm mousing my way
> there...

You can use C-c C-j to switch to normal Emacs line-oriented mode (called
term-line-mode, which is not actually a mode), and C-c C-k to switch
back to terminal mode.  You still have to use the pager to scroll and
search, though.

> Great, thanks.  Installed as revno 112822.

Thanks for the contribution!

--Aidan

Re: [Eshell patch] Visual subcommands and options

[Tassilo Horn]

Aidan Gauland <aidalgol@no8wireless.co.nz> writes:

> Tassilo Horn <tsdh@gnu.org> writes:
>
>> If the output is large as with "git/bzr log", you have wait until
>> it's all there before starting to, say, search.  And since the most
>> recent commits are on top of the output, I prefer starting there than
>> below the initial commit.
>
> Oh yeah, good point.  I should see if there's some other way that's
> non-blocking.  Maybe there's a value for PAGER that tells the programs
> to just dump to stdout.

On a related note: I just found another problem.  When you redirect the
output of a visual command, the redirection is completely ignored.  For
example, in bash or zsh

  $ less file.txt > foo.txt

(although a bit contrieved) copies the contents of file.txt to foo.txt.
In eshell, it just shows file.txt in a term buffer, and foo.txt isn't
created or touched at all.

A more realistic example is

  $ git diff > foo.patch

where I've configured "git diff" as visual command.

So eshell should ignore a command's "visuality" if its output is
redirected.

Bye,
Tassilo

Re: [Eshell patch] Visual subcommands and options

[Aidan Gauland]

Tassilo Horn <tsdh@gnu.org> writes:

> So eshell should ignore a command's "visuality" if its output is
> redirected.

Definitely a bug.  I have added this to my TODO list.

Thanks for pointing this out!
--Aidan

Re: Some more eshell problems

[Aidan Gauland]

Tassilo Horn <tsdh@gnu.org> writes:

> While we're at it, here are some other things:
>
> 1. I'm a bit unsure about what prefixing commands with * actually means.
>    John Wiegley told me on IRC that it means "don't do any special
>    interpretation", and I assumed that includes interpreting commands
>    visually.  But still,
>
>      $ *top
>
>    shows top in a term buffer.

My understanding is that the * prefix tells Eshell to use the external
command instead of the built-in lisp command (if any).  So, for example,
ls invokes the lisp function eshell/ls, but *ls invokes /bin/ls.  I'll
have to take some time to check this in the source code.  (The command
parser is a bit of a mess.)

> 2. When I update my emacs checkout in eshell, I get this:
>
>      $ bzr pull
>      Using saved parent location: bzr+ssh://tsdh@bzr.savannah.gnu.org/emacs/trunk/
>      No revisions or tags to pull.
>      Killed by signal 1.
>
>    Doing the same in zsh or bash, I get the same output except for the
>    "Killed by signal 1.".  In all three cases, the return code $? is 0.
>    It seems that only happens with bzr commands, not with git or hg
>    commands.  So it's probably a bzr problem, right?

OK, that is really weird, and I have no idea where to start debugging
this, but I'd hazard a guess that it is Eshell weirdness, not bzr.

> 3. Sometimes, when I run "git log" or "bzr log" as visual commands, the
>    output is correctly shown in a term buffer, but when I hit q the mode
>    line switches from (Term: char run) to (Term: char no process) and
>    the buffer isn't killed.  I have no clue when this happens, but when
>    it does, it seems to stay that way for the whole emacs session.

That sounds like a problem with term (or ansi-term, whichever Eshell
invokes), but I suppose it's possibly a problem with how Eshell is
(possibly erratically) invoking (ansi-)term.

--Aidan

Re: Some more eshell problems

[Tassilo Horn]

Aidan Gauland <aidalgol@amuri.net> writes:

Hi Aidan,

>> 1. I'm a bit unsure about what prefixing commands with * actually means.
>>    John Wiegley told me on IRC that it means "don't do any special
>>    interpretation", and I assumed that includes interpreting commands
>>    visually.  But still,
>>
>>      $ *top
>>
>>    shows top in a term buffer.
>
> My understanding is that the * prefix tells Eshell to use the external
> command instead of the built-in lisp command (if any).  So, for example,
> ls invokes the lisp function eshell/ls, but *ls invokes /bin/ls.  I'll
> have to take some time to check this in the source code.  (The command
> parser is a bit of a mess.)

Ah, ok, that's also possible.  But still it would be nice to have
something to suppress all "magic" like visual commands.  For example,
I'm very happy with my visual customization of "git log" and friends,
but then it would allow me to work around the visual-while-redirection
bug.  Maybe the prefix ** would make sense.

>> 2. When I update my emacs checkout in eshell, I get this:
>>
>>      $ bzr pull
>>      Using saved parent location: bzr+ssh://tsdh@bzr.savannah.gnu.org/emacs/trunk/
>>      No revisions or tags to pull.
>>      Killed by signal 1.
>>
>>    Doing the same in zsh or bash, I get the same output except for the
>>    "Killed by signal 1.".  In all three cases, the return code $? is 0.
>>    It seems that only happens with bzr commands, not with git or hg
>>    commands.  So it's probably a bzr problem, right?
>
> OK, that is really weird, and I have no idea where to start debugging
> this, but I'd hazard a guess that it is Eshell weirdness, not bzr.

Ok.  But the commands work anyway, so it's more or less only a cosmetic
problem.

>> 3. Sometimes, when I run "git log" or "bzr log" as visual commands, the
>>    output is correctly shown in a term buffer, but when I hit q the mode
>>    line switches from (Term: char run) to (Term: char no process) and
>>    the buffer isn't killed.  I have no clue when this happens, but when
>>    it does, it seems to stay that way for the whole emacs session.
>
> That sounds like a problem with term (or ansi-term, whichever Eshell
> invokes), but I suppose it's possibly a problem with how Eshell is
> (possibly erratically) invoking (ansi-)term.

That actually happens really really seldomly.  If I get to the cause by
accident, I'll give you more information.

Bye,
Tassilo

Re: Some more eshell problems

[Thierry Volpiatto]

Tassilo Horn <tsdh@gnu.org> writes:

> Aidan Gauland <aidalgol@amuri.net> writes:
>
> Hi Aidan,
>
>>> 1. I'm a bit unsure about what prefixing commands with * actually means.
>>>    John Wiegley told me on IRC that it means "don't do any special
>>>    interpretation", and I assumed that includes interpreting commands
>>>    visually.  But still,
>>>
>>>      $ *top
>>>
>>>    shows top in a term buffer.
>>
>> My understanding is that the * prefix tells Eshell to use the external
>> command instead of the built-in lisp command (if any).  So, for example,
>> ls invokes the lisp function eshell/ls, but *ls invokes /bin/ls.  I'll
>> have to take some time to check this in the source code.  (The command
>> parser is a bit of a mess.)
>
> Ah, ok, that's also possible.  But still it would be nice to have
> something to suppress all "magic" like visual commands.  For example,
> I'm very happy with my visual customization of "git log" and friends,
> but then it would allow me to work around the visual-while-redirection
> bug.  Maybe the prefix ** would make sense.

If an `eshell/top' function exists, it will be used when using top,
otherwise, top and *top will be the same.

>>> 2. When I update my emacs checkout in eshell, I get this:
>>>
>>>      $ bzr pull
>>>      Using saved parent location: bzr+ssh://tsdh@bzr.savannah.gnu.org/emacs/trunk/
>>>      No revisions or tags to pull.
>>>      Killed by signal 1.
>>>
>>>    Doing the same in zsh or bash, I get the same output except for the
>>>    "Killed by signal 1.".  In all three cases, the return code $? is 0.
>>>    It seems that only happens with bzr commands, not with git or hg
>>>    commands.  So it's probably a bzr problem, right?
>>
>> OK, that is really weird, and I have no idea where to start debugging
>> this, but I'd hazard a guess that it is Eshell weirdness, not bzr.
>
> Ok.  But the commands work anyway, so it's more or less only a cosmetic
> problem.

Look into /dev/pts you will see the corresponding pts for eshell is
ephemeral (start-process), it is IMO the one that is "killed by signal 1"
when command exit.
You don't have the same in bash/zsh terminal because it exists in
/dev/pts.
It is why we have to type password at every sudo command, the pts can't
be registered.

It is my understanding, maybe I am wrong, correct me if so.

>>> 3. Sometimes, when I run "git log" or "bzr log" as visual commands, the
>>>    output is correctly shown in a term buffer, but when I hit q the mode
>>>    line switches from (Term: char run) to (Term: char no process) and
>>>    the buffer isn't killed.  I have no clue when this happens, but when
>>>    it does, it seems to stay that way for the whole emacs session.
>>
>> That sounds like a problem with term (or ansi-term, whichever Eshell
>> invokes), but I suppose it's possibly a problem with how Eshell is
>> (possibly erratically) invoking (ansi-)term.
>
> That actually happens really really seldomly.  If I get to the cause by
> accident, I'll give you more information.
>
> Bye,
> Tassilo
>
>

-- 
Thierry
Get my Gnupg key:
gpg --keyserver pgp.mit.edu --recv-keys 59F29997 

Re: Some more eshell problems

[Tassilo Horn]

Thierry Volpiatto <thierry.volpiatto@gmail.com> writes:

>> Ah, ok, that's also possible.  But still it would be nice to have
>> something to suppress all "magic" like visual commands.  For example,
>> I'm very happy with my visual customization of "git log" and friends,
>> but then it would allow me to work around the
>> visual-while-redirection bug.  Maybe the prefix ** would make sense.
>
> If an `eshell/top' function exists, it will be used when using top,
> otherwise, top and *top will be the same.

Yes, that's what Aidan said, and it's fine with me.  I just would like
to have something similar that also removes a commands "visuality",
i.e., so that `top` isn't shown in a term buffer but simply dumped
although it's in `eshell-visual-commands'.

>>>>      $ bzr pull
>>>>      Using saved parent location: bzr+ssh://tsdh@bzr.savannah.gnu.org/emacs/trunk/
>>>>      No revisions or tags to pull.
>>>>      Killed by signal 1.
>
> Look into /dev/pts you will see the corresponding pts for eshell is
> ephemeral (start-process), it is IMO the one that is "killed by signal 1"
> when command exit.
> You don't have the same in bash/zsh terminal because it exists in
> /dev/pts.
> It is why we have to type password at every sudo command, the pts can't
> be registered.
>
> It is my understanding, maybe I am wrong, correct me if so.

You might be correct, but then I don't understand why I get that only
with bzr commands but not with git, hg, or any other commands I've used
so far from eshell.

Bye,
Tassilo

Re: Eshell visual commands with redirection bug

[Aidan Gauland]

Tassilo Horn <tsdh@gnu.org> writes:

> So eshell should ignore a command's "visuality" if its output is
> redirected.

Done!  Should be fixed in 112898.  Let me know if you have any problems
(as usual).

--Aidan

Re: Eshell visual commands with redirection bug

[Tassilo Horn]

Aidan Gauland <aidalgol@amuri.net> writes:

Hi Aidan,

>> So eshell should ignore a command's "visuality" if its output is
>> redirected.
>
> Done!  Should be fixed in 112898.  Let me know if you have any
> problems (as usual).

Redirection still doesn't work, but now it fails differently. ;-)

  $ git diff

shows the diff visually as it's supposed to do with my configs.  Now
when I do

  $ git diff > my.patch

I don't see the diff visually as I did before your changes (good), but
the command also doesn't finish.  Using an external terminal, I can see
that now there's a symlink

    .#my.patch -> horn@thinkpad.tsdh.de.20902:1362908452

but horn@thinkpad.tsdh.de.20902:1362908452 doesn't exist.

All I can do is `C-c C-k' to kill the process.

Bye,
Tassilo

Re: Eshell visual commands with redirection bug

[Aidan Gauland]

Tassilo Horn <tsdh@gnu.org> writes:

> Aidan Gauland <aidalgol@amuri.net> writes:
>
>>> So eshell should ignore a command's "visuality" if its output is
>>> redirected.
>>
>> Done!  Should be fixed in 112898.  Let me know if you have any
>> problems (as usual).
>
> Redirection still doesn't work, but now it fails differently. ;-)
>
>   $ git diff
>
> shows the diff visually as it's supposed to do with my configs.  Now
> when I do
>
>   $ git diff > my.patch
>
> I don't see the diff visually as I did before your changes (good), but
> the command also doesn't finish.  Using an external terminal, I can see
> that now there's a symlink
>
>     .#my.patch -> horn@thinkpad.tsdh.de.20902:1362908452
>
> but horn@thinkpad.tsdh.de.20902:1362908452 doesn't exist.
>
> All I can do is `C-c C-k' to kill the process.

Sorry, I cannot reproduce this problem.  I am running GNU Emacs 24.3.1
from Debian, and Eshell from trunk, on Debian wheezy.  Which Emacs
version and OS are you running?

--Aidan

Re: Eshell visual commands with redirection bug

[Tassilo Horn]

Aidan Gauland <aidalgol@amuri.net> writes:

>>>> So eshell should ignore a command's "visuality" if its output is
>>>> redirected.
>>>
>>> Done!  Should be fixed in 112898.  Let me know if you have any
>>> problems (as usual).
>>
>> Redirection still doesn't work, but now it fails differently. ;-)
>>
>>   $ git diff
>>
>> shows the diff visually as it's supposed to do with my configs.  Now
>> when I do
>>
>>   $ git diff > my.patch
>>
>> I don't see the diff visually as I did before your changes (good), but
>> the command also doesn't finish.  Using an external terminal, I can see
>> that now there's a symlink
>>
>>     .#my.patch -> horn@thinkpad.tsdh.de.20902:1362908452
>>
>> but horn@thinkpad.tsdh.de.20902:1362908452 doesn't exist.
>>
>> All I can do is `C-c C-k' to kill the process.
>
> Sorry, I cannot reproduce this problem.  I am running GNU Emacs 24.3.1
> from Debian, and Eshell from trunk, on Debian wheezy.  Which Emacs
> version and OS are you running?

I'm running Emacs from bzr (the last 2 commits are yours) on a
GNU/Gentoo box.  Here's a recipe that works for me.

1. emacs -Q
2. In *scratch*, evaluate
   (setq eshell-visual-commands (append eshell-visual-commands '("htop"))
         eshell-visual-subcommands '(("git" "log" "show" "diff" "help")
                                     ("bzr" "log" "show" "diff" "help"))
         eshell-visual-options '(("git" "--help")))
3. M-x eshell RET
4. cd /path/to/some/git/repo
5. git log > glog.txt
6. Command doesn't return

What I can see is that this command creates a symlink .#glog.txt as I've
written before.  When I hit RET, I get an additional #glog.txt# file
that contains the first page of "git log" output:

--8<---------------cut here---------------start------------->8---
WARNING: terminal is not fully functional
^M-  (press RETURN)^M^Mcommit de57d853dc2818d34cc1257c1b191ea963c20027
Author: Tassilo Horn <horn@uni-koblenz.de>
Date:   Thu Jun 6 13:57:15 2013 +0200

    Added rule application HOFs all and any to in-place.

[...]

lines 1-32
--8<---------------cut here---------------end--------------->8---

When I hit RET a few times, it doesn't seem to change anything (except
for putting newlines in eshell).  But when I hit `q RET', the command
finally finishes and the #glog.txt# file becomes glog.txt.  The contents
are:

--8<---------------cut here---------------start------------->8---
WARNING: terminal is not fully functional
^M-  (press RETURN)^M^Mcommit de57d853dc2818d34cc1257c1b191ea963c20027
Author: Tassilo Horn <horn@uni-koblenz.de>
Date:   Thu Jun 6 13:57:15 2013 +0200

    Added rule application HOFs all and any to in-place.

[...]

lines 1-32^Mcommit a069c9ad8ee941e7bdeca394cf201fcfd40c4867
lines 2-33^MAuthor: Tassilo Horn <horn@uni-koblenz.de>
lines 3-34^MDate:   Fri May 31 17:41:17 2013 +0200
lines 4-35^M
lines 5-36^M    Some updates and use flatland.ordered.
lines 6-37^M
lines 7-38^Mcommit d9d44aa2f57a23e938a58286dd829d05fbfa5786
lines 8-39^MAuthor: Tassilo Horn <horn@uni-koblenz.de>
lines 9-40^MDate:   Mon May 6 11:28:02 2013 +0200
lines 10-41^M
lines 11-42^M    ...
lines 12-43^M
--8<---------------cut here---------------end--------------->8---

With ZSH, I get output like that when issuing

  $ TERM=dumb git log

without a redirection.  But with redirection, the log is just dumped
into a file.

I think the problem is that git uses isatty(3) to detect if its output
is piped or redirected.  If so, then it's just dumped, else it's
paginated.  So probably redirections in eshell can't be detected with
isatty, right?

Bye,
Tassilo

Re: Eshell visual commands with redirection bug

[Aidan Gauland]

Tassilo Horn <tsdh@gnu.org> writes:

> I think the problem is that git uses isatty(3) to detect if its output
> is piped or redirected.  If so, then it's just dumped, else it's
> paginated.  So probably redirections in eshell can't be detected with
> isatty, right?

Ah, yes, that makes sense.  I also figured out why I couldn't reproduce
the bug: I wasn't using less, because I have $PAGER set to "cat". >_<
Shoulda tried it with emacs -Q before shooting my mouth off.

The behaviour you described where
  $ git log > glog.txt
hangs is present in Eshell from Emacs 24.3.1, so this bug existed before
our subcommand changes.

I think the function at fault is `eshell-gather-process-output'; this
seems to be where Eshell invokes external commands.
`eshell-interactive-output-p' should tell us whether output redirection
is being done, but Emacs does not seem to provide mechanisms for
operations as low level as mucking with the stdout file descriptor.  If
this is the case, I think your "**" prefix idea is a good workaround.
(I checked that the single star "*" prefix does, in fact, means "don't
use internal command", and does not disable special treatment of
visual-commands.)

--Aidan