unofficial mirror of bug-gnu-emacs@gnu.org 
 help / color / mirror / code / Atom feed
* bug#46351: 28.0.50; Add convenient way to bypass Eshell's own pipelining
@ 2021-02-06 20:06 Sean Whitton
  2021-02-07  9:17 ` Michael Albinus
  0 siblings, 1 reply; 60+ messages in thread
From: Sean Whitton @ 2021-02-06 20:06 UTC (permalink / raw)
  To: 46351

[-- Attachment #1: Type: text/plain, Size: 1359 bytes --]

Most people who use Eshell a lot need to be aware of the limitations of
its support for shell pipelines: primarily, that all data in pipelines
has to go through Emacs buffers, and that can be slow.  If you're
running pipelines which will move a lot of data, you should use a system
shell which will be more performant.

Currently, if Eshell is your primary shell, it's not particularly
convenient to switch to using an operating system shell when you're
preparing a pipeline that you know is going to move a lot of data.  You
would need either to switch to a shell-mode buffer, or quote and escape
your Eshell input and put something like `bash -c' in front of it.

I think that it would be good to have a way to easily toggle Eshell's
own pipelining support on and off for particular commands.  I have come
up with the attached patches.  When the new functions I've defined are
enabled, you can type

    !! foo | bar 'arg' >baz >>#<buffer *scratch*>

and it will be executed by Eshell as if you had typed

    bash -c 'foo | bar '"'"'arg'"'"' >baz' >>#<buffer *scratch*>

The idea is that you can easily toggle Eshell's pipelining on and off as
appropriate to your needs just by adding and removing the "!!" prefix.

I think that the patches I've prepared are a clean implementation of
this feature that would be good to include in Emacs.

-- 
Sean Whitton

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-eshell-restore-unexpanded-input.patch --]
[-- Type: text/x-diff, Size: 2458 bytes --]

From ccc4427ff06bf6dc63840517aefbb7fb899d4b8c Mon Sep 17 00:00:00 2001
From: Sean Whitton <spwhitton@spwhitton.name>
Date: Sat, 6 Feb 2021 00:01:50 -0700
Subject: [PATCH 1/2] Add eshell-restore-unexpanded-input

* lisp/eshell/esh-mode.el (eshell-send-input): Store the original
input before running eshell-expand-input-functions.
* lisp/eshell/esh-mode.el (eshell-restore-unexpanded-input): Define
new function and register as a customization option for
eshell-input-filter-functions.
---
 lisp/eshell/esh-mode.el | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/lisp/eshell/esh-mode.el b/lisp/eshell/esh-mode.el
index d29b010ea0..6036829941 100644
--- a/lisp/eshell/esh-mode.el
+++ b/lisp/eshell/esh-mode.el
@@ -197,6 +197,7 @@ This is used by `eshell-watch-for-password-prompt'."
 ;; byte-compiler, when compiling other files which `require' this one
 (defvar eshell-mode nil)
 (defvar eshell-command-running-string "--")
+(defvar eshell-last-unexpanded-input nil)
 (defvar eshell-last-input-start nil)
 (defvar eshell-last-input-end nil)
 (defvar eshell-last-output-start nil)
@@ -631,6 +632,7 @@ newline."
 		(progn
 		  (setq input (buffer-substring-no-properties
 			       eshell-last-output-end (1- (point))))
+                  (setq eshell-last-unexpanded-input input)
 		  (run-hook-with-args 'eshell-expand-input-functions
 				      eshell-last-output-end (1- (point)))
 		  (let ((cmd (eshell-parse-command-input
@@ -664,6 +666,21 @@ newline."
 
 (custom-add-option 'eshell-input-filter-functions 'eshell-kill-new)
 
+(defun eshell-restore-unexpanded-input ()
+  "Restore the input text before `eshell-expand-input-functions' ran.
+Useful when you want to see the unexpanded input rather than the
+expanded input in the Eshell buffer, and when you want later
+entries in `eshell-input-filter-functions' to use the unexpanded
+input.  For example, you might want the unexpanded input to be
+what gets stored in the history list."
+  (setf (buffer-substring eshell-last-input-start
+                          (1- eshell-last-input-end))
+        eshell-last-unexpanded-input)
+  (eshell-update-markers eshell-last-output-end))
+
+(custom-add-option 'eshell-input-filter-functions
+                   'eshell-restore-unexpanded-input)
+
 (defun eshell-output-filter (process string)
   "Send the output from PROCESS (STRING) to the interactive display.
 This is done after all necessary filtering has been done."
-- 
2.29.2


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0002-Add-eshell-shell-command-and-eshell-expand-to-eshell.patch --]
[-- Type: text/x-diff, Size: 4354 bytes --]

From d5d48427c75555cd5cf2f6eb5b89e0c832cf7edf Mon Sep 17 00:00:00 2001
From: Sean Whitton <spwhitton@spwhitton.name>
Date: Sat, 6 Feb 2021 00:48:32 -0700
Subject: [PATCH 2/2] Add eshell-shell-command and
 eshell-expand-to-eshell-shell-command

* lisp/eshell/esh-mode.el (eshell-shell-command,
eshell-expand-to-eshell-shell-command): Define new functions.
Register eshell-expand-to-eshell-shell-command as a customization
option for eshell-expand-input-functions.
---
 lisp/eshell/esh-mode.el | 77 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 77 insertions(+)

diff --git a/lisp/eshell/esh-mode.el b/lisp/eshell/esh-mode.el
index 6036829941..b3132393a5 100644
--- a/lisp/eshell/esh-mode.el
+++ b/lisp/eshell/esh-mode.el
@@ -62,6 +62,7 @@
 (require 'esh-module)
 (require 'esh-cmd)
 (require 'esh-arg)                      ;For eshell-parse-arguments
+(require 'subr-x)
 
 (defgroup eshell-mode nil
   "This module contains code for handling input from the user."
@@ -660,6 +661,82 @@ newline."
 	       (run-hooks 'eshell-post-command-hook)
 	       (insert-and-inherit input)))))))))
 
+(defun eshell-shell-command (command)
+  "Execute COMMAND using the operating system shell."
+  (throw 'eshell-replace-command
+	 (eshell-parse-command shell-file-name
+			       (list shell-command-switch command))))
+
+(defun eshell-expand-to-eshell-shell-command (beg end)
+  "Expand Eshell input starting with '!!' to use `eshell-shell-command'.
+
+This is intended to provide a convenient way to bypass Eshell's
+own pipelining which can be slow when executing shell pipelines
+which move a lot of data.  It is also useful to avoid
+roundtripping data when running pipelines on remote hosts.
+
+To enable, add this function to `eshell-expand-input-functions'.
+
+For example, this function expands the input
+
+    !! cat *.ogg | my-cool-decoder >file
+
+to
+
+    eshell-shell-command \"cat *.ogg | my-cool-decoder >file\"
+
+Executing the latter command will not copy all the data in the
+*.ogg files into Emacs buffers, as would normally happen with
+Eshell's own pipelining.
+
+This function tries to extract Eshell-specific redirects and
+avoids passing these to the operating system shell.  For example,
+
+    !! foo | bar >>#<buffer scratch>
+
+will be expanded to
+
+    eshell-shell-command \"foo | bar\" >>#<buffer scratch>
+
+This function works well in combination with adding
+`eshell-restore-unexpanded-input' to `eshell-input-filter-functions'."
+  (save-excursion
+    (goto-char beg)
+    (when (looking-at "!!\\s-*")
+      (let ((end (copy-marker end)) ; needs to be a marker
+            redirects)
+        ;; drop the !!
+        (delete-region beg (match-end 0))
+        ;; extract Eshell-specific redirects
+        (while (search-forward-regexp "[0-9]?>+&?[0-9]?\\s-*\\S-" nil t)
+          (let ((beg (match-beginning 0)))
+            (forward-char -1) ; start from the redirect target
+            (when-let ((end (cond
+                             ;; this is a redirect to a process or a
+                             ;; buffer
+                             ((looking-at "#<")
+                              (forward-char 1)
+                              (1+ (eshell-find-delimiter ?\< ?\>)))
+                             ;; this is a redirect to a virtual target
+                             ((and (looking-at "/\\S-+")
+                                   (assoc (match-string 0)
+                                          eshell-virtual-targets))
+                              (match-end 0)))))
+              (push (buffer-substring-no-properties beg end) redirects)
+              (delete-region beg end)
+              (just-one-space))))
+        ;; wrap the remaining text and reinstate Eshell redirects
+        (let ((pipeline (string-trim
+                         (buffer-substring-no-properties beg end))))
+          (delete-region beg end)
+          (insert "eshell-shell-command ")
+          (prin1 pipeline (current-buffer))
+          (when redirects
+            (insert " " (string-join (nreverse redirects) " "))))))))
+
+(custom-add-option 'eshell-expand-input-functions
+                   'eshell-expand-to-eshell-shell-command)
+
 (defsubst eshell-kill-new ()
   "Add the last input text to the kill ring."
   (kill-ring-save eshell-last-input-start eshell-last-input-end))
-- 
2.29.2


^ permalink raw reply related	[flat|nested] 60+ messages in thread

end of thread, other threads:[~2022-01-26 13:13 UTC | newest]

Thread overview: 60+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-02-06 20:06 bug#46351: 28.0.50; Add convenient way to bypass Eshell's own pipelining Sean Whitton
2021-02-07  9:17 ` Michael Albinus
2021-02-07 19:01   ` Sean Whitton
2021-02-08 10:28     ` Michael Albinus
2021-02-08 18:07       ` Sean Whitton
2021-02-10 11:33         ` Michael Albinus
2021-12-24 21:20       ` Sean Whitton
2021-12-25 13:51         ` Michael Albinus
2021-12-25 22:45           ` Sean Whitton
2021-12-27 14:42             ` Michael Albinus
2021-12-27 18:13               ` Sean Whitton
2021-12-27 18:22                 ` Eli Zaretskii
2021-12-27 19:21                   ` Sean Whitton
2021-12-27 19:35                     ` Eli Zaretskii
2021-12-27 19:53                       ` Sean Whitton
2021-12-27 19:37                     ` Michael Albinus
2021-12-27 19:54                       ` Sean Whitton
2021-12-28  8:58                         ` Michael Albinus
2022-01-18  5:19                           ` Sean Whitton
2022-01-18  9:49                             ` Robert Pluim
2022-01-18 18:27                               ` Sean Whitton
2022-01-18 14:45                             ` Eli Zaretskii
2022-01-18 18:40                               ` Sean Whitton
2022-01-18 19:38                                 ` Eli Zaretskii
2022-01-18 23:16                                   ` Sean Whitton
2022-01-19  7:34                                     ` Eli Zaretskii
2022-01-19 20:39                                       ` Sean Whitton
2022-01-20  6:53                                         ` Eli Zaretskii
2022-01-20 22:16                                           ` Sean Whitton
2022-01-21  6:54                                             ` Eli Zaretskii
2022-01-22  0:16                                               ` Sean Whitton
2022-01-18 18:42                               ` Sean Whitton
2022-01-19 15:52                             ` Michael Albinus
2022-01-19 20:54                               ` Sean Whitton
2022-01-20 18:41                                 ` Michael Albinus
2022-01-20 22:17                                   ` Sean Whitton
2022-01-23 22:39                             ` Sean Whitton
2022-01-24  9:55                               ` Lars Ingebrigtsen
2022-01-24 14:18                               ` Michael Albinus
2022-01-24 20:32                                 ` Sean Whitton
2022-01-24 20:44                                   ` Sean Whitton
2022-01-24 20:48                                     ` Lars Ingebrigtsen
2022-01-24 21:42                                       ` Sean Whitton
2022-01-24 21:51                                         ` Lars Ingebrigtsen
2022-01-24 22:48                                           ` Sean Whitton
2022-01-24 20:46                                   ` Lars Ingebrigtsen
2022-01-25  2:39                                     ` Jim Porter
2022-01-25  5:33                                       ` bug#53518: 29.0.50; em-extpipe breaks input of sharp-quoted Lisp symbols Sean Whitton
2022-01-25  8:50                                       ` Sean Whitton
2022-01-25 12:26                                         ` Lars Ingebrigtsen
2022-01-25 16:48                                           ` Sean Whitton
2022-01-26  5:38                                             ` Jim Porter
2022-01-26 13:13                                             ` Lars Ingebrigtsen
2022-01-25 18:14                                         ` Jim Porter
2022-01-25 20:01                                           ` Sean Whitton
2022-01-25 20:52                                             ` Jim Porter
2022-01-25 22:38                                               ` Sean Whitton
2022-01-25  8:54                                       ` bug#46351: 28.0.50; Add convenient way to bypass Eshell's own pipelining Michael Albinus
2022-01-25 18:22                                         ` Jim Porter
2021-12-27 18:26                 ` Michael Albinus

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).