Re: Executing org shell blocks on remote machine over ssh

[Subhan Michael Tindall <SubhanT@familycareinc.org>]

> -----Original Message-----
> From: emacs-orgmode-bounces+subhant=familycareinc.org@gnu.org
> [mailto:emacs-orgmode-bounces+subhant=familycareinc.org@gnu.org] On
> Behalf Of David Bjergaard
> Sent: Tuesday, November 18, 2014 10:23 AM
> To: Ista Zahn
> Cc: Brett Viren; emacs-orgmode Mailinglist
> Subject: Re: [O] Executing org shell blocks on remote machine over ssh
> 
> Ista Zahn <istazahn@gmail.com> writes:
> 
> > On Tue, Nov 18, 2014 at 10:43 AM, Brett Viren <bv@bnl.gov> wrote:
> >> David Bjergaard <david.b@duke.edu> writes:
> >>
> >>> I use org mode as a lab notebook.  I write org-src blocks to keep
> >>> track of tasks I do at the command line, and then I copy paste them
> >>> into the terminal.  I would really like to hit "C-c C-c" on the
> >>> source block and have it executed on the remote machine.  I know
> >>> that you can specify the remote machine according to [1], however
> >>> the software I use requires a fairly complicated setup to get going.
> >>
> >> Is it just complicated, or is it also prohibitively long-running?
> >>
> >> If just the former, you could maybe bundle the setup into some shell
> >> script and source it in each of your sh source blocks.  Eg:
> >>
> >> #+BEGIN_SRC sh :results output :dir /ssh:lycastus:/home/bviren
> >>   /bin/pwd
> >>   echo $HOSTNAME
> >>   ls -l foo.sh
> >>   echo "---"
> >>   cat foo.sh
> >>   echo "---"
> >>   source ./foo.sh
> >>   echo $FOO
> >> #+END_SRC
> >>
> >> #+RESULTS:
> >> : /home/bviren
> >> : lycastus
> >> : -rw-rw-r-- 1 bviren bviren 16 Nov 18 10:27 foo.sh
> >> : ---
> >> : export FOO=bar
> >> :
> >> : ---
> >> : bar
> >>
> >>
> >>
> >> If the setup is purely environmental, and it takes a long time to
> >> perform, maybe you could do the set up once and then cache the
> >> resulting environment using the output of "env".
> >
> > I guess I'm missing something (like why the OP want's to run a shell
> > in a separate window), but why not just
> >
> > #+BEGIN_SRC sh :results output :dir /ssh:lycastus:/home/bviren :session
> *shell*
> >   /bin/pwd
> >   echo $HOSTNAME
> >   ls -l foo.sh
> >   echo "---"
> >   cat foo.sh
> >   echo "---"
> >   source ./foo.sh
> >   echo $FOO
> > #+END_SRC
> >
> > ?
> >
> > Best,
> > Ista
> >>
> >>
> >> -Brett.
> Hi Ista, Brett,
> 
> Thanks for the hints, I'll try these.
> Some clarification:
> >> If the setup is purely environmental, and it takes a long time to
> >> perform, maybe you could do the set up once and then cache the
> >> resulting environment using the output of "env".
> Unfortunately it takes a long time to set up, and its complicated (I have to
> initialize different versions of the software depending on which project I'm
> working on.) It can take up to 10 seconds to restore a saved environment,
> and it takes some setting up bootstrap the restoration command.
> > I guess I'm missing something (like why the OP want's to run a shell
> > in a separate window), but why not just
> I want the separate window because once the environment is set up, I also
> jump around a lot.  I record the pieces of shell script that are important for
> reproducing results in the notebook.  The snippets are then recycled across
> many sessions, and different pieces are used at different times (some are
> even used with different pieces of software).
> 
> What I would like is to log into the remote machine, set up the software, and
> then have a way to tell emacs to send the org-src block to the set up
> environment so that I can build up the parts I need at that moment.
> 
> I know this is working "against the grain" of the literate programming
> paradigm where the document and the source code are coupled, and
> tangling the document produces a program that can be executed.  I'm just
> wondering if its possible.  If not that's fine.  Really I'm just trying to save
> myself a copy-paste (and the associated issues with it getting recorded in my
> .bash_history).
> 
> Cheers,
> 
>     Dave

It's been quite a while since I used one (or even did much shell programming), so my details are a bit fuzzy. But, I think your solution is in Unix (I'm assuming that's what your using), not emacs itself.

Take a look at the concept of 'named pipes'

If you've got a shared file system or can remote mount, a process like this would probably work:

Create 2 named pipes
One you can write to on system A (eg. my_sys_a_pipe), with a small script on system B that reads my_sys_a_pipe and executes the commands it grabs
Second named pipe is created and stdout on system B is redirected to it, then system A listens to it & feeds what is received to stdout (my_sys_b_pipe)

Echo "my list of commands" > my_sys_a_pipe
echo < my_sys_b_pipe  (see below, probably want a loop here)

You probably also need to include a tag of some sort (echo "sys_b_done" or similar) as the last command and then run a read/echo loop on my_sys_b_pipe to terminate the driver script and make sure it returns to emacs.

Hopefully this isn't too confusing, unfortunately I don't have time to go back and look up my syntax, but I think conceptually it might get you down the road

This message is intended for the sole use of the individual and entity to which it is addressed and may contain information that is privileged, confidential and exempt from disclosure under applicable law. If you are not the intended addressee, nor authorized to receive for the intended addressee, you are hereby notified that you may not use, copy, disclose or distribute to anyone the message or any information contained in the message. If you have received this message in error, please immediately advise the sender by reply email and delete the message.  Thank you.