Re: remote execution in heterogeneous environment

Re: remote execution in heterogeneous environment

[Michael Albinus]

[Cc emacs-orgmode@gnu.org]

George Jones <eludom@gmail.com> writes:

Hi George,

> Did you ever get resolution on these?  I think I'm hitting the same
> exact problems.

In my last message, I gave some hints what to do in org's code. Since I
don't use org myself (yet), I haven't followed that.

Looks, like nobody did take the ball :-(

Maybe I'll check it again over XMas days, and maybe I can propose a
concrete patch then.

> Thanks,
> ---George Jones

Best regards, Michael.

Re: remote execution in heterogeneous environment

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

Hi,

> Looks, like nobody did take the ball :-(
>
> Maybe I'll check it again over XMas days, and maybe I can propose a
> concrete patch then.

The following patch I've committed to Emacs' trunk, it fixes the problem
as far as I could test. Maybe somebody from the org maintainers could
merge it into the org repository.

--8<---------------cut here---------------start------------->8---
~/src/emacs/lisp/org> bzr diff >123
=== modified file 'lisp/org/ChangeLog'
--- lisp/org/ChangeLog	2012-12-13 05:29:15 +0000
+++ lisp/org/ChangeLog	2012-12-20 13:09:40 +0000
@@ -1,3 +1,12 @@
+2012-12-20  Michael Albinus  <michael.albinus@gmx.de>
+
+	* ob.el (org-babel-temp-file): Fix setting of
+	`temporary-file-directory' on remote hosts.
+
+	* ob-eval.el (org-babel-shell-command-on-region): Use
+	`process-file' instead of `call-process-region'.  The latter one
+	does not work on remote hosts.
+
 2012-12-13  Bastien Guerry  <bzg@gnu.org>

 	* org-latex.el (org-export-latex-links): Escape raw path when

=== modified file 'lisp/org/ob-eval.el'
--- lisp/org/ob-eval.el	2012-09-30 15:14:59 +0000
+++ lisp/org/ob-eval.el	2012-12-20 13:06:27 +0000
@@ -134,14 +134,13 @@
 		       current-prefix-arg
 		       shell-command-default-error-buffer
 		       t)))
-  (let ((error-file
-	 (if error-buffer
-	     (make-temp-file
-	      (expand-file-name "scor"
-                                (if (featurep 'xemacs)
-                                    (temp-directory)
-                                  temporary-file-directory)))
-	   nil))
+  (let ((input-file (org-babel-temp-file "input-"))
+	(error-file (if error-buffer (org-babel-temp-file "scor-") nil))
+	(shell-file-name
+	 (if (file-executable-p
+	      (concat (file-remote-p default-directory) shell-file-name))
+	     shell-file-name
+	   "/bin/sh"))
 	exit-status)
     (if (or replace
 	    (and output-buffer
@@ -151,12 +150,14 @@
 	  ;; Don't muck with mark unless REPLACE says we should.
 	  (goto-char start)
 	  (and replace (push-mark (point) 'nomsg))
+	  (write-region start end input-file)
+	  (delete-region start end)
 	  (setq exit-status
-		(call-process-region start end shell-file-name t
-				     (if error-file
-					 (list output-buffer error-file)
-				       t)
-				     nil shell-command-switch command))
+		(process-file shell-file-name input-file
+			      (if error-file
+				  (list output-buffer error-file)
+				t)
+			      nil shell-command-switch command))
 	  ;; It is rude to delete a buffer which the command is not using.
 	  ;; (let ((shell-buffer (get-buffer "*Shell Command Output*")))
 	  ;;   (and shell-buffer (not (eq shell-buffer (current-buffer)))
@@ -175,14 +176,14 @@
 		(progn (setq buffer-read-only nil)
 		       (delete-region (max start end) (point-max))
 		       (delete-region (point-min) (min start end))
+		       (write-region (point-min) (point-max) input-file)
+		       (delete-region (point-min) (point-max))
 		       (setq exit-status
-			     (call-process-region (point-min) (point-max)
-						  shell-file-name t
-						  (if error-file
-						      (list t error-file)
-						    t)
-						  nil shell-command-switch
-						  command)))
+			     (process-file shell-file-name input-file
+					   (if error-file
+					       (list t error-file)
+					     t)
+					   nil shell-command-switch command)))
 	      ;; Clear the output buffer, then run the command with
 	      ;; output there.
 	      (let ((directory default-directory))
@@ -192,11 +193,11 @@
 		      (setq default-directory directory))
 		  (erase-buffer)))
 	      (setq exit-status
-		    (call-process-region start end shell-file-name nil
-					 (if error-file
-					     (list buffer error-file)
-					   buffer)
-					 nil shell-command-switch command)))
+		    (process-file shell-file-name nil
+				  (if error-file
+				      (list buffer error-file)
+				    buffer)
+				  nil shell-command-switch command)))
 	  ;; Report the output.
 	  (with-current-buffer buffer
 	    (setq mode-line-process
@@ -230,6 +231,9 @@
 	    ;; (kill-buffer buffer)
 	    ))))

+    (when (and input-file (file-exists-p input-file))
+      (delete-file input-file))
+
     (when (and error-file (file-exists-p error-file))
       (if (< 0 (nth 7 (file-attributes error-file)))
 	  (with-current-buffer (get-buffer-create error-buffer)

=== modified file 'lisp/org/ob.el'
--- lisp/org/ob.el	2012-10-26 14:42:05 +0000
+++ lisp/org/ob.el	2012-12-20 13:05:00 +0000
@@ -2547,18 +2547,14 @@
 Passes PREFIX and SUFFIX directly to `make-temp-file' with the
 value of `temporary-file-directory' temporarily set to the value
 of `org-babel-temporary-directory'."
-  (if (file-remote-p default-directory)
-      (make-temp-file
-       (concat (file-remote-p default-directory)
-	       (expand-file-name
-		prefix temporary-file-directory)
-	       nil suffix))
-    (let ((temporary-file-directory
+  (let ((temporary-file-directory
+	 (if (file-remote-p default-directory)
+	     (concat (file-remote-p default-directory) "/tmp")
 	   (or (and (boundp 'org-babel-temporary-directory)
 		    (file-exists-p org-babel-temporary-directory)
 		    org-babel-temporary-directory)
-	       temporary-file-directory)))
-      (make-temp-file prefix nil suffix))))
+	       temporary-file-directory))))
+      (make-temp-file prefix nil suffix)))

 (defun org-babel-remove-temporary-directory ()
   "Remove `org-babel-temporary-directory' on Emacs shutdown."
--8<---------------cut here---------------end--------------->8---

Best regards, Michael.

Re: remote execution in heterogeneous environment

[Bastien]

Hi Michael,

Michael Albinus <michael.albinus@gmx.de> writes:

> The following patch I've committed to Emacs' trunk, it fixes the problem
> as far as I could test. Maybe somebody from the org maintainers could
> merge it into the org repository.

I've already merged it in maint and master.

Thanks a lot!

-- 
 Bastien

Re: remote execution in heterogeneous environment

[Achim Gratz]

Bastien writes:
> Michael Albinus <michael.albinus@gmx.de> writes:
>
>> The following patch I've committed to Emacs' trunk, it fixes the problem
>> as far as I could test. Maybe somebody from the org maintainers could
>> merge it into the org repository.
>
> I've already merged it in maint and master.

Unfortunately it breaks testing, at least in batch mode.  See my other
posting about it.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf microQ V2.22R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: remote execution in heterogeneous environment

[Nick Dokos]

Achim Gratz <Stromeko@nexgo.de> wrote:

> Bastien writes:
> > Michael Albinus <michael.albinus@gmx.de> writes:
> >
> >> The following patch I've committed to Emacs' trunk, it fixes the problem
> >> as far as I could test. Maybe somebody from the org maintainers could
> >> merge it into the org repository.
> >
> > I've already merged it in maint and master.
> 
> Unfortunately it breaks testing, at least in batch mode.  See my other
> posting about it.
> 
> 

Isn't ``make test'' standard procedure before a commit? If not, it should be.
Otherwise, why have a test suite?

Nick

Re: remote execution in heterogeneous environment

[Bastien]

Nick Dokos <nicholas.dokos@hp.com> writes:

> Isn't ``make test'' standard procedure before a commit? If not, it should be.
> Otherwise, why have a test suite?

"Errare humanum est... <snip>"

Yes, I should have tested.

But note that this is already committed in Emacs trunk.
My assumption by looking at the code was that if tests 
breaks, then we need to fix the tests, not the patch.

Again, if someone can help fixing the tests, that'd
help me a lot!  

Thanks,

-- 
 Bastien

Re: remote execution in heterogeneous environment

[Bastien]

Achim Gratz <Stromeko@nexgo.de> writes:

> Unfortunately it breaks testing, at least in batch mode.  See my other
> posting about it.

Not sure what other posting I should read -- sorry, I can't read and
process on the threads at once.

If you know how to fix the tests, please go ahead. 

Thanks!

-- 
 Bastien

Re: remote execution in heterogeneous environment

[Achim Gratz]

Bastien writes:
>> Unfortunately it breaks testing, at least in batch mode.  See my other
>> posting about it.
>
> Not sure what other posting I should read -- sorry, I can't read and
> process on the threads at once.

The other posting is "27bcf9a70b Backport revno 111277 from Emacs trunk."

> If you know how to fix the tests, please go ahead. 

I don't.  What I found out is that at least for the first four of these
it only fils when the test is run in batch mode.  The interactive test
works.  The error message indicates that it seems to attempt to create
error-file twice, but I don't see how this might happen — the file is
created with a unique name, so something must later use the bound
variable with that name to again attempt to create the file, perhaps
when it creates a buffer for it (the test used to be in a buffer that
then gets written to file IIRC).


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf rackAttack:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: remote execution in heterogeneous environment

[Michael Albinus]

Achim Gratz <Stromeko@nexgo.de> writes:

>>> The following patch I've committed to Emacs' trunk, it fixes the problem
>>> as far as I could test. Maybe somebody from the org maintainers could
>>> merge it into the org repository.
>>
>> I've already merged it in maint and master.
>
> Unfortunately it breaks testing, at least in batch mode.  See my other
> posting about it.

Could you, please, give me a short pointer how to run the test suite? I
will check then what happened.

Note, that I don't use the org repository (yet). Maybe you must explain
from the beginning ...

Or you give me a short test program I could run for this dedicated test.

> Regards,
> Achim.

Best regards, Michael.

Re: remote execution in heterogeneous environment

[Bastien]

Hi Michael,

thanks for offering your help!

The first step is to clone the repository:

~$ git clone git://orgmode.org/org-mode.git

You are in the master branch, the one we use for latest developments.
The other branch is "maint", the one we use for bug fixing.  The next
version of Org, 7.9.3 will be released from maint, the next major 
version from master.

From the org-mode repo, simply run "make test": it will compile the
files and run the test suite.  

To debug the tests, go to the "testing/" directory, load org-*.el 
files, then find the failing tests in testing/lisp/.

Since there are many tests affected, my guess is that some top-level
testing function (either in testing/ or testing/lisp) is not up to date.

If you have any other question, let us know.

Also, if you can send patches here first next time, that'd of 
*great* help!  

Best,

-- 
 Bastien

Re: remote execution in heterogeneous environment

[Michael Albinus]

Bastien <bzg@altern.org> writes:

> Hi Michael,

Hi everybody,

> From the org-mode repo, simply run "make test": it will compile the
> files and run the test suite.  

I could reproduce the problem.

Finally, it is an error in `process-file', which wasn't detected until
now. Thanks to triggering this!

I've fixed it already in Emacs' trunk. For org-mode, I have used the
following workaround:

--8<---------------cut here---------------start------------->8---
~/src/org-mode> git diff
diff --git a/lisp/ob-eval.el b/lisp/ob-eval.el
index 23e7143..df95d0a 100644
--- a/lisp/ob-eval.el
+++ b/lisp/ob-eval.el
@@ -143,6 +143,11 @@ specifies the value of ERROR-BUFFER."
             shell-file-name
           "/bin/sh"))
        exit-status)
+    ;; There is an error in `process-file', when `error-file' exists.
+    ;; Fixed already in Emacs trunk; for the time being we apply a
+    ;; workaround.
+    (unless (file-remote-p default-directory)
+      (delete-file error-file))
     (if (or replace
            (and output-buffer
                 (not (or (bufferp output-buffer) (stringp output-buffer)))))
--8<---------------cut here---------------end--------------->8---

With that workaround, "make test" results in

--8<---------------cut here---------------start------------->8---
Ran 320 tests, 319 results as expected, 1 unexpected (2012-12-21 09:13:44+0100)
7 expected failures

1 unexpected results:
   FAILED  ob-exp/exports-inline
--8<---------------cut here---------------end--------------->8---

> Best,

Best regards, Michael.

Re: remote execution in heterogeneous environment

[Bastien]

Hi Michael,

Michael Albinus <michael.albinus@gmx.de> writes:

> Finally, it is an error in `process-file', which wasn't detected until
> now. Thanks to triggering this!
>
> I've fixed it already in Emacs' trunk. For org-mode, I have used the
> following workaround:

great, thanks for fixing this so fast, I applied your change in Org.

All tests in org's git repo now run fine, both for maint and for master.

Best,

-- 
 Bastien

Re: remote execution in heterogeneous environment

[Nick Dokos]

Michael Albinus <michael.albinus@gmx.de> wrote:

> Bastien <bzg@altern.org> writes:
> 
> > Hi Michael,
> 
> Hi everybody,
> 
> > From the org-mode repo, simply run "make test": it will compile the
> > files and run the test suite.  
> 
> I could reproduce the problem.
> 
> Finally, it is an error in `process-file', which wasn't detected until
> now. Thanks to triggering this!
> 
> I've fixed it already in Emacs' trunk. For org-mode, I have used the
> following workaround:
> 
> ~/src/org-mode> git diff
> diff --git a/lisp/ob-eval.el b/lisp/ob-eval.el
> index 23e7143..df95d0a 100644
> --- a/lisp/ob-eval.el
> +++ b/lisp/ob-eval.el
> @@ -143,6 +143,11 @@ specifies the value of ERROR-BUFFER."
>              shell-file-name
>            "/bin/sh"))
>         exit-status)
> +    ;; There is an error in `process-file', when `error-file' exists.
> +    ;; Fixed already in Emacs trunk; for the time being we apply a
> +    ;; workaround.
> +    (unless (file-remote-p default-directory)
> +      (delete-file error-file))
>      (if (or replace
>             (and output-buffer
>                  (not (or (bufferp output-buffer) (stringp output-buffer)))))
> 

I can confirm that this fixes the local dir case in the simple test I
posted previously (and the remote dir case was fixed by the process-file
fix previously).

> With that workaround, "make test" results in
> 
> Ran 320 tests, 319 results as expected, 1 unexpected (2012-12-21 09:13:44+0100)
> 7 expected failures
> 
> 1 unexpected results:
>    FAILED  ob-exp/exports-inline
> 

I don't get that failure with the current code, so maybe it has been fixed already (?):

,----
| Ran 334 tests, 334 results as expected (2012-12-21 12:27:17-0500)
| 5 expected failures
`----

Not sure why I have a different number of tests/expected failures either.

But in any case, the main problem is resolved.

Thanks for the fixes!
Nick

Re: remote execution in heterogeneous environment

[George Jones]

[-- Attachment #1: Type: text/plain, Size: 967 bytes --]

Hi.  Patch seems to work.  Tested on fresh emacs-snapshot (24.3.50.1)
install on ubuntu with org-mode freshly pulled from git (7.9.2).

#+NAME: hobbits
| bilbo | frodo |

#+name: execOnExample.com
#+BEGIN_SRC sh :var foo=hobbits  :dir /gmj@example.com: :results output
hostname

echo executed on `hostname` at `date`

echo two $foo

#+END_SRC

Similar procedure on a mac still yeilds:

FYI/Thanks,
---George Jones

Couldn't write region to
`/scpc:gmj@example.com:/var/folders/j6/j6w17khs3vl9s2_yg4rb5zsm0000gn/T/input-61683xyS',
decode using `mimencode -u -b >%s' f\
ailed

On Thu, Dec 20, 2012 at 11:39 AM, Bastien <bzg@altern.org> wrote:

> Hi Michael,
>
> Michael Albinus <michael.albinus@gmx.de> writes:
>
> > The following patch I've committed to Emacs' trunk, it fixes the problem
> > as far as I could test. Maybe somebody from the org maintainers could
> > merge it into the org repository.
>
> I've already merged it in maint and master.
>
> Thanks a lot!
>

[-- Attachment #2: Type: text/html, Size: 2494 bytes --]

Re: remote execution in heterogeneous environment

[Michael Albinus]

George Jones <eludom@gmail.com> writes:

> Hi.

Hi George,

> Similar procedure on a mac still yeilds:
>
> Couldn't write region to
> `/scpc:gmj@example.com:/var/folders/j6/j6w17khs3vl9s2_yg4rb5zsm0000gn/T/input-
> 61683xyS', decode using `mimencode -u -b >%s' f\
> ailed

Could you, please, show the exact test case? And please apply the
workaround first (see my other email).

> FYI/Thanks,
> ---George Jones

Best regards, Michael.

Re: remote execution in heterogeneous environment

[George]

Michael Albinus <michael.albinus@gmx.de> writes:

>> Similar procedure on a mac still yeilds:
>>
>> Couldn't write region to
>> `/scpc:gmj@example.com:/var/folders/j6/j6w17khs3vl9s2_yg4rb5zsm0000gn/T/input-
>> 61683xyS', decode using `mimencode -u -b >%s' f\
>> ailed
>
> Could you, please, show the exact test case? And please apply the
> workaround first (see my other email).
>

Everything works if I do, as you suggested:

  (setq temporary-file-directory "~/tmp/")

-------------------------------cut here-------------------------------
#+NAME: hobbits
| bilbo | frodo |

#+name: executeOnLocalVM
#+BEGIN_SRC sh :var foo=hobbits  :dir /george@localhost#2222: :results output
echo executed on `hostname` at `date`                                                                                                                 echo two $foo                                                                                                                                         #+END_SRC                                                                                                                                              

#+RESULTS: executeOnLocalVM
: executed on garlic at Sat Dec 22 13:25:35 UTC 2012
: two bilbo frodo

#+name: relevantVars
#+BEGIN_SRC emacs-lisp
(concat
  "temporary-file-directory " temporary-file-directory
)
#+END_SRC

#+RESULTS: relevantVars
: temporary-file-directory /tmp/

#+name: versions
#+BEGIN_SRC emacs-lisp
(concat (emacs-version)
 "
tramp version: " tramp-version
"
org version " org-version)
#+END_SRC

#+RESULTS: versions
: GNU Emacs 24.2.1 (x86_64-apple-darwin11.4.2)
:  of 2012-11-19 on tenseven-slave.macports.org
: tramp version: 2.2.6
: org version 7.9.2

-------------------------------cut here-------------------------------

Thanks,
---George Jones

Re: remote execution in heterogeneous environment

[George]

Michael Albinus <michael.albinus@gmx.de> writes:

>> Similar procedure on a mac still yeilds:
>>
>> Couldn't write region to
>> `/scpc:gmj@example.com:/var/folders/j6/j6w17khs3vl9s2_yg4rb5zsm0000gn/T/input-
>> 61683xyS', decode using `mimencode -u -b >%s' f\
>> ailed
>
> Could you, please, show the exact test case? And please apply the
> workaround first (see my other email).

Make everything works with:

  (setq temporary-file-directory "/tmp/")

Thanks,
---George Jones

Re: remote execution in heterogeneous environment

[Michael Albinus]

George <eludom@gmail.com> writes:

Hi George,

> Make everything works with:
>
>   (setq temporary-file-directory "/tmp/")

That's a misunderstanding. I meant the workaround I have proposed
yesterday on the org-mode ML, and which has been committed by Bastien.

Meanwhile, everything shall work out-of-the box. Does it for you?

> Thanks,
> ---George Jones

Best regards, Michael.

Re: remote execution in heterogeneous environment

[Michael Albinus]

George Jones <eludom@gmail.com> writes:

> Still need to 
> (setq temporary-file-directory "/tmp/") 
>
> for things to work, else I get:
>
> Couldn't write region to
> `/scpc:george@localhost#2222:/var/folders/j6/j6w17khs3vl9s2_yg4rb5zsm0000gn/T/input-
> 11249kNp', decode using `base64 -d -i >%s\
> ' failed 
>
> Test script and output, without explicitly setting
> temporary-file-directory, attached:

Indeed. I've published a patch to ob.el (as of Emacs trunk). Meanwhile,
ob.el has been renamed to ob-core.el; likely that patch was lost :-(

Bastien, could you please apply this patch (for `org-babel-temp-file')?

> Thanks,
> ---George Jones

Best regards, Michael.

Re: remote execution in heterogeneous environment

[Bastien]

Hi Michael,

Michael Albinus <michael.albinus@gmx.de> writes:

> Indeed. I've published a patch to ob.el (as of Emacs trunk). Meanwhile,
> ob.el has been renamed to ob-core.el; likely that patch was lost :-(
>
> Bastien, could you please apply this patch (for `org-babel-temp-file')?

Mhh... not sure what patch you refer to -- can you send me the patch
itself and/or a link to it?

Thanks!

-- 
 Bastien

Re: remote execution in heterogeneous environment

[Michael Albinus]

Bastien <bzg@altern.org> writes:

> Hi Michael,

Hi Bastien,

>> Indeed. I've published a patch to ob.el (as of Emacs trunk). Meanwhile,
>> ob.el has been renamed to ob-core.el; likely that patch was lost :-(
>>
>> Bastien, could you please apply this patch (for `org-babel-temp-file')?
>
> Mhh... not sure what patch you refer to -- can you send me the patch
> itself and/or a link to it?

<http://thread.gmane.org/gmane.emacs.orgmode/63586/focus=63709>

The last patch in this message. Again, the function seems to be moved to
ob-core.el now.

> Thanks!

Re: remote execution in heterogeneous environment

[Bastien]

Hi Michael,

Michael Albinus <michael.albinus@gmx.de> writes:

> <http://thread.gmane.org/gmane.emacs.orgmode/63586/focus=63709>
>
> The last patch in this message. Again, the function seems to be moved to
> ob-core.el now.

Ah okay, applied, thanks!

-- 
 Bastien

Re: remote execution in heterogeneous environment

[George Jones]

[-- Attachment #1: Type: text/plain, Size: 444 bytes --]

Should a git pull of development version of org pull the right patch?

Thanks,
---george jones
On Dec 23, 2012 11:47 AM, "Bastien" <bzg@altern.org> wrote:

> Hi Michael,
>
> Michael Albinus <michael.albinus@gmx.de> writes:
>
> > <http://thread.gmane.org/gmane.emacs.orgmode/63586/focus=63709>
> >
> > The last patch in this message. Again, the function seems to be moved to
> > ob-core.el now.
>
> Ah okay, applied, thanks!
>
> --
>  Bastien
>

[-- Attachment #2: Type: text/html, Size: 908 bytes --]

Re: remote execution in heterogeneous environment

[Bastien]

George Jones <eludom@gmail.com> writes:

> Should a git pull of development version of org pull the right patch?

Yes.  

Simply ~$ git clone git://orgmode.org/org-mode.git (our master branch 
is the development branch.)

HTH,

-- 
 Bastien

Re: remote execution in heterogeneous environment

[George Jones]

[-- Attachment #1: Type: text/plain, Size: 406 bytes --]

Success.   No setting of temporary-file-directory needed.

Thanks,
---George



On Sun, Dec 23, 2012 at 2:54 PM, Bastien <bzg@altern.org> wrote:

> George Jones <eludom@gmail.com> writes:
>
> > Should a git pull of development version of org pull the right patch?
>
> Yes.
>
> Simply ~$ git clone git://orgmode.org/org-mode.git (our master branch
> is the development branch.)
>
> HTH,
>
> --
>  Bastien
>

[-- Attachment #2: Type: text/html, Size: 1001 bytes --]

Re: remote execution in heterogeneous environment

[Michael Albinus]

George Jones <eludom@gmail.com> writes:

> Success. No setting of temporary-file-directory needed.

Great! Thanks for your inisistent testing.

> Thanks,
> ---George

Best regards, Micvhael.

Re: [BUG] remote execution in heterogeneous environment

[Achim Gratz]

Michael Albinus writes:
> +  (let ((input-file (org-babel-temp-file "input-"))
> +	(error-file (if error-buffer (org-babel-temp-file "scor-") nil))
> +	(shell-file-name
> +	 (if (file-executable-p
> +	      (concat (file-remote-p default-directory) shell-file-name))
> +	     shell-file-name
> +	   "/bin/sh"))

The hard-coded /bin/sh breaks testing with NT-Emacs and using Cygwin as
the backend because of the different path conventions in both
environments.  This is a hack, but it works when setting SHELL=dash,
note that there are no path components in there and even if your test
would fail: NTEmacs knows nothing about Cygwin's path.  Can you please
arrange that shell-file-name is used without alteration when the
execution is local?  Even for remote execution hardcoding /bin/sh seems
a bit heavy-handed to me…


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

Re: [BUG] remote execution in heterogeneous environment

[Michael Albinus]

Achim Gratz <Stromeko@nexgo.de> writes:

>> +  (let ((input-file (org-babel-temp-file "input-"))
>> +	(error-file (if error-buffer (org-babel-temp-file "scor-") nil))
>> +	(shell-file-name
>> +	 (if (file-executable-p
>> +	      (concat (file-remote-p default-directory) shell-file-name))
>> +	     shell-file-name
>> +	   "/bin/sh"))
>
> The hard-coded /bin/sh breaks testing with NT-Emacs and using Cygwin as
> the backend because of the different path conventions in both
> environments.  This is a hack, but it works when setting SHELL=dash,
> note that there are no path components in there and even if your test
> would fail: NTEmacs knows nothing about Cygwin's path.  Can you please
> arrange that shell-file-name is used without alteration when the
> execution is local?  Even for remote execution hardcoding /bin/sh seems
> a bit heavy-handed to me…

What about this:

--8<---------------cut here---------------start------------->8---
--- a/lisp/ob-eval.el
+++ b/lisp/ob-eval.el
@@ -137,11 +137,17 @@ specifies the value of ERROR-BUFFER."
                       t)))
   (let ((input-file (org-babel-temp-file "input-"))
        (error-file (if error-buffer (org-babel-temp-file "scor-") nil))
+       ;; Unfortunately, `executable-find' does not support file name
+       ;; handlers.  Therefore, we could use it in the local case
+       ;; only.
        (shell-file-name
-        (if (file-executable-p
-             (concat (file-remote-p default-directory) shell-file-name))
-            shell-file-name
-          "/bin/sh"))
+        (cond ((and (not (file-remote-p default-directory))
+                    (executable-find shell-file-name))
+               shell-file-name)
+              ((file-executable-p
+                (concat (file-remote-p default-directory) shell-file-name))
+               shell-file-name)
+              ("/bin/sh")))
        exit-status)
     ;; There is an error in `process-file' when `error-file' exists.
     ;; This is fixed in Emacs trunk as of 2012-12-21; let's use this
--8<---------------cut here---------------end--------------->8---

The use of "/bin/sh" as fallback should be OK. In the local case, the
first cond clause shall be selected (otherwise there is something really
wrong). And for remote execution, "/bin/sh" should exist execept in case
of remote MS Windows or Android devices. Once you want run shells from
org there, we could even tweak this :-)

> Regards,
> Achim.

Best regards, Michael.

Re: [BUG] remote execution in heterogeneous environment

[Achim Gratz]

Michael Albinus writes:
> What about this:
[...]

Looks good, I can test it tomorrow.

> The use of "/bin/sh" as fallback should be OK. In the local case, the
> first cond clause shall be selected (otherwise there is something really
> wrong).

Yes.  But on Windows there is always something wrong… :-)

> And for remote execution, "/bin/sh" should exist execept in case
> of remote MS Windows or Android devices. Once you want run shells from
> org there, we could even tweak this :-)

I was thinking towards being able to use a restricted shell.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf rackAttack V1.04R1:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [BUG] remote execution in heterogeneous environment

[Eric Schulte]

Achim Gratz <Stromeko@nexgo.de> writes:

> Michael Albinus writes:
>> What about this:
> [...]
>
> Looks good, I can test it tomorrow.
>
>> The use of "/bin/sh" as fallback should be OK. In the local case, the
>> first cond clause shall be selected (otherwise there is something really
>> wrong).
>
> Yes.  But on Windows there is always something wrong… :-)
>
>> And for remote execution, "/bin/sh" should exist execept in case
>> of remote MS Windows or Android devices. Once you want run shells from
>> org there, we could even tweak this :-)
>
> I was thinking towards being able to use a restricted shell.
>

I'm jumping in late here a without context, so forgive me if this
suggestion is not useful.

Do either the `org-babel-sh-command' or the :shebang header argument
affect this behavior?  If not should they, or should we add a new header
argument (say :remote-shell) to allow for customization of the choice of
shell for remote execution?

-- 
Eric Schulte
http://cs.unm.edu/~eschulte

Re: [BUG] remote execution in heterogeneous environment

[Michael Albinus]

Achim Gratz <Stromeko@nexgo.de> writes:

>> The use of "/bin/sh" as fallback should be OK. In the local case, the
>> first cond clause shall be selected (otherwise there is something really
>> wrong).
>
> Yes.  But on Windows there is always something wrong… :-)

Again: under Windows, on the local host, `shell-file-name' should do. For
the remote case you are right, but I doubt that any org user has ever
tried to run a remote Windows shell via Tramp.

>> And for remote execution, "/bin/sh" should exist execept in case
>> of remote MS Windows or Android devices. Once you want run shells from
>> org there, we could even tweak this :-)
>
> I was thinking towards being able to use a restricted shell.

`shell-file-name' is your friend. "/bin/sh" is just the last resort. If
you dislike it, you could raise an error instead :-)

> Regards,
> Achim.

Best regards, Michael.

Re: [BUG] remote execution in heterogeneous environment

[Achim Gratz]

Michael Albinus <michael.albinus <at> gmx.de> writes:
> What about this:

Works a treat.  Please install!


Regards,
Achim.

Re: [BUG] remote execution in heterogeneous environment

[Michael Albinus]

Achim Gratz <Stromeko@NexGo.DE> writes:

> Works a treat.  Please install!

I have no write access to the org repo. Maybe somebody else could install?

> Regards,
> Achim.

Best regards, Michael.

Re: [BUG] remote execution in heterogeneous environment

[Bastien]

Hi Michael,

Michael Albinus <michael.albinus@gmx.de> writes:

> Achim Gratz <Stromeko@NexGo.DE> writes:
>
>> Works a treat.  Please install!
>
> I have no write access to the org repo. Maybe somebody else could
> install?

Please send me your public key so that I can give your write access.
Otherwise can you just resent the patch as a git patch with a commit
message have an Emacs-ready ChangeLog entry?

Thanks!

-- 
 Bastien

Re: [BUG] remote execution in heterogeneous environment

[Michael Albinus]

[-- Attachment #1: Type: text/plain, Size: 351 bytes --]

Bastien <bzg@altern.org> writes:

> Hi Michael,

Hi Bastien,

> Please send me your public key so that I can give your write access.
> Otherwise can you just resent the patch as a git patch with a commit
> message have an Emacs-ready ChangeLog entry?

I don't believe I need permanent write access. Patch appended.

> Thanks!

Best regards, Michael.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ob-eval.el-org-babel-shell-command-on-region-Use.patch --]
[-- Type: text/x-diff, Size: 1397 bytes --]

From a39b35c3967ac7c95e3442434a3ca44cba54fca5 Mon Sep 17 00:00:00 2001
From: Michael Albinus <michael.albinus@gmx.de>
Date: Wed, 9 Jan 2013 13:48:30 +0100
Subject: [PATCH] * ob-eval.el (org-babel-shell-command-on-region): Use
 `executable-find' for local `shell-file-name'

---
 lisp/ob-eval.el |   14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/lisp/ob-eval.el b/lisp/ob-eval.el
index 8764920..8018111 100644
--- a/lisp/ob-eval.el
+++ b/lisp/ob-eval.el
@@ -137,11 +137,17 @@ specifies the value of ERROR-BUFFER."
 		       t)))
   (let ((input-file (org-babel-temp-file "input-"))
 	(error-file (if error-buffer (org-babel-temp-file "scor-") nil))
+	;; Unfortunately, `executable-find' does not support file name
+	;; handlers.  Therefore, we could use it in the local case
+	;; only.
 	(shell-file-name
-	 (if (file-executable-p
-	      (concat (file-remote-p default-directory) shell-file-name))
-	     shell-file-name
-	   "/bin/sh"))
+	 (cond ((and (not (file-remote-p default-directory))
+		     (executable-find shell-file-name))
+		shell-file-name)
+	       ((file-executable-p
+		 (concat (file-remote-p default-directory) shell-file-name))
+		shell-file-name)
+	       ("/bin/sh")))
 	exit-status)
     ;; There is an error in `process-file' when `error-file' exists.
     ;; This is fixed in Emacs trunk as of 2012-12-21; let's use this
-- 
1.7.10.4

Re: [BUG] remote execution in heterogeneous environment

[Bastien]

Hi Michael,

Michael Albinus <michael.albinus@gmx.de> writes:

> I don't believe I need permanent write access. Patch appended.

Applied, thanks!

Best,

-- 
 Bastien