Re: ob-python newline & indentation behavior

[Jack Kamm <jackkamm@gmail.com>]

[-- Attachment #1.1: Type: text/plain, Size: 2849 bytes --]

Hello,

Attached please find my revised patch based on Kyle's feedback. I
simplified the behavior to send the whole block to the tmpfile (including
the lastline), but restricted this change to ":results output" only
(":results value" retains its old, broken behavior). I also added a test
and fixed minor style issues.

I haven't heard back from the FSF yet, any idea how long it takes to hear
from them?

Given how broken ":session :results value" is, and the difficulty in
implementing it correctly, I agree with Kyle's suggestion to remove it.
Should we start another thread to discuss this?

Jack

On Tue, Nov 21, 2017 at 8:28 AM, Jack Kamm <jackkamm@gmail.com> wrote:

> Yes, I'm starting to see now how difficult it is to properly support
> ":session :results value". I would vote to remove it from ob-python...
>
> I think the patch still improves ":session :results output" so I will
> simplify it and restrict to that case, leaving ":session :results value"
> unchanged for now.
>
> Sorry for sending this twice Kyle, forgot to reply all.
>
> On 21 Nov 2017 4:04 am, "Kyle Meyer" <kyle@kyleam.com> wrote:
>
>> Jack Kamm <jackkamm@gmail.com> writes:
>>
>> > In response to this:
>> >
>> >> I can't think of a good solution, though.  Stepping back a bit, I think
>> >> it's unfortunate that python blocks handle ":results value" differently
>> >> depending on whether the block is hooked up to a session or not.  For
>> >> non-sessions, you have to use return.  Using the same approach
>> >> (org-babel-python-wrapper-method) for ":session :results value", we
>> >> could then get the return value reliably, but the problem with this
>> >> approach is that any variables defined in a ":results value" block
>> >> wouldn't be defined in the session after executing the block because
>> the
>> >> code is wrapped in a function.
>> >
>> > How about if we used the "globals()" and "locals()" functions in Python?
>> >
>> > Something like this at the end of the wrapper block, before return:
>> >
>> > for k, v in locals().items():
>> >     globals()[k] = v
>>
>> Hmm, placing that code "before return" is a problem.  Like with
>> non-session ":results value" blocks, the user would be responsible for
>> inserting the return (or even multiple return's), so we can't know where
>> to insert the above code without parsing the block :/
>>
>> > Another bug with the current approach is that it breaks if common idioms
>> > like "for _ in range(10)" are used. ("_" is used to inspect the last
>> output
>> > of the shell, an obscure feature I hadn't known about until now).
>>
>> Right.  Also, IIRC the built-in interactive python and ipython treat
>> multiline blocks differently.  With
>>
>>     if True:
>>         "ipython ignores my existence"
>>
>> the built-in shell binds "_" to the string's value, but ipython doesn't.
>>
>> --
>> Kyle
>>
>

[-- Attachment #1.2: Type: text/html, Size: 4426 bytes --]

[-- Attachment #2: 0001-ob-python.el-fix-session-results-output-multiline-be.patch --]
[-- Type: text/x-patch, Size: 3376 bytes --]

From e46458004b83b034cb1388e707e866e84809b420 Mon Sep 17 00:00:00 2001
From: Jack Kamm <jackkamm@gmail.com>
Date: Sun, 26 Nov 2017 08:00:29 +0000
Subject: [PATCH] ob-python.el: fix :session :results output multiline behavior

* ob-python.el (orb-babel-python-evaluate-session
org-babel-python--exec-tmpfile): When :session :results output, send
multiline code blocks to tmpfile and execute in Python with exec()
* test-ob-python.el (test-ob-python/session-multiline): test for
:session with multiple lines and indentation
---
 lisp/ob-python.el              | 38 ++++++++++++++++++++++++++++----------
 testing/lisp/test-ob-python.el | 16 ++++++++++++++++
 2 files changed, 44 insertions(+), 10 deletions(-)

diff --git a/lisp/ob-python.el b/lisp/ob-python.el
index 60ec5fa47..3af5fb6bb 100644
--- a/lisp/ob-python.el
+++ b/lisp/ob-python.el
@@ -306,16 +306,19 @@ last statement in BODY, as elisp."
          (results
           (pcase result-type
             (`output
-             (mapconcat
-              #'org-trim
-              (butlast
-               (org-babel-comint-with-output
-                   (session org-babel-python-eoe-indicator t body)
-                 (funcall input-body body)
-                 (funcall send-wait) (funcall send-wait)
-                 (insert org-babel-python-eoe-indicator)
-                 (funcall send-wait))
-               2) "\n"))
+	     (let ((body (if (string-match-p ".\n+." body) ; Multiline
+			     (org-babel-python--replace-body-tmpfile body)
+			   body)))
+	       (mapconcat
+		#'org-trim
+		(butlast
+		 (org-babel-comint-with-output
+		     (session org-babel-python-eoe-indicator t body)
+		   (funcall input-body body)
+		   (funcall send-wait) (funcall send-wait)
+		   (insert org-babel-python-eoe-indicator)
+		   (funcall send-wait))
+		 2) "\n")))
             (`value
              (let ((tmp-file (org-babel-temp-file "python-")))
                (org-babel-comint-with-output
@@ -340,6 +343,21 @@ last statement in BODY, as elisp."
       (substring string 1 -1)
     string))
 
+(defconst org-babel-python--exec-tmpfile
+  (concat
+   "__org_babel_python_fname = '%s'; "
+   "__org_babel_python_fh = open(__org_babel_python_fname); "
+   "exec(compile("
+   "__org_babel_python_fh.read(), __org_babel_python_fname, 'exec'"
+   ")); "
+   "__org_babel_python_fh.close()"))
+
+(defun org-babel-python--replace-body-tmpfile (body)
+  "Place BODY in tmpfile, and return string to exec the tmpfile."
+  (let ((tmp-file (org-babel-temp-file "python-")))
+    (with-temp-file tmp-file (insert body))
+    (format org-babel-python--exec-tmpfile tmp-file)))
+
 (provide 'ob-python)
 
 
diff --git a/testing/lisp/test-ob-python.el b/testing/lisp/test-ob-python.el
index d9fca220f..e77f9f36c 100644
--- a/testing/lisp/test-ob-python.el
+++ b/testing/lisp/test-ob-python.el
@@ -101,6 +101,22 @@ return x
     (should (equal '(("col") ("a") ("b"))
 		   (org-babel-execute-src-block)))))
 
+(ert-deftest test-ob-python/session-multiline ()
+  (run-python)
+  (sleep-for 0 10)
+  (org-test-with-temp-text "
+#+begin_src python :session :results output
+  foo = 0
+  for _ in range(10):
+      foo += 1
+
+      foo += 1
+
+  print(foo)
+#+end_src"
+   (org-babel-next-src-block)
+   (should (equal "20" (org-babel-execute-src-block)))))
+
 (provide 'test-ob-python)
 
 ;;; test-ob-python.el ends here
-- 
2.15.0