Re: org-babel -- Improper syntax error in session mode?

[Herbert Sitz <hsitz@nwlink.com>]

Eric Schulte <schulte.eric <at> gmail.com> writes:

> 
> Babel sessions explicitly are thin wrappers around the interactive mode
> of the language in question (whatever that may be).  That is why Babel
> happily doesn't implement sessions for all languages, the contract
> simply is that if a language supports interactive evaluation, Babel will
> try to allow access to that functionality.
> 

That the Babel session is only a "thin" wrapper is not clear at all from the
docs.  The docs do mention 'interactive session'--and even that the output may
be slightly different from the non-session mode.  The docs don't mention
anything about the user needing to do extra preparation to avoid syntax errors
in evaluation of already valid code.  If I take an existing Org block of
non-session code and stick a :session on I don't expect to get syntax errors.
 If that's the expected behavior then it needs to be emphasized more in the
docs, otherwise most users will think something is broken in Org.

> If the contract was rather simply "Babel provides session based
> evaluation", then we'd be on the hook for implementing session
> evaluation where it doesn't already exist and may not make sense (e.g.,
> C, ditaa), and normalizing everywhere else as you're suggesting here.
> Which would in turn requires us to spell out Babel-specific semantics of
> session based evaluation -- something we have thus-far avoided.

Again, the docs reference 'Session based evaluation' in big bold letters and
don't indicate the thinness of the wrapper.

> 
> Some positive points *for* the "thin wrapper" approach include;
> 
> 1. It is the simplest to implement
>    - easier to implement support for new language
>    - easier to maintain -- especially if interactive tools change
>    - easier to reason about -- thinking about bug fixing here
>    - works with alternate back-ends e.g., ipython
> 

I think I have a solution for Python that gives full "session-based evaluation"
and which is better than the current "thin wrapper" for all four concerns above
(See bottom of my message for an example.)  This same approach, I think, would
work for Ruby, but I would need to do a little digging to find exact method. 
Let me know if you want me to do that digging.

> 2. It is the least surprising.  I'd maintain that for users familiar
>    with using Python sessions this behavior is the easiest to quickly
>    understand and use.  As opposed to if we did something fancy (even if
>    it was an improvement) it would be another environment for language
>    users to have to familiarize themselves with -- they'd have both
>    normal session rules and babel session rules

As a sometimes Python user it had me totally confused.  I don't think of
"session-based" as having any essential tie to "interactive shell" at all.  Why
would I worry about state being maintained between statements in an org block? 
State is already maintained between statements in non-session Org blocks. The
only difference with session-based blocks is the starting state may not be
"fresh".

When I look at Org and think of "session-based" blocks, I see a potential big
benefit in having multiple source-blocks throughout my document share the same
session.  That way I don't need to use Org's (clever and useful but) somewhat
clunky and inflexible method of passing data explicitly.  In statistics-based
docs, for example, I can imagine that the sheer amount of data would make using
Org's explicit variable-passing method unfeasible.  Using
session-based-evaluation is at the same time both simpler and more heavy-duty.

I never thought the use of session-based code blocks as something like a
scratchpad was much more than toy. If you're typing in code that's not intended
to be an integral part of an Org doc then you could just as well jump to the
shell and enter it there.

Here's an example of how to run a block of Python code of any length in the
interactive session and save its output in a file.  Note that this is _not_
interactive, even though the command runs in the interactive shell's session:

Start with this block of code:
-------------------------------------
#+begin_src python :results output :session mypy 
    x = 1
    y = 1
    z = 1
    for i in range(1,2):
        x = x + i
        print x
    
        for y in range(10,11):
            print y
    # comment here
        for z in range(5,6):
            print z
            while y > 0:
                print y
                y=y-1

    print "Did it work?"
#+end_src
------------------------------------

Strip out the block of empty space at the left side and save in a file in
current directory (say, 'pyfile.py') with a couple commands prepended and
appended:
-----------------------------
# beginning of file
# prepend code to redirect stdout to file
import sys
saveout = sys.stdout
fsock = open('pyfile-output.log','w')
sys.stdout = fsock

# code block here, with beginning space block removed
x = 1
y = 1
z = 1
for i in range(1,2):
    x = x + i
    print x

    for y in range(10,11):
        print y
# comment here
    for z in range(5,6):
        print z
        while y > 0:
            print y
            y=y-1
print "Did it work?"
# end of the org source code block

# return stdout to what it was
sys.stdout = saveout
# and close the log file
fsock.close()
# end of file
----------------------------

Now you can go into the python shell and execute the code in pyfile.py:

>>> execfile('pyfile.py')

The output will be saved in the log file in the working directory.  I think it
should be exactly the same as the output from the non-session based code
(assuming the session-based code started from a fresh session).

-- Herb