Using exec functions to simplify session code

Using exec functions to simplify session code

[Ivar Fredholm]

[-- Attachment #1: Type: text/plain, Size: 649 bytes --]

I have been browsing the orgmode code and it feels like we have to do a lot to implement sessions. At least for Python or Julia, I am wondering if we could instantiate an interpreter, and send the code as a single 'exec' or 'include' command. Other languages like R and groovy also have facilities to run code from a file. I believe this would reduce the amount of code needing maintenance, as we would only need to manipulate the string to pass to the interpreter based on the language. If this has been considered before, could someone explain why this approach was not used?
Best, Guacho

Sent with [Proton Mail](https://proton.me/) secure email.

[-- Attachment #2: Type: text/html, Size: 1279 bytes --]

Re: Using exec functions to simplify session code

[Tim Cross]

Ivar Fredholm <freddyholms@protonmail.com> writes:

> I have been browsing the orgmode code and it feels like we have to do a lot to implement sessions. At least for Python
> or Julia, I am wondering if we could instantiate an interpreter, and send the code as a single 'exec' or 'include'
> command. Other languages like R and groovy also have facilities to run code from a file. I believe this would reduce the
> amount of code needing maintenance, as we would only need to manipulate the string to pass to the interpreter based
> on the language. If this has been considered before, could someone explain why this approach was not used?
> Best, Guacho
>

I agree that supporting session can be complex, especially with some
languages. The big limitation I can see with your suggestion (assuming I
understand what your proposing) is that you may not necessarily want to
execute all the blocks at once or only execute a block once. 

For example, I might have an org file with 4 code blocks which I want to
all run in the same session. However, I might want to execute each block
interactively, not run all four blocks sequentially all in one batch. I
might execute block 1, then based on the result/output form that block,
execute block 2 or 3, but maybe not both or perhaps I will execute block
2 twice before executing block 3 etc. 

If you adopt your approach, we would loose that flexibility. The session
argument would essentially boil down to execute all code blocks in this
file within the same batch rather than execute each one in a separate
environment/instance. While this could be a valid interpretation of
session, we would need to be sure it was one which all users found
agreeable. It is possible, for stability and maintainability, a new
definition of session would be acceptable, but it is a change with
implications that would need to be managed carefully. 

Re: Using exec functions to simplify session code

[Ivar Fredholm]

Hi, Tim,
Let's say in your example that we are executing Python cells with the same session. When you execute block 1, the routines in org-babel-execute-src-block process the code into something that can be inserted into a file (say 'tmp_code.py') and be run from the command line with python3 tmp_code.py.

At this point, Babel calls org-babel-execute:python, which decides, based on the 'session' header, whether or not to call python3 tmp_code.py or to feed each line in tmp_code.py into the session interpreter. What I am wondering is if we could replace the code for the latter option (in ob-comint.el) by just sending the interpreter the command 'exec(open("tmp_code.py").read())'.

Now you can still run block 2 or 3, or block 2 twice, or neither. I think this would simplify a lot of the comint code. All we would need to do is change how we implement the 'exec(open...' code in other languages, for instance, I believe in julia this could be done by an 'include'.

Best, Guacho





Sent with Proton Mail secure email.

------- Original Message -------
On Thursday, June 23rd, 2022 at 7:53 PM, Tim Cross <theophilusx@gmail.com> wrote:


> Ivar Fredholm freddyholms@protonmail.com writes:
>
> > I have been browsing the orgmode code and it feels like we have to do a lot to implement sessions. At least for Python
> > or Julia, I am wondering if we could instantiate an interpreter, and send the code as a single 'exec' or 'include'
> > command. Other languages like R and groovy also have facilities to run code from a file. I believe this would reduce the
> > amount of code needing maintenance, as we would only need to manipulate the string to pass to the interpreter based
> > on the language. If this has been considered before, could someone explain why this approach was not used?
> > Best, Guacho
>
>
> I agree that supporting session can be complex, especially with some
> languages. The big limitation I can see with your suggestion (assuming I
> understand what your proposing) is that you may not necessarily want to
> execute all the blocks at once or only execute a block once.
>
> For example, I might have an org file with 4 code blocks which I want to
> all run in the same session. However, I might want to execute each block
> interactively, not run all four blocks sequentially all in one batch. I
> might execute block 1, then based on the result/output form that block,
> execute block 2 or 3, but maybe not both or perhaps I will execute block
> 2 twice before executing block 3 etc.
>
> If you adopt your approach, we would loose that flexibility. The session
> argument would essentially boil down to execute all code blocks in this
> file within the same batch rather than execute each one in a separate
> environment/instance. While this could be a valid interpretation of
> session, we would need to be sure it was one which all users found
> agreeable. It is possible, for stability and maintainability, a new
> definition of session would be acceptable, but it is a change with
> implications that would need to be managed carefully.

Re: Using exec functions to simplify session code

[Ihor Radchenko]

Ivar Fredholm <freddyholms@protonmail.com> writes:

> At this point, Babel calls org-babel-execute:python, which decides, based on the 'session' header, whether or not to call python3 tmp_code.py or to feed each line in tmp_code.py into the session interpreter. What I am wondering is if we could replace the code for the latter option (in ob-comint.el) by just sending the interpreter the command 'exec(open("tmp_code.py").read())'.

ob-comint.el knows nothing about the interpreter. So, what you are
describing should be implemented by the corresponding ob-*.el backends.
They know all the details about the interpreter being used.

Also, some interpreters behave differently if they run commands
line-by-line vs. inside script file. In particular, the script commands
may be echoed before their output in some cases (e.g. bash scripts and
.bat files, AFAIR).

Best,
Ihor