bug#49822: master e32c7d2: Change Python eval to send directly instead of using temporary files

["Barton, Mark" <Mark.Barton@disney.com>]

[-- Attachment #1: Type: text/plain, Size: 5445 bytes --]



On Sep 7, 2021, at 10:37 AM, Augusto Stoffel <arstoffel@gmail.com<mailto:arstoffel@gmail.com>> wrote:

On Mon,  6 Sep 2021 at 13:53, Andreas Schwab <schwab@linux-m68k.org<mailto:schwab@linux-m68k.org>> wrote:

On Sep 06 2021, Michael Albinus wrote:

Andreas Schwab <schwab@linux-m68k.org<mailto:schwab@linux-m68k.org>> writes:

Okay, so is the TTY line length limit of the OS available in Lisp
somewhere?  Otherwise, can we at least trust that 1024 is the universal
lower bound?

On systems with a POSIX shell:

(let ((default-directory "/"))
 (read (shell-command-to-string "getconf LINE_MAX")))

LINE_MAX has nothing to do with the tty line discipline.

I'll never learn it :-(

We've discussed this issue some years ago already. IIRC, the outcome was
to use "getconf PIPE_BUF /" instead.

Neither has PIPE_BUF.

Andreas.

Okay then. Since there seem to be no better alternatives, I have
attached a new patch reducing the limit to a hard-coded 1024 bytes.  If
some day someone adds a variable specifying a more precise limit, then
we can change this.

I have also rearranged things a bit so that the setup code is sent to
the inferior process just once, rather than of on every call to
`python-shell-send-string'.  This way, the smaller line length limit
doesn't increase too much the use of temp files, which, as I mentioned,
is slow over ssh.

It would be great if someone with access to a BSD-like OS could test
this.  I can only test locally on Linux and over ssh between Linux
machines.

PS: I have some more suggestions around the Python shell.  Is the ideal
workflow to keep creating bugs with a small patch to each improvement,
or do you prefer to review a larger collection of changes bundled
together?

<0001-Reduce-limit-of-line-length-sent-to-the-Python-infer.patch>

I tested the patch, 0001-Reduce-limit-of-line-length-sent-to-the-Python-infer.patch, on macOS 11.5.1 and although my minimal example runs, my original document using org babel session with multiple python blocks is failing with the message in the mini buffer “Inline error: multiline result cannot be used”. I can try some lower value than 1024 and report back, but below is what I saw with attempting to use the debugger to see what the string looks like in my use case.

Setting debug-on-entry to comint-send-string and running a src block, I get:

Debugger entered--returning value: nil
  comint-send-string(#<process filedate> "__PYTHON_EL_eval_file(\"/var/folders/kf/zdpzgs9d30b3jj4bdkdjf1vw0000gn/T/pybQ7oBh\", \"/var/folders/kf/...")
  python-shell-send-file("/var/folders/kf/zdpzgs9d30b3jj4bdkdjf1vw0000gn/T/p..." #<process filedate> "/var/folders/kf/zdpzgs9d30b3jj4bdkdjf1vw0000gn/T/p..." t)
  python-shell-send-string("try:\n    import ast\n    with open('/var/folders/kf...")
  org-babel-python--send-string("*filedate*" "import ast\nwith open('/var/folders/kf/zdpzgs9d30b3...")
  org-babel-python-evaluate-session("*filedate*" "file=\"Roster_Report.csv\"\nimport os\nimport time\npri..." value ("raw" "replace"))
  org-babel-python-evaluate("*filedate*" "file=\"Roster_Report.csv\"\nimport os\nimport time\npri..." value ("raw" "replace") nil)
  org-babel-execute:python("import os\nimport time\nprint(time.strftime(\"%Y-%m-%..." ((:var file . "Roster_Report.csv") (:colname-names) (:rowname-names) (:result-params "raw" "replace") (:result-type . value) (:results . "raw replace") (:exports . "results") (:cache . "no") (:noweb . "no") (:hlines . "no") (:async . "yes") (:tangle . "reports.py") (:comments . "both") (:session . "filedate")))
  org-babel-execute-src-block(nil ("python" "import os\nimport time\nprint(time.strftime(\"%Y-%m-%..." ((:var file . "Roster_Report.csv") (:colname-names) (:rowname-names) (:result-params "replace" "raw") (:result-type . value) (:results . "replace raw") (:exports . "results") (:session . "filedate") (:comments . "both") (:tangle . "reports.py") (:async . "yes") (:hlines . "no") (:noweb . "no") (:cache . "no")) "" "file_date" 7735 "(ref:%s)"))
  org-ctrl-c-ctrl-c(nil)
  funcall-interactively(org-ctrl-c-ctrl-c nil)
  command-execute(org-ctrl-c-ctrl-c)

If I look at the file passed, pybQ7oBh, the contents looks like it will open another file.

try:
    import ast
    with open('/var/folders/kf/zdpzgs9d30b3jj4bdkdjf1vw0000gn/T/babel-K4MbU2/python-hNII4l') as __org_babel_python_tmpfile:
        __org_babel_python_ast = ast.parse(__org_babel_python_tmpfile.read())
    __org_babel_python_final = __org_babel_python_ast.body[-1]
    if isinstance(__org_babel_python_final, ast.Expr):
        __org_babel_python_ast.body = __org_babel_python_ast.body[:-1]
        exec(compile(__org_babel_python_ast, '<string>', 'exec'))
        __org_babel_python_final = eval(compile(ast.Expression(
            __org_babel_python_final.value), '<string>', 'eval'))
        with open('/var/folders/kf/zdpzgs9d30b3jj4bdkdjf1vw0000gn/T/babel-K4MbU2/python-ra4ysF', 'w') as __org_babel_python_tmpfile:
            if False:
                import pprint
                __org_babel_python_tmpfile.write(pprint.pformat(__org_babel_python_final))
            else:
                __org_babel_python_tmpfile.write(str(__org_babel_python_final))
    else:
        exec(compile(__org_babel_python_ast, '<string>', 'exec'))
        __org_babel_python_final = None
except:
    raise
finally:
    print('org_babel_python_eoe')

[-- Attachment #2: Type: text/html, Size: 10818 bytes --]