From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <emacs-orgmode-bounces+larch=yhetil.org@gnu.org>
Received: from mp0 ([2001:41d0:2:4a6f::])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	by ms11 with LMTPS
	id sFNQHyKVll90PgAA0tVLHw
	(envelope-from <emacs-orgmode-bounces+larch=yhetil.org@gnu.org>)
	for <larch@yhetil.org>; Mon, 26 Oct 2020 09:21:38 +0000
Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	by mp0 with LMTPS
	id IKFfGyKVll/kbgAA1q6Kng
	(envelope-from <emacs-orgmode-bounces+larch=yhetil.org@gnu.org>)
	for <larch@yhetil.org>; Mon, 26 Oct 2020 09:21:38 +0000
Received: from lists.gnu.org (lists.gnu.org [209.51.188.17])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(No client certificate requested)
	by aspmx1.migadu.com (Postfix) with ESMTPS id 59B8394043B
	for <larch@yhetil.org>; Mon, 26 Oct 2020 09:21:37 +0000 (UTC)
Received: from localhost ([::1]:35430 helo=lists1p.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.90_1)
	(envelope-from <emacs-orgmode-bounces+larch=yhetil.org@gnu.org>)
	id 1kWyh5-0000f6-7y
	for larch@yhetil.org; Mon, 26 Oct 2020 05:21:35 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:55908)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <numbchild@gmail.com>)
 id 1kWyg8-0000ey-8C
 for emacs-orgmode@gnu.org; Mon, 26 Oct 2020 05:20:36 -0400
Received: from mail-pj1-x1044.google.com ([2607:f8b0:4864:20::1044]:52562)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
 (Exim 4.90_1) (envelope-from <numbchild@gmail.com>)
 id 1kWyg5-0005we-K5
 for emacs-orgmode@gnu.org; Mon, 26 Oct 2020 05:20:35 -0400
Received: by mail-pj1-x1044.google.com with SMTP id o1so2894657pjt.2
 for <emacs-orgmode@gnu.org>; Mon, 26 Oct 2020 02:20:33 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=message-id:references:user-agent:from:to:cc:subject:reply-to
 :in-reply-to:date:mime-version;
 bh=VJYgZvgyE6le30BConU/lBEQyn88TGZ5s5iE/qANi8A=;
 b=LBNCE99xyppDHrgqQBZlfusJFZMpL3lCvlyCQZhABSm3Cvh0+6ozX6m9g8pCxr5fFg
 Es5h9sjCpm7WJDDz30ceeDCPEMdgp0rGOA/toh2RB/Vgek4Ax2hi2WuDNJedX0FUPnBK
 Zg1u9VX9F8Ah5PCy4JyrXct/htEG66ynX6rWE81OMrRsKNdO8AAhofynPy0FmY2Ibinp
 1UsbeLSi6pWJhvNOTmcUSRSTKfNlgb/IoZ7SnckqWfX9VDqtQDjNqeJq4Av9EaBQPBrD
 qMDfefAxWTjYQc6vGS4NqTRYb/d9FDaYqyGG5dNX8fjim/hjzNOwvRZmgwWPcV8QEeaz
 xDNQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:message-id:references:user-agent:from:to:cc
 :subject:reply-to:in-reply-to:date:mime-version;
 bh=VJYgZvgyE6le30BConU/lBEQyn88TGZ5s5iE/qANi8A=;
 b=prpp6IZaCCDixXnvyRZui6bONw2WFqGhCgphbH9rpJFHTZ/KnIAun7BzGWpy5FIqRI
 H1IOEB2/YlYOAGIGPsHgB4l95xxD0o04rocnt0H8JIS+DOHFeCaMCcEKKL4OgM9vPAGp
 l6FYevsnezwe0HapQ2DZxXQrlYAy7bOWxoFLh0vZ2/rAtFo2ax9eRLlG8JCwWzca2ZC1
 LJ6uRyXSbHBUy9AXZrJzSQJnrohUNx79zdR2WiExlZXovitg2JJ3NbVuMkDIbTfQIfqA
 s4GfNoIUK5RmTFFSexCxc8caONvs4+UQOpHjgaxLJvbAaLvR/92TJaITuQ+0tTd7i2QM
 zcFw==
X-Gm-Message-State: AOAM531A/oSTU/oJPlw9YyfPjGNrAteaYSEGSScF90ab0Cn7NU3kUlt8
 Lk0eMlPyFyA7fTvejnF36Q==
X-Google-Smtp-Source: ABdhPJw8YGFpFxzLDqmhqZ81bh4Uav6fYDMVFU/KPyCyRI4urrM1iLVTRW2hjeyAGg4jMhm8cITA2A==
X-Received: by 2002:a17:902:b601:b029:d3:e6c5:5112 with SMTP id
 b1-20020a170902b601b02900d3e6c55112mr9040090pls.65.1603704032051; 
 Mon, 26 Oct 2020 02:20:32 -0700 (PDT)
Received: from dark.localdomain ([183.246.144.170])
 by smtp.gmail.com with ESMTPSA id t13sm11566128pfc.1.2020.10.26.02.20.30
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Oct 2020 02:20:30 -0700 (PDT)
Message-ID: <5f9694de.1c69fb81.c1e21.85d3@mx.google.com>
X-Google-Original-Message-ID: <87lfftene5.fsf@numbchild@gmail.com>
Received: by dark.localdomain (Postfix, from userid 1000)
 id 140513C026F; Mon, 26 Oct 2020 10:23:14 +0800 (CST)
References: <87h7qi2l2m.fsf@gmail.com>
User-agent: mu4e 1.5.5; emacs 28.0.50
From: stardiviner <numbchild@gmail.com>
To: Jack Kamm <jackkamm@gmail.com>
Subject: Re: [PATCH] Async session eval (2nd attempt)
In-reply-to: <87h7qi2l2m.fsf@gmail.com>
Date: Mon, 26 Oct 2020 10:23:14 +0800
MIME-Version: 1.0
Content-Type: text/plain
Received-SPF: pass client-ip=2607:f8b0:4864:20::1044;
 envelope-from=numbchild@gmail.com; helo=mail-pj1-x1044.google.com
X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache.
 That's all we know.
X-Spam_score_int: -5
X-Spam_score: -0.6
X-Spam_bar: /
X-Spam_report: (-0.6 / 5.0 requ) BAYES_00=-1.9, DATE_IN_PAST_06_12=1.543,
 DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 FREEMAIL_FROM=0.001, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_NONE=-0.0001,
 SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no
X-Spam_action: no action
X-BeenThere: emacs-orgmode@gnu.org
X-Mailman-Version: 2.1.23
Precedence: list
List-Id: "General discussions about Org-mode." <emacs-orgmode.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/emacs-orgmode>,
 <mailto:emacs-orgmode-request@gnu.org?subject=unsubscribe>
List-Archive: <https://lists.gnu.org/archive/html/emacs-orgmode>
List-Post: <mailto:emacs-orgmode@gnu.org>
List-Help: <mailto:emacs-orgmode-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/emacs-orgmode>,
 <mailto:emacs-orgmode-request@gnu.org?subject=subscribe>
Reply-To: numbchild@gmail.com
Cc: emacs-orgmode@gnu.org
Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org
Sender: "Emacs-orgmode" <emacs-orgmode-bounces+larch=yhetil.org@gnu.org>
X-Scanner: scn0
Authentication-Results: aspmx1.migadu.com;
	dkim=pass header.d=gmail.com header.s=20161025 header.b=LBNCE99x;
	dmarc=pass (policy=none) header.from=gmail.com;
	spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org
X-Spam-Score: -1.71
X-TUID: 2ZYzpBcAoMj6


This really is an good idea. Some other Babel languages like Ruby, JavaScript
might benefit from this ob-comint async evaluation. Awesome!

Jack Kamm <jackkamm@gmail.com> writes:

> This patch adds asynchronous evaluation for session blocks in
> Python. It also adds functionality to implement async session eval for
> other languages using ob-comint.el.
>
> To test the attached patch, add ":async" to a Python session block
> with a long computation (or "time.sleep") in it. Upon evaluation, your
> Emacs won't freeze to wait for the result -- instead, a placeholder
> will be inserted, and replaced with the true result when it's ready.
>
> I'll note how this is different from some related projects. ob-async
> implements asynchronous evaluation for Babel, but it doesn't work with
> sessions. emacs-jupyter, ein, and ob-ipython all implement
> asynchronous session evaluation, but only for Jupyter kernels. Jupyter
> is great for some cases, but sometimes I prefer to use the built-in
> org-babel languages without jupyter.
>
> The new functionality is mainly implemented in
> `org-babel-comint-async-filter', which I've defined in ob-comint.el,
> and added as a hook to `comint-output-filter-functions'.  Whenever new
> output is added to the comint buffer, the filter scans for an
> indicator token (this is inspired by
> `org-babel-comint-with-output'). Upon encountering the token, the
> filter uses a regular expression to extract a UUID or temp-file
> associated with the result, then searches for the appropriate location
> to add the result to.
>
> This is my 2nd attempt at this patch [0]. I have also ported it to an
> external package [1], but would like to have this functionality in Org
> proper, to permit better code reuse between async and sync
> implementations. The external package also includes an R
> implementation that I regularly use, as well as a Ruby implementation,
> but I've left these out to keep this initial patch smaller, and also I
> need to confirm copyright assignment on the Ruby implementation which
> was externally contributed.
>
> [0] https://orgmode.org/list/87muj04xim.fsf@jaheira.i-did-not-set--mail-host-address--so-tickle-me/
> [1] https://github.com/jackkamm/ob-session-async
>
> From 8b7695a148d1831c916737650e115833cb7fc752 Mon Sep 17 00:00:00 2001
> From: Jack Kamm <jackkamm@gmail.com>
> Date: Sun, 25 Oct 2020 11:40:10 -0700
> Subject: [PATCH] ob-comint.el, ob-python.el: Async session evaluation
>
> Adds functionality to ob-comint.el to implement async session eval on
> a per-language basis.  Adds a reference implementation for ob-python.
>
> * lisp/ob-comint.el (org-babel-comint-with-output): Remove comment.
> (org-babel-comint-async-indicator, org-babel-comint-async-buffers,
> org-babel-comint-async-file-callback,
> org-babel-comint-async-chunk-callback,
> org-babel-comint-async-dangling): Add buffer-local variables used for
> async comint evaluation.
> (org-babel-comint-use-async): Add function to determine whether block
> should be evaluated asynchronously.
> (org-babel-comint-async-filter): Add filter function to attach to
> comint-output-filter-functions for babel async eval.
> (org-babel-comint-async-register): Add function to setup buffer
> variables and hooks for session eval.
> (org-babel-comint-async-delete-dangling-and-eval): Add helper function
> for async session eval.
>
> * lisp/ob-python.el (org-babel-execute:python): Check for async header
> argument.
> (org-babel-python-evaluate): Check whether to use async evaluation.
> (org-babel-python-async-indicator): Add constant for indicating the
> start/end of async evaluations.
> (org-babel-python-async-evaluate-session): Add function for Python
> async eval.
>
> *
> testing/lisp/test-ob-python.el (test-ob-python/async-simple-session-output):
> Unit test for Python async session eval.
> (test-ob-python/async-named-output): Unit test that Python async eval
> can replace named output.
> (test-ob-python/async-output-drawer): Unit test that Python async eval
> works with drawer results.
> ---
>  etc/ORG-NEWS                   |  15 +++
>  lisp/ob-comint.el              | 172 +++++++++++++++++++++++++++++++--
>  lisp/ob-python.el              |  56 ++++++++++-
>  testing/lisp/test-ob-python.el |  61 ++++++++++++
>  4 files changed, 294 insertions(+), 10 deletions(-)
>
> diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
> index 7f935bf52..9d5fbbe30 100644
> --- a/etc/ORG-NEWS
> +++ b/etc/ORG-NEWS
> @@ -88,6 +88,21 @@ package, to convert pandas Dataframes into orgmode tables:
>  | 2 | 3 | 6 |
>  #+end_src
>  
> +*** Async session evaluation
> +
> +The =:async= header argument can be used for asynchronous evaluation
> +in session blocks for certain languages.
> +
> +Currently, async evaluation is supported in Python.  There is also
> +functionality to implement async evaluation in other languages that
> +use comint, but this needs to be done on a per-language basis.
> +
> +By default, async evaluation is disabled unless the =:async= header
> +argument is present.  You can also set =:async no= to force it off
> +(for example if you've set =:async= in a property drawer).
> +
> +Async evaluation is disabled during export.
> +
>  * Version 9.4
>  ** Incompatible changes
>  *** Possibly broken internal file links: please check and fix
> diff --git a/lisp/ob-comint.el b/lisp/ob-comint.el
> index d3484bb7c..591754dac 100644
> --- a/lisp/ob-comint.el
> +++ b/lisp/ob-comint.el
> @@ -94,12 +94,7 @@ (defmacro org-babel-comint-with-output (meta &rest body)
>  			       (regexp-quote ,eoe-indicator) nil t)
>  			      (re-search-forward
>  			       comint-prompt-regexp nil t)))))
> -	   (accept-process-output (get-buffer-process (current-buffer)))
> -	   ;; thought the following this would allow async
> -	   ;; background running, but I was wrong...
> -	   ;; (run-with-timer .5 .5 'accept-process-output
> -	   ;; 		 (get-buffer-process (current-buffer)))
> -	   )
> +	   (accept-process-output (get-buffer-process (current-buffer))))
>  	 ;; replace cut dangling text
>  	 (goto-char (process-mark (get-buffer-process (current-buffer))))
>  	 (insert dangling-text)
> @@ -149,6 +144,171 @@ (defun org-babel-comint-eval-invisibly-and-wait-for-file
>     (if (= (aref string (1- (length string))) ?\n) string (concat string "\n")))
>    (while (not (file-exists-p file)) (sit-for (or period 0.25))))
>  
> +
> +;; Async evaluation
> +
> +(defvar-local org-babel-comint-async-indicator nil
> +  "Regular expression that `org-babel-comint-async-filter' scans for.
> +It should have 2 parenthesized expressions,
> +e.g. \"org_babel_async_\\(start\\|end\\|file\\)_\\(.*\\)\". The
> +first parenthesized expression determines whether the token is
> +delimiting a result block, or whether the result is in a file. If
> +delimiting a block, the second expression gives a UUID for the
> +location to insert the result. Otherwise, the result is in a tmp
> +file, and the second expression gives the file name.")
> +
> +(defvar-local org-babel-comint-async-buffers nil
> +  "List of org-mode buffers to check for Babel async output results.")
> +
> +(defvar-local org-babel-comint-async-file-callback nil
> +  "Callback to clean and insert Babel async results from a temp file.
> +The callback function takes two arguments: the alist of params of the Babel
> +source block, and the name of the temp file.")
> +
> +(defvar-local org-babel-comint-async-chunk-callback nil
> +  "Callback function to clean Babel async output results before insertion.
> +Its single argument is a string consisting of output from the
> +comint process. It should return a string that will be be passed
> +to `org-babel-insert-result'.")
> +
> +(defvar-local org-babel-comint-async-dangling nil
> +  "Dangling piece of the last process output, in case
> +`org-babel-comint-async-indicator' is spread across multiple
> +comint outputs due to buffering.")
> +
> +(defun org-babel-comint-use-async (params)
> +  "Determine whether to use session async evaluation.
> +PARAMS are the header arguments as passed to
> +`org-babel-execute:lang'."
> +  (let ((async (assq :async params))
> +        (session (assq :session params)))
> +    (and async
> +	 (not org-babel-exp-reference-buffer)
> +         (not (equal (cdr async) "no"))
> +         (not (equal (cdr session) "none")))))
> +
> +(defun org-babel-comint-async-filter (string)
> +  "Captures Babel async output from comint buffer back to org-mode buffers.
> +This function is added as a hook to `comint-output-filter-functions'.
> +STRING contains the output originally inserted into the comint buffer."
> +  ;; Remove outdated org-mode buffers
> +  (setq org-babel-comint-async-buffers
> +	(cl-loop for buf in org-babel-comint-async-buffers
> +	      if (buffer-live-p buf)
> +	      collect buf))
> +  (let* ((indicator org-babel-comint-async-indicator)
> +	 (org-buffers org-babel-comint-async-buffers)
> +	 (file-callback org-babel-comint-async-file-callback)
> +	 (combined-string (concat org-babel-comint-async-dangling string))
> +	 (new-dangling combined-string)
> +	 ;; list of UUID's matched by `org-babel-comint-async-indicator'
> +	 uuid-list)
> +    (with-temp-buffer
> +      (insert combined-string)
> +      (goto-char (point-min))
> +      (while (re-search-forward indicator nil t)
> +	;; update dangling
> +	(setq new-dangling (buffer-substring (point) (point-max)))
> +	(cond ((equal (match-string 1) "end")
> +	       ;; save UUID for insertion later
> +	       (push (match-string 2) uuid-list))
> +	      ((equal (match-string 1) "file")
> +	       ;; insert results from tmp-file
> +	       (let ((tmp-file (match-string 2)))
> +		 (cl-loop for buf in org-buffers
> +		       until
> +		       (with-current-buffer buf
> +			 (save-excursion
> +			   (goto-char (point-min))
> +			   (when (search-forward tmp-file nil t)
> +			     (org-babel-previous-src-block)
> +                             (let* ((info (org-babel-get-src-block-info))
> +                                    (params (nth 2 info))
> +                                    (result-params
> +                                     (cdr (assq :result-params params))))
> +                               (org-babel-insert-result
> +                                 (funcall file-callback
> +                                          (nth
> +                                           2 (org-babel-get-src-block-info))
> +                                          tmp-file)
> +                                result-params info))
> +			     t))))))))
> +      ;; Truncate dangling to only the most recent output
> +      (when (> (length new-dangling) (length string))
> +	(setq new-dangling string)))
> +    (setq-local org-babel-comint-async-dangling new-dangling)
> +    (when uuid-list
> +      ;; Search for results in the comint buffer
> +      (save-excursion
> +	(goto-char (point-max))
> +	(while uuid-list
> +	  (re-search-backward indicator)
> +	  (when (equal (match-string 1) "end")
> +	    (let* ((uuid (match-string-no-properties 2))
> +		   (res-str-raw
> +		    (buffer-substring
> +		     ;; move point to beginning of indicator
> +                     (- (match-beginning 0) 1)
> +		     ;; find the matching start indicator
> +		     (cl-loop for pos = (re-search-backward indicator)
> +			   until (and (equal (match-string 1) "start")
> +				      (equal (match-string 2) uuid))
> +			   finally return (+ 1 (match-end 0)))))
> +		   ;; Apply callback to clean up the result
> +		   (res-str (funcall org-babel-comint-async-chunk-callback
> +                                     res-str-raw)))
> +	      ;; Search for uuid in associated org-buffers to insert results
> +	      (cl-loop for buf in org-buffers
> +		    until (with-current-buffer buf
> +			    (save-excursion
> +			      (goto-char (point-min))
> +			      (when (search-forward uuid nil t)
> +				(org-babel-previous-src-block)
> +                                (let* ((info (org-babel-get-src-block-info))
> +                                       (params (nth 2 info))
> +                                       (result-params
> +                                        (cdr (assq :result-params params))))
> +				  (org-babel-insert-result
> +                                   res-str result-params info))
> +				t))))
> +	      ;; Remove uuid from the list to search for
> +	      (setq uuid-list (delete uuid uuid-list)))))))))
> +
> +(defun org-babel-comint-async-register
> +    (session-buffer org-buffer indicator-regexp
> +		    chunk-callback file-callback)
> +  "Sets local org-babel-comint-async variables in SESSION-BUFFER.
> +ORG-BUFFER is added to `org-babel-comint-async-buffers' if not
> +present.  `org-babel-comint-async-indicator',
> +`org-babel-comint-async-chunk-callback', and
> +`org-babel-comint-async-file-callback' are set to
> +INDICATOR-REGEXP, CHUNK-CALLBACK, and FILE-CALLBACK
> +respectively."
> +  (org-babel-comint-in-buffer session-buffer
> +    (setq org-babel-comint-async-indicator indicator-regexp
> +	  org-babel-comint-async-chunk-callback chunk-callback
> +	  org-babel-comint-async-file-callback file-callback)
> +    (unless (memq org-buffer org-babel-comint-async-buffers)
> +      (setq org-babel-comint-async-buffers
> +	    (cons org-buffer org-babel-comint-async-buffers)))
> +    (add-hook 'comint-output-filter-functions
> +	      'org-babel-comint-async-filter nil t)))
> +
> +(defmacro org-babel-comint-async-delete-dangling-and-eval
> +    (session-buffer &rest body)
> +  "Remove dangling text in SESSION-BUFFER and evaluate BODY.
> +This is analogous to `org-babel-comint-with-output', but meant
> +for asynchronous output, and much shorter because inserting the
> +result is delegated to `org-babel-comint-async-filter'."
> +  (declare (indent 1))
> +  `(org-babel-comint-in-buffer ,session-buffer
> +     (goto-char (process-mark (get-buffer-process (current-buffer))))
> +     (delete-region (point) (point-max))
> +     ,@body))
> +(def-edebug-spec org-babel-comint-async-with-output (sexp body))
> +
>  (provide 'ob-comint)
>  
> +
> +
>  ;;; ob-comint.el ends here
> diff --git a/lisp/ob-python.el b/lisp/ob-python.el
> index 6752adc17..e26c34b64 100644
> --- a/lisp/ob-python.el
> +++ b/lisp/ob-python.el
> @@ -84,6 +84,7 @@ (defun org-babel-execute:python (body params)
>  	 (return-val (when (eq result-type 'value)
>  		       (cdr (assq :return params))))
>  	 (preamble (cdr (assq :preamble params)))
> +	 (async (org-babel-comint-use-async params))
>           (full-body
>  	  (concat
>  	   (org-babel-expand-body:generic
> @@ -92,7 +93,8 @@ (defun org-babel-execute:python (body params)
>  	   (when return-val
>  	     (format (if session "\n%s" "\nreturn %s") return-val))))
>           (result (org-babel-python-evaluate
> -		  session full-body result-type result-params preamble)))
> +		  session full-body result-type
> +		  result-params preamble async)))
>      (org-babel-reassemble-table
>       result
>       (org-babel-pick-name (cdr (assq :colname-names params))
> @@ -278,11 +280,14 @@ (defun org-babel-python-format-session-value
>  	  (if (member "pp" result-params) "True" "False")))
>  
>  (defun org-babel-python-evaluate
> -  (session body &optional result-type result-params preamble)
> +  (session body &optional result-type result-params preamble async)
>    "Evaluate BODY as Python code."
>    (if session
> -      (org-babel-python-evaluate-session
> -       session body result-type result-params)
> +      (if async
> +	  (org-babel-python-async-evaluate-session
> +	   session body result-type result-params)
> +	(org-babel-python-evaluate-session
> +	 session body result-type result-params))
>      (org-babel-python-evaluate-external-process
>       body result-type result-params preamble)))
>  
> @@ -391,6 +396,49 @@ (defun org-babel-python-read-string (string)
>        (substring string 1 -1)
>      string))
>  
> +;; Async session eval
> +
> +(defconst org-babel-python-async-indicator "print ('ob_comint_async_python_%s_%s')")
> +
> +(defun org-babel-python-async-value-callback (params tmp-file)
> +  (let ((result-params (cdr (assq :result-params params)))
> +	(results (org-babel-eval-read-file tmp-file)))
> +    (org-babel-result-cond result-params
> +      results
> +      (org-babel-python-table-or-string results))))
> +
> +(defun org-babel-python-async-evaluate-session
> +    (session body &optional result-type result-params)
> +  "Asynchronously evaluate BODY in SESSION.
> +Returns a placeholder string for insertion, to later be replaced
> +by `org-babel-comint-async-filter'."
> +  (org-babel-comint-async-register
> +   session (current-buffer)
> +   "ob_comint_async_python_\\(.+\\)_\\(.+\\)"
> +   'org-babel-chomp 'org-babel-python-async-value-callback)
> +  (let ((python-shell-buffer-name (org-babel-python-without-earmuffs session)))
> +    (pcase result-type
> +      (`output
> +       (let ((uuid (md5 (number-to-string (random 100000000)))))
> +         (with-temp-buffer
> +           (insert (format org-babel-python-async-indicator "start" uuid))
> +           (insert "\n")
> +           (insert body)
> +           (insert "\n")
> +           (insert (format org-babel-python-async-indicator "end" uuid))
> +           (python-shell-send-buffer))
> +         uuid))
> +      (`value
> +       (let ((tmp-results-file (org-babel-temp-file "python-"))
> +             (tmp-src-file (org-babel-temp-file "python-")))
> +         (with-temp-file tmp-src-file (insert body))
> +         (with-temp-buffer
> +           (insert (org-babel-python-format-session-value tmp-src-file tmp-results-file result-params))
> +           (insert "\n")
> +           (insert (format org-babel-python-async-indicator "file" tmp-results-file))
> +           (python-shell-send-buffer))
> +         tmp-results-file)))))
> +
>  (provide 'ob-python)
>  
>  ;;; ob-python.el ends here
> diff --git a/testing/lisp/test-ob-python.el b/testing/lisp/test-ob-python.el
> index a2cc7b79c..0267678cd 100644
> --- a/testing/lisp/test-ob-python.el
> +++ b/testing/lisp/test-ob-python.el
> @@ -207,6 +207,67 @@ (ert-deftest test-ob-python/session-value-sleep ()
>  #+end_src"
>  	    (org-babel-execute-src-block)))))
>  
> +(ert-deftest test-ob-python/async-simple-session-output ()
> +  (let ((org-babel-temporary-directory "/tmp")
> +        (org-confirm-babel-evaluate nil))
> +    (org-test-with-temp-text
> +     "#+begin_src python :session :async yes :results output
> +import time
> +time.sleep(.1)
> +print('Yep!')
> +#+end_src\n"
> +     (should (let ((expected "Yep!"))
> +	       (and (not (string= expected (org-babel-execute-src-block)))
> +		    (string= expected
> +			     (progn
> +			       (sleep-for 0 200)
> +			       (goto-char (org-babel-where-is-src-block-result))
> +			       (org-babel-read-result)))))))))
> +
> +(ert-deftest test-ob-python/async-named-output ()
> +  (let (org-confirm-babel-evaluate
> +        (org-babel-temporary-directory "/tmp")
> +        (src-block "#+begin_src python :async :session :results output
> +print(\"Yep!\")
> +#+end_src")
> +        (results-before "
> +
> +#+NAME: foobar
> +#+RESULTS:
> +: Nope!")
> +        (results-after "
> +
> +#+NAME: foobar
> +#+RESULTS:
> +: Yep!
> +"))
> +    (org-test-with-temp-text
> +     (concat src-block results-before)
> +     (should (progn (org-babel-execute-src-block)
> +                    (sleep-for 0 200)
> +                    (string= (concat src-block results-after)
> +                             (buffer-string)))))))
> +
> +(ert-deftest test-ob-python/async-output-drawer ()
> +  (let (org-confirm-babel-evaluate
> +        (org-babel-temporary-directory "/tmp")
> +        (src-block "#+begin_src python :async :session :results output drawer
> +print(list(range(3)))
> +#+end_src")
> +        (result "
> +
> +#+RESULTS:
> +:results:
> +[0, 1, 2]
> +:end:
> +"))
> +    (org-test-with-temp-text
> +     src-block
> +     (should (progn (org-babel-execute-src-block)
> +                    (sleep-for 0 200)
> +                    (string= (concat src-block result)
> +                             (buffer-string)))))))
> +
>  (provide 'test-ob-python)
>  
>  ;;; test-ob-python.el ends here


-- 
[ stardiviner ]
       I try to make every word tell the meaning that I want to express.

       Blog: https://stardiviner.github.io/
       IRC(freenode): stardiviner, Matrix: stardiviner
       GPG: F09F650D7D674819892591401B5DF1C95AE89AC3