From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <emacs-orgmode-bounces+larch=yhetil.org@gnu.org>
Received: from mp0 ([2001:41d0:2:4a6f::])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	by ms11 with LMTPS
	id CfnQAs2R8V9QXwAA0tVLHw
	(envelope-from <emacs-orgmode-bounces+larch=yhetil.org@gnu.org>)
	for <larch@yhetil.org>; Sun, 03 Jan 2021 09:43:41 +0000
Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	by mp0 with LMTPS
	id GCnVOcyR8V/8cwAA1q6Kng
	(envelope-from <emacs-orgmode-bounces+larch=yhetil.org@gnu.org>)
	for <larch@yhetil.org>; Sun, 03 Jan 2021 09:43:40 +0000
Received: from lists.gnu.org (lists.gnu.org [209.51.188.17])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(No client certificate requested)
	by aspmx1.migadu.com (Postfix) with ESMTPS id 4DE3C9403D5
	for <larch@yhetil.org>; Sun,  3 Jan 2021 09:43:40 +0000 (UTC)
Received: from localhost ([::1]:54214 helo=lists1p.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.90_1)
	(envelope-from <emacs-orgmode-bounces+larch=yhetil.org@gnu.org>)
	id 1kvzvG-0006Wd-1m
	for larch@yhetil.org; Sun, 03 Jan 2021 04:43:38 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10]:36246)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <tecosaur@gmail.com>)
 id 1kvzud-0006WK-Ox
 for emacs-orgmode@gnu.org; Sun, 03 Jan 2021 04:42:59 -0500
Received: from mail-pf1-x42f.google.com ([2607:f8b0:4864:20::42f]:41454)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
 (Exim 4.90_1) (envelope-from <tecosaur@gmail.com>)
 id 1kvzub-0005Lc-MB
 for emacs-orgmode@gnu.org; Sun, 03 Jan 2021 04:42:59 -0500
Received: by mail-pf1-x42f.google.com with SMTP id t8so14538140pfg.8
 for <emacs-orgmode@gnu.org>; Sun, 03 Jan 2021 01:42:56 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=references:user-agent:from:to:cc:subject:date:in-reply-to
 :message-id:mime-version;
 bh=tE/Roe97JcwHvDSt2PjYywZEGTxE5InBhnQFr1Ht5qY=;
 b=REiLIFyywNQREM7ymhO2QP5neXcjsqJvtVdhJrvEFjDA3NvWh4wyvNcZe/D9o+uxpL
 om/udxy01P/P/0SGYnl+KXiaso7WEhMFNi8uZuoAl9doIixjv+QBj+dAGjT3V2M7UrZv
 B7ekFtcJzxcfSUcNOY7Wite99SojExR8japj90kGp1EzXdPlZnfHti99EoEwKVZxwt5B
 L+G4JbWxeNxyKJAXAzGqCxNbvAmGFSdiOx8wKFBoDWU0InKl7c0ojjQRYeB3wQkn0zPZ
 lpEqKfmpJWDSfnx+6FZ8uPA1Z/0a0VpBUEr1ukd4gBTnx3PL+hPm9hhkulA7ygOFtAX7
 RBtw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:references:user-agent:from:to:cc:subject:date
 :in-reply-to:message-id:mime-version;
 bh=tE/Roe97JcwHvDSt2PjYywZEGTxE5InBhnQFr1Ht5qY=;
 b=nEOHm+q7Wzv0Rp2fP/JRt1UGrzMUqYpVVJ1syk8h5dYWDly6T+vvRHOuKwAb0FkwTx
 iaj5f8TFMwAmdrV0Q1sSS6Kt+uo7T1Hg53uxeL+L+1xmSaYBlQ+ZBj+yi2Hyr5ABSqtH
 vasIZWlTJu8hchcdwBwj+MKHQL0O4hyvl4QnvGp+T4FwjTS/m0faWj42reNK8zgj+koz
 bCIQnK/zCOXYG95jL2VYr+twiV5vNwui4iJ793zinH0ZPcBImew1s9ecrZJeUVYwJV3E
 wWUIwaGqtjjLwCEzjv6iOmBhLfPsgUmONjOLnl/3PMo/52F1oBpuQDMkmoylhm1rNKvc
 Gr3g==
X-Gm-Message-State: AOAM532424n9Xf8aY5A8apKRqH2QCpZTwmG0fEKNyc1Ci0fGOMgeB7Nw
 5j1iSo+1QVw50kZzOT3ENbQ=
X-Google-Smtp-Source: ABdhPJwRAcuWYN8nrDgWmTNluH0DJ1Z+Qg/IVEvTZX78hjNZo5n4zEdUN0OJPSDEtEvcRm29iuAlYA==
X-Received: by 2002:a63:5407:: with SMTP id i7mr42411015pgb.418.1609666975801; 
 Sun, 03 Jan 2021 01:42:55 -0800 (PST)
Received: from localhost (180-150-91-8.b4965b.per.nbn.aussiebb.net.
 [180.150.91.8])
 by smtp.gmail.com with ESMTPSA id c23sm43057815pgc.72.2021.01.03.01.42.54
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sun, 03 Jan 2021 01:42:55 -0800 (PST)
References: <87h7qi2l2m.fsf@gmail.com>
User-agent: mu4e 1.4.13; emacs 27.1
From: TEC <tecosaur@gmail.com>
To: Jack Kamm <jackkamm@gmail.com>
Subject: Re: [PATCH] Async session eval (2nd attempt)
Date: Sun, 03 Jan 2021 16:51:15 +0800
In-reply-to: <87h7qi2l2m.fsf@gmail.com>
Message-ID: <87lfdatkys.fsf@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain
Received-SPF: pass client-ip=2607:f8b0:4864:20::42f;
 envelope-from=tecosaur@gmail.com; helo=mail-pf1-x42f.google.com
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
 RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-BeenThere: emacs-orgmode@gnu.org
X-Mailman-Version: 2.1.23
Precedence: list
List-Id: "General discussions about Org-mode." <emacs-orgmode.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/emacs-orgmode>,
 <mailto:emacs-orgmode-request@gnu.org?subject=unsubscribe>
List-Archive: <https://lists.gnu.org/archive/html/emacs-orgmode>
List-Post: <mailto:emacs-orgmode@gnu.org>
List-Help: <mailto:emacs-orgmode-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/emacs-orgmode>,
 <mailto:emacs-orgmode-request@gnu.org?subject=subscribe>
Cc: emacs-orgmode@gnu.org
Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org
Sender: "Emacs-orgmode" <emacs-orgmode-bounces+larch=yhetil.org@gnu.org>
X-Migadu-Flow: FLOW_IN
X-Migadu-Spam-Score: -3.03
Authentication-Results: aspmx1.migadu.com;
	dkim=pass header.d=gmail.com header.s=20161025 header.b=REiLIFyy;
	dmarc=pass (policy=none) header.from=gmail.com;
	spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org
X-Migadu-Queue-Id: 4DE3C9403D5
X-Spam-Score: -3.03
X-Migadu-Scanner: scn1.migadu.com
X-TUID: 1kdky0tbwjv5


Hi Jack,

I love the look of this! Thanks for submitting a patch.

Sorry it's taken so long for someone to take a look at it, I think a lot
of the 'main' Org people have been pretty busy over the last few months.

I just tried to give this a shot.
First up, I had to remove the ORG-NEWS part of the patch to be able to
provide it. It would be nice if you could update the patch so this
applies cleanly.

#+begin_example
error: patch failed: etc/ORG-NEWS:88
error: etc/ORG-NEWS: patch does not apply
#+end_example

To test this, after applying your patch (with ORG-NEWS removed), I
started emacs -Q, loaded Org, and opened a new file.

I was initially unable to get this to seem to work, until I changed the
:results type to "output".

See a excerpt from my test file below:

----- excerpt start -----

#+begin_src python :async :session blah :results output
from time import sleep

a=2

sleep(2)
print("Hi")
#+end_src

#+RESULTS:
: Hi

#+begin_src python :async :session blah
return(a)
#+end_src

#+RESULTS:
: /tmp/babel-62cQRX/python-EfJ4o4

#+begin_src python :async :session blah :results output
print(a)
#+end_src

#+RESULTS:
: 2

----- excerpt end -----


I'm surprised this didn't work with the non-output block though.

Other than this, I'm rather happy to see that when I tried to execute
two long running blocks at once, the second one was not executed until
the first completed :)

Finally, I see that this requires :session to be set in order to work.
Might it be possible to have this work for non-session blocks too? It
seems odd that what I'd imagine is the harder case (session blocks) is
supported, but one-shot (non-session) blocks aren't.

Thanks again for your work, and I look forward to seeing what else you
have in the future!

--
Timothy

p.s. After this is merged, it would be great to see support for other
languages grow :)

Jack Kamm <jackkamm@gmail.com> writes:

> This patch adds asynchronous evaluation for session blocks in
> Python. It also adds functionality to implement async session eval for
> other languages using ob-comint.el.
>
> To test the attached patch, add ":async" to a Python session block
> with a long computation (or "time.sleep") in it. Upon evaluation, your
> Emacs won't freeze to wait for the result -- instead, a placeholder
> will be inserted, and replaced with the true result when it's ready.
>
> I'll note how this is different from some related projects. ob-async
> implements asynchronous evaluation for Babel, but it doesn't work with
> sessions. emacs-jupyter, ein, and ob-ipython all implement
> asynchronous session evaluation, but only for Jupyter kernels. Jupyter
> is great for some cases, but sometimes I prefer to use the built-in
> org-babel languages without jupyter.
>
> The new functionality is mainly implemented in
> `org-babel-comint-async-filter', which I've defined in ob-comint.el,
> and added as a hook to `comint-output-filter-functions'.  Whenever new
> output is added to the comint buffer, the filter scans for an
> indicator token (this is inspired by
> `org-babel-comint-with-output'). Upon encountering the token, the
> filter uses a regular expression to extract a UUID or temp-file
> associated with the result, then searches for the appropriate location
> to add the result to.
>
> This is my 2nd attempt at this patch [0]. I have also ported it to an
> external package [1], but would like to have this functionality in Org
> proper, to permit better code reuse between async and sync
> implementations. The external package also includes an R
> implementation that I regularly use, as well as a Ruby implementation,
> but I've left these out to keep this initial patch smaller, and also I
> need to confirm copyright assignment on the Ruby implementation which
> was externally contributed.
>
> [0] https://orgmode.org/list/87muj04xim.fsf@jaheira.i-did-not-set--mail-host-address--so-tickle-me/
> [1] https://github.com/jackkamm/ob-session-async