From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id cAWiJJhX4WPg9QAAbAwnHQ (envelope-from ) for ; Mon, 06 Feb 2023 20:40:08 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id gNi1I5hX4WP3CwAAG6o9tA (envelope-from ) for ; Mon, 06 Feb 2023 20:40:08 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 3DF87DFC4 for ; Mon, 6 Feb 2023 20:40:08 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pP7Kb-0005Er-Qm; Mon, 06 Feb 2023 14:39:13 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pP7KZ-00055f-Em for emacs-orgmode@gnu.org; Mon, 06 Feb 2023 14:39:11 -0500 Received: from sender4-op-o14.zoho.com ([136.143.188.14]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pP7KX-0005aN-Qa for emacs-orgmode@gnu.org; Mon, 06 Feb 2023 14:39:11 -0500 ARC-Seal: i=1; a=rsa-sha256; t=1675712342; cv=none; d=zohomail.com; s=zohoarc; b=gmDeCvs7LYkrjMLKRuC1CDMaV2ZjwzTSoUAMwzOEkXb4yvGuQzbbckeVrHZYtVvEq2WZ0g8TL6deIRPHCMjSc0TKo/lQJSJHWffmwDH9O5mdE4H/X3P/Fm5mY6sTOd3Q92CmdcVqesKSPJaBS7KBmYtJ+UcpS3NRUuk/6bqTzw0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675712342; h=Content-Type:Date:From:MIME-Version:Message-ID:Subject:To; bh=a3rlsGXYfHB1I3PTSwU5n7HqSkKCYvZdiJoM23n4y5g=; b=E1+LkZT+nTLi88r2yX2tz1MkNOWOdNgkEpB8Xri/hP7jZcXCds/kScTRMuZqTKzTPWImqFu0nAGA2CRMumpZuY/vbjT+GVgqoXjZ64yttFiu72kRSZynjdvbLd4NlenXTedpErT3T1lh3vRJVjcW2m8YPUxy3JJ2TWA/xXhhQA8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=excalamus.com; spf=pass smtp.mailfrom=matt@excalamus.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1675712342; s=zmail; d=excalamus.com; i=matt@excalamus.com; h=Date:Date:From:From:To:To:Message-Id:Message-Id:In-Reply-To:Subject:Subject:MIME-Version:Content-Type:Reply-To:Cc; bh=a3rlsGXYfHB1I3PTSwU5n7HqSkKCYvZdiJoM23n4y5g=; b=KHVkkAOz/14A8SEQrAspjmSEINKzaPRTeukJqgRsJFN3Gr/bkdfNwemCsQkERROw zsb5DypVDmIFzASiUInkbZixS+O/1L4SsVYbvdIwxZiSbayz3e8tdjYKDWPz5fq+m7U wvw10XAb0eYiRU8UzFHqD6aAV2ThFyn1i17P6x2o= Received: from mail.zoho.com by mx.zohomail.com with SMTP id 1675712340778145.91483007047498; Mon, 6 Feb 2023 11:39:00 -0800 (PST) Date: Mon, 06 Feb 2023 14:39:00 -0500 From: Matt To: "emacs-orgmode" Message-Id: <186283d230a.129f5feb61660123.3289004102603503414@excalamus.com> In-Reply-To: Subject: [PATCH] Async evaluation in ob-shell MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_5344843_1333486556.1675712340746" Importance: Medium User-Agent: Zoho Mail X-Mailer: Zoho Mail X-Zoho-Virus-Status: 1 Received-SPF: pass client-ip=136.143.188.14; envelope-from=matt@excalamus.com; helo=sender4-op-o14.zoho.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN ARC-Seal: i=2; s=key1; d=yhetil.org; t=1675712408; a=rsa-sha256; cv=pass; b=WpulNfatWRGQ7iMWDL4HEzhIwmQzWobOowJ1DXlU0FgZ5QB7akiVBlIS3WMyJsDlrT2GGt 7VqtWGbKuocrPhVDuW3gMhRbiphy/H6BXkQV12S7oOEyL+/3mQC7Fj2nsFeAva7XUe9MH7 Ci2vnVnnua1TrUjE4xlNzpER7XJ/UQaiyeQdJpxSioyIi9Gbu9yqFQj+jl3G8Ckvs/KC45 yaZy7YfuHOSegHug3yhIvCcQj0jDZHDTPAMdefOCn74sTmU92HhVokcYdKt1LB70TR5uWM yBYy8nCXR/uQVV+qzFSNYVrMQ49xI8Sn6zocfvTIrbvBpgFEc6bppkRpxURwpA== ARC-Authentication-Results: i=2; aspmx1.migadu.com; dkim=pass header.d=excalamus.com header.s=zmail header.b=KHVkkAOz; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; arc=pass ("zohomail.com:s=zohoarc:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1675712408; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=a3rlsGXYfHB1I3PTSwU5n7HqSkKCYvZdiJoM23n4y5g=; b=fVAfG6Ofik/cMed5I/Nmf4WI7FzQtABQTNIZAjRxi2nBN8dPUkrUtHfmI9Gs5V6Rwf+1Mn 3spKh902bHVQHmDy7Qj6qsJR+YkNY0ael2NPQ2qpbF0dhQurvW7p0rwFDaZV3lRbB4DOmD 4dMwYJzRm5Gv1Wsh3JpHwJFkqJ1Y5w9lKEk8Hl7ZcbKnADaU6g23oGkh2LdGDtsSuXrP/d VETaF/POkai2aNxozO6CVRGTX3MG9L/gkv3Sy9pRrNfzB2ZGTX/Tyo6oa9K87gZ8CBh/zK KuAGLhq3xA4tm9si8def7601B6Lxq1/sWKrP02yAf5iAz6O2xnPivZjx6Q3WeA== Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=excalamus.com header.s=zmail header.b=KHVkkAOz; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; arc=pass ("zohomail.com:s=zohoarc:i=1") X-Migadu-Scanner: scn1.migadu.com X-Migadu-Spam-Score: -5.32 X-Spam-Score: -5.32 X-Migadu-Queue-Id: 3DF87DFC4 X-TUID: U+FNs406YitI ------=_Part_5344843_1333486556.1675712340746 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit I'm excited to share that I've got async evaluation working (crudely) with ob-shell. A rough implementation is attached. It has clear issues, such as the prompt being present in the output: #+begin_src sh :session tester :async t echo "By sending delimiters separately..." sleep 3 slep 1 echo "typos don't cause problems--^" #+end_src #+RESULTS: : org_babel_sh_prompt> By sending delimiters separately... : org_babel_sh_prompt> org_babel_sh_prompt> sh: slep: command not found : org_babel_sh_prompt> typos don't cause problems--^ : org_babel_sh_prompt> It's not clear to me if that's something that a better regexp would handle or if it should be cleaned up in the callback function. I'm still figuring out how it's done in ob-python and ob-R. Any feedback or advice is welcome. ------=_Part_5344843_1333486556.1675712340746 Content-Type: application/octet-stream; name=ob-shell-async-separate-calls.patch Content-Transfer-Encoding: 7bit X-ZM_AttachId: 138950051407470210 Content-Disposition: attachment; filename=ob-shell-async-separate-calls.patch diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el index 9e7b45a89..8adf7744b 100644 --- a/lisp/ob-shell.el +++ b/lisp/ob-shell.el @@ -269,12 +269,15 @@ var of the same value." (set-marker comint-last-output-start (point)) (get-buffer (current-buffer))))))) +(defconst ob-shell-async-indicator "echo 'ob_comint_async_shell_%s_%s'") + (defun org-babel-sh-evaluate (session body &optional params stdin cmdline) "Pass BODY to the Shell process in BUFFER. If RESULT-TYPE equals `output' then return a list of the outputs of the statements in BODY, if RESULT-TYPE equals `value' then return the value of the last statement in BODY." (let* ((shebang (cdr (assq :shebang params))) + (async (org-babel-comint-use-async params)) (results-params (cdr (assq :result-params params))) (value-is-exit-status (or (and @@ -305,6 +308,23 @@ return the value of the last statement in BODY." (list shell-command-switch (concat (file-local-name script-file) " " cmdline))))) (buffer-string)))) + (async + (let ((uuid (org-id-uuid))) + (org-babel-comint-async-register + session + (current-buffer) + "ob_comint_async_shell_\\(.+?\\)_\\(.+\\)" + 'org-babel-chomp + 'org-babel-chomp) + (org-babel-comint-async-delete-dangling-and-eval + session + (insert (format ob-shell-async-indicator "start" uuid)) + (comint-send-input nil t) + (insert (org-trim body)) + (comint-send-input nil t) + (insert (format ob-shell-async-indicator "end" uuid)) + (comint-send-input nil t)) + uuid)) (session ; session evaluation (mapconcat #'org-babel-sh-strip-weird-long-prompt ------=_Part_5344843_1333486556.1675712340746--