From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms13.migadu.com with LMTPS id iNxRIfscF2egDgEA62LTzQ:P1 (envelope-from ) for ; Tue, 22 Oct 2024 03:33:15 +0000 Received: from aspmx1.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1.migadu.com with LMTPS id iNxRIfscF2egDgEA62LTzQ (envelope-from ) for ; Tue, 22 Oct 2024 05:33:15 +0200 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="Ac/9RmU/"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" ARC-Seal: i=1; s=key1; d=yhetil.org; t=1729567995; a=rsa-sha256; cv=none; b=pSkAK3YSi/P6zMH5niDpGnAGv4wox6hCmPM2vMK0HbzlWsjUfnJVRzTWcE/1/rT3LP3loY 0xDI2yPzS/7rVFkydq6TWwvBP0GW8WvdItajyl4cFBaAHD7JGiB5RUAisoh0pUnvzBs3Yz LV/mPW9GVuRIY10RHC9gwHAGyjBfdEi+pNAJCCY2P4Vv4sFQ5N8TILT6WlhkbJLc95+m1L 0NwE17ZAUYl36nPOkRqtmdWTOQFZCq0Y6i0reAV1P7ZzBt7hbSpUHgXO7PFwetewCfw0Ef ZDrSTDecBD8EMA0wSxHpNFHrmkVnWCQ5pQ4RY5pXWyVoE4O3jsUy/Ya4g8JQGQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="Ac/9RmU/"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1729567995; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=30xoUy18C19dWGUnJNVFYzcSz4bx8ATJOSyG5sEDR7c=; b=No6qOiYdNoQHzJFpQ7xOPY4hboFSkZsdPfz+hrSWCHREgw/SQ++7PxZSO8Q0bQIHKCnATe yD6mn/+N6Rz4liYLuMeadOy/79PAV/OPEPM8VAx8lTzf0SOKndK5zv+dtE+iGysdtw1VbI f/4RTN1ObAxqsCdScDWTltsIwcG2dYxlQ9kjQbHP5qn8T275/ahJ4L2I6RJlfLuE6Qh4Uu Lx523wAr20w6yGL75a7EUDY3HZzranpnQGmQsJwcfljesXajCdxVNmMZgKZ1EocZZK7nFx cmErqoluLJOii5eZbymXagaaFQpquZxWY/htRpbzsYgyHGB21dowbTt8StbDsw== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 10A001FE20 for ; Tue, 22 Oct 2024 05:33:15 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t35d8-0006G3-Dr; Mon, 21 Oct 2024 23:32:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t35d2-0006Fn-0O for emacs-orgmode@gnu.org; Mon, 21 Oct 2024 23:32:17 -0400 Received: from mail-pl1-x633.google.com ([2607:f8b0:4864:20::633]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1t35cx-0008WA-IF for emacs-orgmode@gnu.org; Mon, 21 Oct 2024 23:32:14 -0400 Received: by mail-pl1-x633.google.com with SMTP id d9443c01a7336-20c70abba48so41562955ad.0 for ; Mon, 21 Oct 2024 20:32:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729567930; x=1730172730; darn=gnu.org; h=mime-version:message-id:date:references:in-reply-to:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=30xoUy18C19dWGUnJNVFYzcSz4bx8ATJOSyG5sEDR7c=; b=Ac/9RmU/jd+KEyJ1KMaTXJQ6WBBAD6WDMQy6wCO1rLVodB5AXgmxlZY2ewgV6lNohi wg/HIMwvc6UGgzi+5NuGhk39M4+f7kQaUiaYRsrLK9dtYPIj/gwjV11YgrhOxIkQja2L HK5jCa/ne2KFZJ9YJLAUu3tQ7/GQ62fy4zlpUbOWPNuKiXrXOm0+UKkaYcVbIMtFwikX o66P3iAn+AN16g5LFiUV8mnE1OzwgBRzqGONWH0LLH7Q5sZvGRD4jFJFO9Y5DUkXNwUm fhgdpwDv/YAoI72s2UCu+i2l3hILnYcbSorbPMyAXVTkFa140hNUkNLhprf6NONuTY3p LlbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729567930; x=1730172730; h=mime-version:message-id:date:references:in-reply-to:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=30xoUy18C19dWGUnJNVFYzcSz4bx8ATJOSyG5sEDR7c=; b=l6sANs2QaCpRaLcBJKLo7F0Qn8STb1E7w0X4uJ55KKcJuIJ+B1u2q3k95J9UmCIUrt N+l7TDV4O0KkW/wRaYkh/L+dJo3bbTpmyt6EZy/jcmKxNTbQulW4ak+VWkNothMFGaZz ftoAWRDU18DZ7z/SfzWEOnRmkzoeN1F9Ot6efSLMHGSgrC0zug52p1DEAmzeytd/6D6d 3VnJaCA0e8ygrnA6LnqKvfm03hNOdkLJkti7/CCsYRbmSeCxC4zdZUL56p2cSF3SGkH4 sbiXGf0LgvC/B3RPHGz/G+47+R6lF+V2sg8xSdehGnjM5ag8BU7bVw7uer1cfVWXvPDY wL+g== X-Gm-Message-State: AOJu0YytzRGUuXnYKXjs7VLCUbVggPhHaCmBBuuAnKKh+oghrWUi+/lS 24nusaNUmhJID9jHA3FfBHwiKP3flgdeWZrJ37xd9gBSa28zJcyJuSRUSA== X-Google-Smtp-Source: AGHT+IHO0zKus4yNoCoBMe9yFyn0NKGEppWWLMtFJ34aR5YHUlGkPSB1LyIIWrBqC1J3W7yO55zh4Q== X-Received: by 2002:a17:903:984:b0:20c:e875:12c2 with SMTP id d9443c01a7336-20e5a93decamr190268995ad.51.1729567929913; Mon, 21 Oct 2024 20:32:09 -0700 (PDT) Received: from localhost ([198.27.183.102]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20e7f0c0bb7sm33492015ad.143.2024.10.21.20.32.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Oct 2024 20:32:08 -0700 (PDT) From: Jack Kamm To: Ihor Radchenko Cc: emacs-orgmode@gnu.org, matt@excalamus.com, jeremiejuste@gmail.com Subject: Re: [PATCH] Async sessions: Fix prompt removal regression in ob-R In-Reply-To: <875xpnrubg.fsf@localhost> References: <87setrqs4z.fsf@gmail.com> <87wmiqigfp.fsf@localhost> <87wmi9etku.fsf@gmail.com> <87h698wml2.fsf@localhost> <87ed4b45r7.fsf@gmail.com> <875xpnrubg.fsf@localhost> Date: Mon, 21 Oct 2024 20:32:07 -0700 Message-ID: <8734koert4.fsf@gmail.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=2607:f8b0:4864:20::633; envelope-from=jackkamm@gmail.com; helo=mail-pl1-x633.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Spam-Score: -1.55 X-Spam-Score: -1.55 X-Migadu-Queue-Id: 10A001FE20 X-Migadu-Scanner: mx12.migadu.com X-TUID: aasCO0qDWl6j --=-=-= Content-Type: text/plain Ihor Radchenko writes: > I think that it is ok for bugfix as the patch essentially reverses the > commit that introduced the regression for ob-R and ob-python. (The > original patch was fixing a problem with ob-shell). So, that patch is > fairly trivial. > > We might want to document the signature change in ORG-NEWS on main > though, as an additional patch for main. > > nitpick: I'd rather add (eq prompt-handling nil) as a cond clause, to > make it more explicit. Great, thanks. I'm attaching an updated patch with the more explicit cond clause, and rebased onto latest bugfix. I also attach a second patch with the NEWS entry on main. --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0001-Disable-async-prompt-removal-in-ob-R-python.patch >From 000b7ea1025a7f076c0af0b69d6a2a653c415b40 Mon Sep 17 00:00:00 2001 From: Jack Kamm Date: Sun, 22 Sep 2024 13:48:45 -0700 Subject: [PATCH] Disable async prompt removal in ob-R,python * lisp/ob-comint.el (org-babel-comint-async-remove-prompts-p): New variable to disable prompt removal in async output. (org-babel-comint-async-filter): Check `org-babel-comint-async-remove-prompts-p' before calling `org-babel-comint--prompt-filter'. (org-babel-comint-async-register): Added argument for whether prompts should be removed from async output. * lisp/ob-python.el (org-babel-python-async-evaluate-session): Set option to inhibit prompt removal when registering async evaluators. * lisp/ob-R.el (ob-session-async-org-babel-R-evaluate-session): Set option to inhibit prompt removal when registering async evaluators. * testing/lisp/test-ob-R.el (test-ob-R/async-prompt-filter): Test for over-aggressive prompt removal. --- lisp/ob-R.el | 3 ++- lisp/ob-comint.el | 34 ++++++++++++++++++++++++++++------ lisp/ob-python.el | 3 ++- testing/lisp/test-ob-R.el | 28 ++++++++++++++++++++++++++++ 4 files changed, 60 insertions(+), 8 deletions(-) diff --git a/lisp/ob-R.el b/lisp/ob-R.el index de2d27a9a..5a8dfe22c 100644 --- a/lisp/ob-R.el +++ b/lisp/ob-R.el @@ -486,7 +486,8 @@ (defun ob-session-async-org-babel-R-evaluate-session session (current-buffer) "^\\(?:[>.+] \\)*\\[1\\] \"ob_comint_async_R_\\(start\\|end\\|file\\)_\\(.+\\)\"$" 'org-babel-chomp - 'ob-session-async-R-value-callback) + 'ob-session-async-R-value-callback + 'disable-prompt-filtering) (cl-case result-type (value (let ((tmp-file (org-babel-temp-file "R-"))) diff --git a/lisp/ob-comint.el b/lisp/ob-comint.el index 764927af7..efec7badc 100644 --- a/lisp/ob-comint.el +++ b/lisp/ob-comint.el @@ -239,6 +239,9 @@ (defvar-local org-babel-comint-async-chunk-callback nil comint process. It should return a string that will be passed to `org-babel-insert-result'.") +(defvar-local org-babel-comint-async-remove-prompts-p t + "Whether prompts should be detected and removed from async output.") + (defvar-local org-babel-comint-async-dangling nil "Dangling piece of the last process output, as a string. Used when `org-babel-comint-async-indicator' is spread across multiple @@ -326,10 +329,16 @@ (defun org-babel-comint-async-filter (string) until (and (equal (match-string 1) "start") (equal (match-string 2) uuid)) finally return (+ 1 (match-end 0))))) - ;; Remove prompt - (res-promptless (org-trim (string-join (mapcar #'org-trim (org-babel-comint--prompt-filter res-str-raw)) "\n") "\n")) ;; Apply user callback - (res-str (funcall org-babel-comint-async-chunk-callback res-promptless))) + (res-str (funcall org-babel-comint-async-chunk-callback + (if org-babel-comint-async-remove-prompts-p + (org-trim (string-join + (mapcar #'org-trim + (org-babel-comint--prompt-filter + res-str-raw)) + "\n") + t) + res-str-raw)))) ;; Search for uuid in associated org-buffers to insert results (cl-loop for buf in org-buffers until (with-current-buffer buf @@ -350,18 +359,31 @@ (defun org-babel-comint-async-filter (string) (defun org-babel-comint-async-register (session-buffer org-buffer indicator-regexp - chunk-callback file-callback) + chunk-callback file-callback + &optional prompt-handling) "Set local org-babel-comint-async variables in SESSION-BUFFER. ORG-BUFFER is added to `org-babel-comint-async-buffers' if not present. `org-babel-comint-async-indicator', `org-babel-comint-async-chunk-callback', and `org-babel-comint-async-file-callback' are set to -INDICATOR-REGEXP, CHUNK-CALLBACK, and FILE-CALLBACK -respectively." +INDICATOR-REGEXP, CHUNK-CALLBACK, and FILE-CALLBACK respectively. +PROMPT-HANDLING may be either of the symbols `filter-prompts', in +which case prompts matching `comint-prompt-regexp' are filtered +from output before it is passed to CHUNK-CALLBACK, or +`disable-prompt-filtering', in which case this behavior is +disabled. For backward-compatibility, the default value of `nil' +is equivalent to `filter-prompts'." (org-babel-comint-in-buffer session-buffer (setq org-babel-comint-async-indicator indicator-regexp org-babel-comint-async-chunk-callback chunk-callback org-babel-comint-async-file-callback file-callback) + (setq org-babel-comint-async-remove-prompts-p + (cond + ((eq prompt-handling 'disable-prompt-filtering) nil) + ((eq prompt-handling 'filter-prompts) t) + ((eq prompt-handling nil) t) + (t (error (format "Unrecognized prompt handling behavior %s" + (symbol-name prompt-handling)))))) (unless (memq org-buffer org-babel-comint-async-buffers) (setq org-babel-comint-async-buffers (cons org-buffer org-babel-comint-async-buffers))) diff --git a/lisp/ob-python.el b/lisp/ob-python.el index 8a3c24f70..f41f44dbd 100644 --- a/lisp/ob-python.el +++ b/lisp/ob-python.el @@ -538,7 +538,8 @@ (defun org-babel-python-async-evaluate-session (org-babel-comint-async-register session (current-buffer) "ob_comint_async_python_\\(start\\|end\\|file\\)_\\(.+\\)" - 'org-babel-chomp 'org-babel-python-async-value-callback) + 'org-babel-chomp 'org-babel-python-async-value-callback + 'disable-prompt-filtering) (pcase result-type (`output (let ((uuid (org-id-uuid))) diff --git a/testing/lisp/test-ob-R.el b/testing/lisp/test-ob-R.el index 9ffbf3afd..05b91afd6 100644 --- a/testing/lisp/test-ob-R.el +++ b/testing/lisp/test-ob-R.el @@ -316,6 +316,34 @@ (org-test-with-temp-text-in-file (string= (concat text result) (buffer-string))))))) +(ert-deftest test-ob-R/async-prompt-filter () + "Test that async evaluation doesn't remove spurious prompts and leading indentation." + (let* (ess-ask-for-ess-directory + ess-history-file + org-confirm-babel-evaluate + (session-name "*R:test-ob-R/session-async-results*") + (kill-buffer-query-functions nil) + (start-time (current-time)) + (wait-time (time-add start-time 3)) + uuid-placeholder) + (org-test-with-temp-text + (concat "#+begin_src R :session " session-name " :async t :results output +table(c('ab','ab','c',NA,NA), useNA='always') +#+end_src") + (setq uuid-placeholder (org-trim (org-babel-execute-src-block))) + (catch 'too-long + (while (string-match uuid-placeholder (buffer-string)) + (progn + (sleep-for 0.01) + (when (time-less-p wait-time (current-time)) + (throw 'too-long (ert-fail "Took too long to get result from callback")))))) + (search-forward "#+results") + (beginning-of-line 2) + (when (should (re-search-forward "\ +:\\([ ]+ab\\)[ ]+c[ ]+[ ]* +:\\([ ]+2\\)[ ]+1[ ]+2")) + (should (equal (length (match-string 1)) (length (match-string 2)))) + (kill-buffer session-name))))) (provide 'test-ob-R) -- 2.46.2 --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0001-NEWS-entry-for-the-new-argument-of-org-babel-comint-.patch >From ab249b7f518115f8dfd85191a64656f8f40ab58e Mon Sep 17 00:00:00 2001 From: Jack Kamm Date: Mon, 21 Oct 2024 20:22:00 -0700 Subject: [PATCH] NEWS entry for the new argument of org-babel-comint-async-register The optional argument was added on bugfix branch, but we are adding this extra NEWS entry on main. See also: https://list.orgmode.org/875xpnrubg.fsf@localhost/T/#m179d313e1db284ff28eb4098129bb49418824a25 --- etc/ORG-NEWS | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS index 5d421172f..17ffc4068 100644 --- a/etc/ORG-NEWS +++ b/etc/ORG-NEWS @@ -186,6 +186,24 @@ accept the INFO channel and return a string. This makes it possible to dynamically generate the content of the resulting ~~ tag in the resulting HTML document. +*** ~org-babel-comint-async-register~: Added argument to specify prompt handling + +The new argument ~prompt-handling~ allows Babel languages to specify +how prompts should be handled when passing output to +~org-babel-comint-async-chunk-callback~. If equal to +~filter-prompts~, prompts are removed beforehand, similar to the +default behavior of ~org-babel-comint-with-output~. If equal to +~disable-prompt-filtering~, then the prompt filtering is skipped. If +unset, then the default behavior is the same as ~filter-prompts~ for +backwards compatibility. + +Prompt filtering is needed for some Babel languages, such as ob-shell, +which leave extra prompts in the output as a side effect of +evaluation. However other Babel languages, like ob-python, don't +leave extra prompts after evaluation, and skipping the prompt +filtering can be more robust for such languages (as this avoids +removing false positive prompts). + ** Miscellaneous *** Org mode no longer prevents =flyspell= from spell-checking inside =LOGBOOK= drawers -- 2.46.2 --=-=-=--