* [BUG] ob-python hangs on second start [9.5.4 (release_9.5.4-763-g06373a @ ~/emacs/org-mode/lisp/)]
@ 2023-08-25 4:44 Peter Mao
2023-08-25 8:52 ` Ihor Radchenko
0 siblings, 1 reply; 6+ messages in thread
From: Peter Mao @ 2023-08-25 4:44 UTC (permalink / raw)
To: org-mode-email
[-- Attachment #1: Type: text/plain, Size: 5108 bytes --]
Expectation: When running ob-python code blocks, I should be able to
kill the python session in the *Python* buffer and run another code
block (or the same one).
Problem: ob-python works fine on the first execution, but after
`exit()`ing the python session, it hangs without executing the code.
After a `C-g`, the prompt in the *Python* session shows up, but one then
has to re-execute the code block, as none of it has run.
The following three code blocks demonstrate the problem. I have
bisected the problem, finding 06373a64 as the first bad commit. I tried
to reproduce the problem on Emacs 28.2, but this commit of Org won't
compile on Emacs 28.2 for me.
#+begin_src elisp :results none
(org-babel-do-load-languages 'org-babel-load-languages '((python . t)))
(setq-local org-confirm-babel-evaluate nil)
#+end_src
#+begin_src python :results output
print("Running first, this one issues a warning but competes
successfully.")
exit()
#+end_src
#+begin_src python :results output
print("Running second, this one hangs until user enters C-g.")
print("None of the code executes.")
exit()
#+end_src
Emacs : GNU Emacs 29.1 (build 2, x86_64-pc-linux-gnu, GTK+ Version
3.24.33, cairo version 1.16.0)
of 2023-07-31
Package: Org mode version 9.5.4 (release_9.5.4-763-g06373a @
/home/peterm/emacs/org-mode/lisp/)
current state:
==============
(setq
org-link-elisp-confirm-function 'yes-or-no-p
org-bibtex-headline-format-function #[257 "\300 \236A\207" [:title] 3
"\n\n(fn ENTRY)"]
org-persist-after-read-hook '(org-element--cache-persist-after-read)
org-export-before-parsing-hook '(org-attach-expand-links)
org-cycle-tab-first-hook '(org-babel-hide-result-toggle-maybe
org-babel-header-arg-expand)
org-archive-hook '(org-attach-archive-delete-maybe)
org-cycle-hook '(org-cycle-hide-archived-subtrees
org-cycle-show-empty-lines
org-cycle-optimize-window-after-visibility-change)
org-persist-before-read-hook '(org-element--cache-persist-before-read)
org-mode-hook '(#[0 "\300\301\302\303\304$\207"
[add-hook change-major-mode-hook org-fold-show-all append
local]
5]
#[0 "\300\301\302\303\304$\207"
[add-hook change-major-mode-hook org-babel-show-result-all
append local]
5]
org-babel-result-hide-spec org-babel-hide-all-hashes)
org-babel-load-languages '((python . t))
org-confirm-shell-link-function 'yes-or-no-p
outline-isearch-open-invisible-function 'outline-isearch-open-invisible
org-agenda-before-write-hook '(org-agenda-add-entry-text)
org-babel-tangle-lang-exts '(("python" . "py") ("emacs-lisp" . "el")
("elisp" . "el"))
org-src-mode-hook '(org-src-babel-configure-edit-buffer
org-src-mode-configure-edit-buffer)
org-confirm-elisp-link-function 'yes-or-no-p
org-speed-command-hook '(org-speed-command-activate
org-babel-speed-command-activate)
org-fold-core-isearch-open-function 'org-fold--isearch-reveal
org-persist-before-write-hook '(org-element--cache-persist-before-write)
org-tab-first-hook '(org-babel-hide-result-toggle-maybe
org-babel-header-arg-expand)
org-link-shell-confirm-function 'yes-or-no-p
org-babel-pre-tangle-hook '(save-buffer)
org-agenda-loop-over-headlines-in-active-region nil
org-occur-hook '(org-first-headline-recenter)
org-metadown-hook '(org-babel-pop-to-session-maybe)
org-link-parameters '(("attachment" :follow org-attach-follow :complete
org-attach-complete-link)
("id" :follow org-id-open)
("eww" :follow org-eww-open :store org-eww-store-link)
("rmail" :follow org-rmail-open :store
org-rmail-store-link)
("mhe" :follow org-mhe-open :store org-mhe-store-link)
("irc" :follow org-irc-visit :store org-irc-store-link
:export org-irc-export)
("info" :follow org-info-open :export org-info-export
:store org-info-store-link :insert-description
org-info-description-as-command)
("gnus" :follow org-gnus-open :store
org-gnus-store-link)
("docview" :follow org-docview-open :export
org-docview-export :store org-docview-store-link)
("bibtex" :follow org-bibtex-open :store
org-bibtex-store-link)
("bbdb" :follow org-bbdb-open :export org-bbdb-export
:complete org-bbdb-complete-link :store
org-bbdb-store-link)
("w3m" :store org-w3m-store-link)
("doi" :follow org-link-doi-open :export
org-link-doi-export)
("file+sys") ("file+emacs")
("shell" :follow org-link--open-shell)
("news" :follow
#[514 "\301\300\302 Q \"\207"
["news" browse-url ":"] 6 "\n\n(fn URL ARG)"]
)
("mailto" :follow
#[514 "\301\300\302 Q \"\207"
["mailto" browse-url ":"] 6 "\n\n(fn URL ARG)"]
)
("https" :follow
#[514 "\301\300\302 Q \"\207"
["https" browse-url ":"] 6 "\n\n(fn URL ARG)"]
)
("http" :follow
#[514 "\301\300\302 Q \"\207"
["http" browse-url ":"] 6 "\n\n(fn URL ARG)"]
)
("ftp" :follow
#[514 "\301\300\302 Q \"\207" ["ftp" browse-url ":"]
6 "\n\n(fn URL ARG)"]
)
("help" :follow org-link--open-help :store
org-link--store-help)
("file" :complete org-link-complete-file)
("elisp" :follow org-link--open-elisp))
org-metaup-hook '(org-babel-load-in-session-maybe)
)
[-- Attachment #2: Type: text/html, Size: 6557 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [BUG] ob-python hangs on second start [9.5.4 (release_9.5.4-763-g06373a @ ~/emacs/org-mode/lisp/)]
2023-08-25 4:44 [BUG] ob-python hangs on second start [9.5.4 (release_9.5.4-763-g06373a @ ~/emacs/org-mode/lisp/)] Peter Mao
@ 2023-08-25 8:52 ` Ihor Radchenko
2023-08-27 21:26 ` Jack Kamm
0 siblings, 1 reply; 6+ messages in thread
From: Ihor Radchenko @ 2023-08-25 8:52 UTC (permalink / raw)
To: Peter Mao, Jack Kamm; +Cc: org-mode-email
Peter Mao <peter.mao@gmail.com> writes:
> Expectation: When running ob-python code blocks, I should be able to
> kill the python session in the *Python* buffer and run another code
> block (or the same one).
>
> Problem: ob-python works fine on the first execution, but after
> `exit()`ing the python session, it hangs without executing the code.
> After a `C-g`, the prompt in the *Python* session shows up, but one then
> has to re-execute the code block, as none of it has run.
Confirmed.
It looks like `python-shell-first-prompt-hook' does not get triggered
in the described scenario with exit() and we enter infinite loop in the
code below.
(add-hook
'python-shell-first-prompt-hook
#'org-babel-python--setup-session
nil 'local)))
;; Wait until Python initializes
;; This is more reliable compared to
;; `org-babel-comint-wait-for-output' as python may emit
;; multiple prompts during initialization.
(with-current-buffer py-buffer
(while (not org-babel-python--initialized)
(org-babel-comint-wait-for-output py-buffer)))
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [BUG] ob-python hangs on second start [9.5.4 (release_9.5.4-763-g06373a @ ~/emacs/org-mode/lisp/)]
2023-08-25 8:52 ` Ihor Radchenko
@ 2023-08-27 21:26 ` Jack Kamm
2023-08-28 0:29 ` Peter Mao
2023-08-28 9:02 ` Ihor Radchenko
0 siblings, 2 replies; 6+ messages in thread
From: Jack Kamm @ 2023-08-27 21:26 UTC (permalink / raw)
To: Ihor Radchenko, Peter Mao; +Cc: org-mode-email
[-- Attachment #1: Type: text/plain, Size: 1087 bytes --]
Ihor Radchenko <yantar92@posteo.net> writes:
> Peter Mao <peter.mao@gmail.com> writes:
>
>> Expectation: When running ob-python code blocks, I should be able to
>> kill the python session in the *Python* buffer and run another code
>> block (or the same one).
>>
>> Problem: ob-python works fine on the first execution, but after
>> `exit()`ing the python session, it hangs without executing the code.
>> After a `C-g`, the prompt in the *Python* session shows up, but one then
>> has to re-execute the code block, as none of it has run.
>
> Confirmed.
>
> It looks like `python-shell-first-prompt-hook' does not get triggered
> in the described scenario with exit() and we enter infinite loop in the
> code below.
I think `python-shell-first-prompt-hook' is actually still triggered,
but rather it is `org-babel-comint-wait-for-output' that is hanging
while waiting for startup. However, we can just replace it with a
`sleep-for' instead, because we now wait for
`org-babel-python--initialized' anyways.
See the attached patch. Please let me know if it fixes the problem for
you.
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ob-python-Fix-hanging-on-second-start.patch --]
[-- Type: text/x-patch, Size: 2499 bytes --]
From d5721aa37a399afcd527906e5d9f1b6bce37fdb9 Mon Sep 17 00:00:00 2001
From: Jack Kamm <jackkamm@gmail.com>
Date: Sun, 27 Aug 2023 14:13:15 -0700
Subject: [PATCH] ob-python: Fix hanging on second start
See: https://list.orgmode.org/87ttsnh5bx.fsf@localhost/T/#t
* lisp/ob-python.el (org-babel-python-initiate-session-by-key): Switch
from `org-babel-comint-wait-for-output' to `sleep-for' while waiting
for `org-babel-python--initialized', to prevent hanging on restarted
Python process.
* testing/lisp/test-ob-python.el (test-ob-python/session-restart): New
test for restarting ob-python session process.
---
lisp/ob-python.el | 2 +-
testing/lisp/test-ob-python.el | 19 +++++++++++++++++++
2 files changed, 20 insertions(+), 1 deletion(-)
diff --git a/lisp/ob-python.el b/lisp/ob-python.el
index b9dab91b4..6b216ce89 100644
--- a/lisp/ob-python.el
+++ b/lisp/ob-python.el
@@ -272,7 +272,7 @@ (defun org-babel-python-initiate-session-by-key (&optional session)
;; multiple prompts during initialization.
(with-current-buffer py-buffer
(while (not org-babel-python--initialized)
- (org-babel-comint-wait-for-output py-buffer)))
+ (sleep-for 0 10)))
(setq org-babel-python-buffers
(cons (cons session py-buffer)
(assq-delete-all session org-babel-python-buffers)))
diff --git a/testing/lisp/test-ob-python.el b/testing/lisp/test-ob-python.el
index 82fbca36e..c11e1d0c2 100644
--- a/testing/lisp/test-ob-python.el
+++ b/testing/lisp/test-ob-python.el
@@ -310,6 +310,25 @@ (ert-deftest test-ob-python/async-local-python-shell ()
#+end_src"
(should (org-babel-execute-src-block))))
+(ert-deftest test-ob-python/session-restart ()
+ ;; Disable the test on older Emacs as built-in python.el sometimes
+ ;; fail to initialize session.
+ (skip-unless (version<= "28" emacs-version))
+ (should
+ (equal "success"
+ (progn
+ (org-test-with-temp-text "#+begin_src python :session :results output
+print('start')
+#+end_src"
+ (org-babel-execute-src-block))
+ (let ((proc (python-shell-get-process)))
+ (python-shell-send-string "exit()")
+ (while (accept-process-output proc)))
+ (org-test-with-temp-text "#+begin_src python :session :results output
+print('success')
+#+end_src"
+ (org-babel-execute-src-block))))))
+
(provide 'test-ob-python)
;;; test-ob-python.el ends here
--
2.41.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [BUG] ob-python hangs on second start [9.5.4 (release_9.5.4-763-g06373a @ ~/emacs/org-mode/lisp/)]
2023-08-27 21:26 ` Jack Kamm
@ 2023-08-28 0:29 ` Peter Mao
2023-08-28 9:02 ` Ihor Radchenko
1 sibling, 0 replies; 6+ messages in thread
From: Peter Mao @ 2023-08-28 0:29 UTC (permalink / raw)
To: Jack Kamm; +Cc: Ihor Radchenko, org-mode-email
[-- Attachment #1: Type: text/plain, Size: 867 bytes --]
On Sun, Aug 27, 2023 at 2:26 PM Jack Kamm <jackkamm@gmail.com> wrote:
> Ihor Radchenko <yantar92@posteo.net> writes:
>
> > Confirmed.
> >
> > It looks like `python-shell-first-prompt-hook' does not get triggered
> > in the described scenario with exit() and we enter infinite loop in the
> > code below.
>
> I think `python-shell-first-prompt-hook' is actually still triggered,
> but rather it is `org-babel-comint-wait-for-output' that is hanging
> while waiting for startup. However, we can just replace it with a
> `sleep-for' instead, because we now wait for
> `org-babel-python--initialized' anyways.
>
> See the attached patch. Please let me know if it fixes the problem for
> you.
>
Yes, this fixes the problem for me. I made the change locally to
release_9.6.6 on emacs 29.1. I have the expected behavior back. thank you!
Peter
[-- Attachment #2: Type: text/html, Size: 1283 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [BUG] ob-python hangs on second start [9.5.4 (release_9.5.4-763-g06373a @ ~/emacs/org-mode/lisp/)]
2023-08-27 21:26 ` Jack Kamm
2023-08-28 0:29 ` Peter Mao
@ 2023-08-28 9:02 ` Ihor Radchenko
2023-08-31 4:52 ` Jack Kamm
1 sibling, 1 reply; 6+ messages in thread
From: Ihor Radchenko @ 2023-08-28 9:02 UTC (permalink / raw)
To: Jack Kamm; +Cc: Peter Mao, org-mode-email
Jack Kamm <jackkamm@gmail.com> writes:
> I think `python-shell-first-prompt-hook' is actually still triggered,
> but rather it is `org-babel-comint-wait-for-output' that is hanging
> while waiting for startup. However, we can just replace it with a
> `sleep-for' instead, because we now wait for
> `org-babel-python--initialized' anyways.
>
> See the attached patch. Please let me know if it fixes the problem for
> you.
LGTM! Feel free to push.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [BUG] ob-python hangs on second start [9.5.4 (release_9.5.4-763-g06373a @ ~/emacs/org-mode/lisp/)]
2023-08-28 9:02 ` Ihor Radchenko
@ 2023-08-31 4:52 ` Jack Kamm
0 siblings, 0 replies; 6+ messages in thread
From: Jack Kamm @ 2023-08-31 4:52 UTC (permalink / raw)
To: Ihor Radchenko; +Cc: Peter Mao, org-mode-email
Ihor Radchenko <yantar92@posteo.net> writes:
> LGTM! Feel free to push.
Pushed to bugfix (c81dba2fb) and merged to main (b49275acb). Pushed the
unit test as a separate commit to main only (8000b1120).
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2023-08-31 4:53 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-08-25 4:44 [BUG] ob-python hangs on second start [9.5.4 (release_9.5.4-763-g06373a @ ~/emacs/org-mode/lisp/)] Peter Mao
2023-08-25 8:52 ` Ihor Radchenko
2023-08-27 21:26 ` Jack Kamm
2023-08-28 0:29 ` Peter Mao
2023-08-28 9:02 ` Ihor Radchenko
2023-08-31 4:52 ` Jack Kamm
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).