[PATCH] Set Python shell in Org edit buffer

[PATCH] Set Python shell in Org edit buffer

[Liu Hui]

[-- Attachment #1: Type: text/plain, Size: 676 bytes --]

Hi,

When editing python src block using C-c ', the python shell is not set
in Org edit buffer according to the :session header argument of the
block. Consequently, commands such as python-shell-send-region cannot
send python code to the correct python process. To address this, the
attached patch defines org-babel-edit-prep:python in ob-python.el.

In addition, I tried to use org-src-associate-babel-session at first,
but found it doesn't work because it is called when enabling
org-src-mode, where org-src--babel-info is still nil. It seems it has
stopped working since 203bf5870, and I think it is safe to remove it
and related stuffs to avoid confusion.


Best,
Liu Hui

[-- Attachment #2: 0001-Set-Python-shell-in-Org-edit-buffer.patch --]
[-- Type: text/x-patch, Size: 1072 bytes --]

From c8b9c174cf643bd625cedc311d2604e6fc3bb83a Mon Sep 17 00:00:00 2001
From: Liu Hui <liuhui1610@gmail.com>
Date: Tue, 5 Dec 2023 11:40:38 +0800
Subject: [PATCH] Set Python shell in Org edit buffer

* lisp/ob-python.el (org-babel-edit-prep:python): New function.
---
 lisp/ob-python.el | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/lisp/ob-python.el b/lisp/ob-python.el
index 6c568a36d..8ff2c7a1d 100644
--- a/lisp/ob-python.el
+++ b/lisp/ob-python.el
@@ -67,6 +67,13 @@ (defcustom org-babel-python-None-to 'hline
   :package-version '(Org . "8.0")
   :type 'symbol)
 
+(defun org-babel-edit-prep:python (info)
+  "Set Python shell in Org edit buffer according to INFO."
+  (let ((session (cdr (assq :session (nth 2 info)))))
+    (when (and session (not (string= session "none")))
+      (setq-local python-shell-buffer-name
+                  (org-babel-python-without-earmuffs session)))))
+
 (defun org-babel-execute:python (body params)
   "Execute Python BODY according to PARAMS.
 This function is called by `org-babel-execute-src-block'."
-- 
2.25.1


[-- Attachment #3: 0001-Remove-org-src-babel-configure-edit-buffer-and-relat.patch --]
[-- Type: text/x-patch, Size: 2328 bytes --]

From f3c0d401de55d7ad8873c18658abd1d9f6b49d77 Mon Sep 17 00:00:00 2001
From: Liu Hui <liuhui1610@gmail.com>
Date: Tue, 5 Dec 2023 11:36:07 +0800
Subject: [PATCH] Remove 'org-src-babel-configure-edit-buffer' and related
 stuffs

The functionality has stopped working since 203bf5870.
org-babel-edit-prep:lang should be used instead.

* lisp/org-src.el (org-src-associate-babel-session):
(org-src-babel-configure-edit-buffer):
* lisp/ob-R.el (org-babel-R-associate-session): Remove.
---
 lisp/ob-R.el    |  8 --------
 lisp/org-src.el | 23 -----------------------
 2 files changed, 31 deletions(-)

diff --git a/lisp/ob-R.el b/lisp/ob-R.el
index c48b2cdb7..f3cce20b7 100644
--- a/lisp/ob-R.el
+++ b/lisp/ob-R.el
@@ -294,14 +294,6 @@ (defun org-babel-R-initiate-session (session params)
 	       (buffer-name))))
 	  (current-buffer))))))
 
-(defun org-babel-R-associate-session (session)
-  "Associate R code buffer with an R session.
-Make SESSION be the inferior ESS process associated with the
-current code buffer."
-  (setq ess-local-process-name
-	(process-name (get-buffer-process session)))
-  (ess-make-buffer-current))
-
 (defvar org-babel-R-graphics-devices
   '((:bmp "bmp" "filename")
     (:jpg "jpeg" "filename")
diff --git a/lisp/org-src.el b/lisp/org-src.el
index 866ff2dbf..6406b8d29 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -914,29 +914,6 @@ (defun org-src-mode-configure-edit-buffer ()
 
 (add-hook 'org-src-mode-hook #'org-src-mode-configure-edit-buffer)
 
-
-\f
-;;; Babel related functions
-
-(defun org-src-associate-babel-session (info)
-  "Associate edit buffer with comint session.
-INFO should be a list similar in format to the return value of
-`org-babel-get-src-block-info'."
-  (interactive)
-  (let ((session (cdr (assq :session (nth 2 info)))))
-    (and session (not (string= session "none"))
-	 (org-babel-comint-buffer-livep session)
-	 (let ((f (intern (format "org-babel-%s-associate-session"
-                                  (nth 0 info)))))
-           (and (fboundp f) (funcall f session))))))
-
-(defun org-src-babel-configure-edit-buffer ()
-  "Configure src editing buffer."
-  (when org-src--babel-info
-    (org-src-associate-babel-session org-src--babel-info)))
-
-(add-hook 'org-src-mode-hook #'org-src-babel-configure-edit-buffer)
-
 \f
 ;;; Public API
 
-- 
2.25.1

Re: [PATCH] Set Python shell in Org edit buffer

[Ihor Radchenko]

Liu Hui <liuhui1610@gmail.com> writes:

> When editing python src block using C-c ', the python shell is not set
> in Org edit buffer according to the :session header argument of the
> block. Consequently, commands such as python-shell-send-region cannot
> send python code to the correct python process. To address this, the
> attached patch defines org-babel-edit-prep:python in ob-python.el.

Makes sense.
I think we may drop a note about this new feature in ORG-NEWS.

> In addition, I tried to use org-src-associate-babel-session at first,
> but found it doesn't work because it is called when enabling
> org-src-mode, where org-src--babel-info is still nil. It seems it has
> stopped working since 203bf5870, and I think it is safe to remove it
> and related stuffs to avoid confusion.

I do not see it being used much, except
https://github.com/frederic-santos/ob-ess-julia and
https://github.com/alphapapa/outshine
Although, ob-ess-julia is deprecated and outshine only mentions this in
"TODO" comment.

To be safe, I'd prefer to mark `org-src-associate-babel-session'
obsolete and mention the removal in ORG-NEWS, pointing users to
`org-babel-edit-prep:..' instead.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Set Python shell in Org edit buffer

[Liu Hui]

[-- Attachment #1: Type: text/plain, Size: 1394 bytes --]

Ihor Radchenko <yantar92@posteo.net> 于2023年12月5日周二 19:48写道：
>
> Liu Hui <liuhui1610@gmail.com> writes:
>
> > When editing python src block using C-c ', the python shell is not set
> > in Org edit buffer according to the :session header argument of the
> > block. Consequently, commands such as python-shell-send-region cannot
> > send python code to the correct python process. To address this, the
> > attached patch defines org-babel-edit-prep:python in ob-python.el.
>
> Makes sense.
> I think we may drop a note about this new feature in ORG-NEWS.

Thanks, I have added it in the attached patch.

> > In addition, I tried to use org-src-associate-babel-session at first,
> > but found it doesn't work because it is called when enabling
> > org-src-mode, where org-src--babel-info is still nil. It seems it has
> > stopped working since 203bf5870, and I think it is safe to remove it
> > and related stuffs to avoid confusion.
>
> I do not see it being used much, except
> https://github.com/frederic-santos/ob-ess-julia and
> https://github.com/alphapapa/outshine
> Although, ob-ess-julia is deprecated and outshine only mentions this in
> "TODO" comment.
>
> To be safe, I'd prefer to mark `org-src-associate-babel-session'
> obsolete and mention the removal in ORG-NEWS, pointing users to
> `org-babel-edit-prep:..' instead.

Fine by me.

[-- Attachment #2: 0001-Set-Python-shell-in-Org-edit-buffer.patch --]
[-- Type: text/x-patch, Size: 1857 bytes --]

From 295b28fece2f2125b40635118a5a705af4c8ce74 Mon Sep 17 00:00:00 2001
From: Liu Hui <liuhui1610@gmail.com>
Date: Tue, 5 Dec 2023 11:40:38 +0800
Subject: [PATCH] Set Python shell in Org edit buffer

* lisp/ob-python.el (org-babel-edit-prep:python): New function.
* etc/ORG-NEWS (ob-python now sets ~python-shell-buffer-name~ in Org
edit buffers): Announce the change.
---
 etc/ORG-NEWS      | 6 ++++++
 lisp/ob-python.el | 7 +++++++
 2 files changed, 13 insertions(+)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index aef7e1184..826ea3526 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -640,6 +640,12 @@ return a matplotlib Figure object to plot. For output results, the
 current figure (as returned by =pyplot.gcf()=) is cleared before
 evaluation, and then plotted afterwards.
 
+*** ob-python now sets ~python-shell-buffer-name~ in Org edit buffers
+
+When editing a Python src block, the editing buffer is now associated
+with the Python shell specified by the src block's ~:session~ header
+argument.
+
 *** =ob-maxima.el=: Support for ~batch~ and ~draw~
 
 =ob-maxima= has two new header arguments: ~:batch~ and
diff --git a/lisp/ob-python.el b/lisp/ob-python.el
index 6c568a36d..8ff2c7a1d 100644
--- a/lisp/ob-python.el
+++ b/lisp/ob-python.el
@@ -67,6 +67,13 @@ (defcustom org-babel-python-None-to 'hline
   :package-version '(Org . "8.0")
   :type 'symbol)
 
+(defun org-babel-edit-prep:python (info)
+  "Set Python shell in Org edit buffer according to INFO."
+  (let ((session (cdr (assq :session (nth 2 info)))))
+    (when (and session (not (string= session "none")))
+      (setq-local python-shell-buffer-name
+                  (org-babel-python-without-earmuffs session)))))
+
 (defun org-babel-execute:python (body params)
   "Execute Python BODY according to PARAMS.
 This function is called by `org-babel-execute-src-block'."
-- 
2.25.1

Re: [PATCH] Set Python shell in Org edit buffer

[Ihor Radchenko]

Liu Hui <liuhui1610@gmail.com> writes:

>> Makes sense.
>> I think we may drop a note about this new feature in ORG-NEWS.
>
> Thanks, I have added it in the attached patch.
> ...
> +When editing a Python src block, the editing buffer is now associated
> +with the Python shell specified by the src block's ~:session~ header
> +argument.

May you also mention what this means in practice? Like that users can
now send region for evaluation right from the edit src buffer?
Otherwise, it may not be very clear for ordinary users what this feature
adds.

>> To be safe, I'd prefer to mark `org-src-associate-babel-session'
>> obsolete and mention the removal in ORG-NEWS, pointing users to
>> `org-babel-edit-prep:..' instead.
>
> Fine by me.

Done, on main.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=7926045ff

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Set Python shell in Org edit buffer

[Liu Hui]

[-- Attachment #1: Type: text/plain, Size: 749 bytes --]

Ihor Radchenko <yantar92@posteo.net> 于2023年12月6日周三 21:20写道：
>
> Liu Hui <liuhui1610@gmail.com> writes:
>
> >> Makes sense.
> >> I think we may drop a note about this new feature in ORG-NEWS.
> >
> > Thanks, I have added it in the attached patch.
> > ...
> > +When editing a Python src block, the editing buffer is now associated
> > +with the Python shell specified by the src block's ~:session~ header
> > +argument.
>
> May you also mention what this means in practice? Like that users can
> now send region for evaluation right from the edit src buffer?
> Otherwise, it may not be very clear for ordinary users what this feature
> adds.

Yes, I have updated the text and you're welcome to improve it. Thanks!

[-- Attachment #2: 0001-Set-Python-shell-in-Org-edit-buffer.patch --]
[-- Type: text/x-patch, Size: 1984 bytes --]

From c503b2ed5116e2abae25459b09abc919074ac54a Mon Sep 17 00:00:00 2001
From: Liu Hui <liuhui1610@gmail.com>
Date: Tue, 5 Dec 2023 11:40:38 +0800
Subject: [PATCH] Set Python shell in Org edit buffer

* lisp/ob-python.el (org-babel-edit-prep:python): New function.
* etc/ORG-NEWS (ob-python now sets ~python-shell-buffer-name~ in Org
edit buffers): Announce the change.
---
 etc/ORG-NEWS      | 7 +++++++
 lisp/ob-python.el | 7 +++++++
 2 files changed, 14 insertions(+)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index aef7e1184..f30019d70 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -640,6 +640,13 @@ return a matplotlib Figure object to plot. For output results, the
 current figure (as returned by =pyplot.gcf()=) is cleared before
 evaluation, and then plotted afterwards.
 
+*** ob-python now sets ~python-shell-buffer-name~ in Org edit buffers
+
+When editing a Python src block, the editing buffer is now associated
+with the Python shell specified by the src block's ~:session~ header,
+which means users can now send code directly from the edit buffer,
+e.g., using ~C-c C-c~, to the session specified in the Org buffer.
+
 *** =ob-maxima.el=: Support for ~batch~ and ~draw~
 
 =ob-maxima= has two new header arguments: ~:batch~ and
diff --git a/lisp/ob-python.el b/lisp/ob-python.el
index 6c568a36d..8ff2c7a1d 100644
--- a/lisp/ob-python.el
+++ b/lisp/ob-python.el
@@ -67,6 +67,13 @@ (defcustom org-babel-python-None-to 'hline
   :package-version '(Org . "8.0")
   :type 'symbol)
 
+(defun org-babel-edit-prep:python (info)
+  "Set Python shell in Org edit buffer according to INFO."
+  (let ((session (cdr (assq :session (nth 2 info)))))
+    (when (and session (not (string= session "none")))
+      (setq-local python-shell-buffer-name
+                  (org-babel-python-without-earmuffs session)))))
+
 (defun org-babel-execute:python (body params)
   "Execute Python BODY according to PARAMS.
 This function is called by `org-babel-execute-src-block'."
-- 
2.25.1

Re: [PATCH] Set Python shell in Org edit buffer

[Ihor Radchenko]

Liu Hui <liuhui1610@gmail.com> writes:

>> May you also mention what this means in practice? Like that users can
>> now send region for evaluation right from the edit src buffer?
>> Otherwise, it may not be very clear for ordinary users what this feature
>> adds.
>
> Yes, I have updated the text and you're welcome to improve it. Thanks!

Thanks for the update!

> +(defun org-babel-edit-prep:python (info)
> +  "Set Python shell in Org edit buffer according to INFO."
> +  (let ((session (cdr (assq :session (nth 2 info)))))
> +    (when (and session (not (string= session "none")))
> +      (setq-local python-shell-buffer-name
> +                  (org-babel-python-without-earmuffs session)))))

Will this work if Python session does not exist yet?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Set Python shell in Org edit buffer

[Liu Hui]

Ihor Radchenko <yantar92@posteo.net> 于2023年12月7日周四 18:33写道：

> > +(defun org-babel-edit-prep:python (info)
> > +  "Set Python shell in Org edit buffer according to INFO."
> > +  (let ((session (cdr (assq :session (nth 2 info)))))
> > +    (when (and session (not (string= session "none")))
> > +      (setq-local python-shell-buffer-name
> > +                  (org-babel-python-without-earmuffs session)))))
>
> Will this work if Python session does not exist yet?

If the specified session does not exist, users have to start the
session manually if they want to evaluate code directly in the edit
buffer. In fact, python-shell-send-* commands will clearly suggest
users using 'run-python' or C-c C-p to create the session in this
case.

Another choice is to use '(org-babel-python-initiate-session session)'
in org-babel-edit-prep:python, like ob-R, to create the session when
the specified session does not exist, but I feel it is invasive as
users may just want to edit the code.

Re: [PATCH] Set Python shell in Org edit buffer

[Ihor Radchenko]

Liu Hui <liuhui1610@gmail.com> writes:

> Ihor Radchenko <yantar92@posteo.net> 于2023年12月7日周四 18:33写道：
>
>> > +(defun org-babel-edit-prep:python (info)
>> > +  "Set Python shell in Org edit buffer according to INFO."
>> > +  (let ((session (cdr (assq :session (nth 2 info)))))
>> > +    (when (and session (not (string= session "none")))
>> > +      (setq-local python-shell-buffer-name
>> > +                  (org-babel-python-without-earmuffs session)))))
>>
>> Will this work if Python session does not exist yet?
>
> If the specified session does not exist, users have to start the
> session manually if they want to evaluate code directly in the edit
> buffer. In fact, python-shell-send-* commands will clearly suggest
> users using 'run-python' or C-c C-p to create the session in this
> case.

I am afraid that manually starting a python session with `run-python'
will be misleading. Look at how much
`org-babel-python-initiate-session-by-key' does. If users instead start
the session with `run-python' they may get inconsistent results

> Another choice is to use '(org-babel-python-initiate-session session)'
> in org-babel-edit-prep:python, like ob-R, to create the session when
> the specified session does not exist, but I feel it is invasive as
> users may just want to edit the code.

What about displaying a yes/no query about starting a new session when
there is none?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Set Python shell in Org edit buffer

[Liu Hui]

Ihor Radchenko <yantar92@posteo.net> 于2023年12月7日周四 23:16写道：
>
> Liu Hui <liuhui1610@gmail.com> writes:
>
> > Ihor Radchenko <yantar92@posteo.net> 于2023年12月7日周四 18:33写道：
> >
> >> > +(defun org-babel-edit-prep:python (info)
> >> > +  "Set Python shell in Org edit buffer according to INFO."
> >> > +  (let ((session (cdr (assq :session (nth 2 info)))))
> >> > +    (when (and session (not (string= session "none")))
> >> > +      (setq-local python-shell-buffer-name
> >> > +                  (org-babel-python-without-earmuffs session)))))
> >>
> >> Will this work if Python session does not exist yet?
> >
> > If the specified session does not exist, users have to start the
> > session manually if they want to evaluate code directly in the edit
> > buffer. In fact, python-shell-send-* commands will clearly suggest
> > users using 'run-python' or C-c C-p to create the session in this
> > case.
>
> I am afraid that manually starting a python session with `run-python'
> will be misleading. Look at how much
> `org-babel-python-initiate-session-by-key' does. If users instead start
> the session with `run-python' they may get inconsistent results

ob-python has good support for using the existing session that created
by 'run-python', which contributes to a large part of
org-babel-python-initiate-session-by-key. It is common for users to
start a ob-python session with `run-python', even in the edit buffer.

But it is indeed possible that two sessions are inconsistent, if users
intend to have different org-babel-python-command and
python-shell-interpreter, which are used by
`org-babel-python-initiate-session' and `run-python', respectively.

> > Another choice is to use '(org-babel-python-initiate-session session)'
> > in org-babel-edit-prep:python, like ob-R, to create the session when
> > the specified session does not exist, but I feel it is invasive as
> > users may just want to edit the code.
>
> What about displaying a yes/no query about starting a new session when
> there is none?

I think it is OK. I can add an option to allow users to disable the
query. WDYT?

Re: [PATCH] Set Python shell in Org edit buffer

[Ihor Radchenko]

Liu Hui <liuhui1610@gmail.com> writes:

>> What about displaying a yes/no query about starting a new session when
>> there is none?
>
> I think it is OK. I can add an option to allow users to disable the
> query. WDYT?

I now have second thoughts about obsoleting
org-babel-<lang>-associate-session.
If we need a customization for the new session dialogue, I expect that
users will be willing to set it for all src blocks that support such a
feature, not just python. But then we somehow need to make this global
customization visible to babel backend authors - something we normally
do using the set of org-babel-...:<lang> functions.

What I am thinking now is
1. Introduce a global customization for users to choose whether to
   start a new session in org-src buffers with allowed values t, nil,
   'ask, or an alist of (lang . t/nil/ask) for per-language customization.
2. Check the new variable and attempt to run
   `org-babel-<lang>-associate-session' in org-src-mode definition.

WDYT?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Set Python shell in Org edit buffer

[Liu Hui]

Ihor Radchenko <yantar92@posteo.net> 于2023年12月8日周五 21:06写道：
>
> Liu Hui <liuhui1610@gmail.com> writes:
>
> >> What about displaying a yes/no query about starting a new session when
> >> there is none?
> >
> > I think it is OK. I can add an option to allow users to disable the
> > query. WDYT?
>
> I now have second thoughts about obsoleting
> org-babel-<lang>-associate-session.
> If we need a customization for the new session dialogue, I expect that
> users will be willing to set it for all src blocks that support such a
> feature, not just python. But then we somehow need to make this global
> customization visible to babel backend authors - something we normally
> do using the set of org-babel-...:<lang> functions.
>
> What I am thinking now is
> 1. Introduce a global customization for users to choose whether to
>    start a new session in org-src buffers with allowed values t, nil,
>    'ask, or an alist of (lang . t/nil/ask) for per-language customization.

I think it makes sense. Currently, org-babel-edit-prep:R/julia always
start the session when editing src blocks with a valid session header,
and other backends don't. It would allow users to change the behavior.

> 2. Check the new variable and attempt to run
>    `org-babel-<lang>-associate-session' in org-src-mode definition.

I think associating the edit buffer with some session doesn't require
starting the session, which is at least feasible for ob-python. When
editing python src block, users can use C-c C-p to start the session
themselves for evaluating code. So it would be nice to allow a value
of 'associate in the customization, which means just running
`org-babel-<lang>-associate-session'.

Re: [PATCH] Set Python shell in Org edit buffer

[Ihor Radchenko]

Liu Hui <liuhui1610@gmail.com> writes:

>> 2. Check the new variable and attempt to run
>>    `org-babel-<lang>-associate-session' in org-src-mode definition.
>
> I think associating the edit buffer with some session doesn't require
> starting the session, which is at least feasible for ob-python. When
> editing python src block, users can use C-c C-p to start the session
> themselves for evaluating code. So it would be nice to allow a value
> of 'associate in the customization, which means just running
> `org-babel-<lang>-associate-session'.

I think we have a misunderstanding here.

Didn't we just discuss that C-c C-p in python is not equivalent to
`org-babel-python-initiate-session'?

Also, by "start a new session" I meant "start a new session if there is
none running already; if a session is already running, unconditionally
associate Org Src buffer with that running session".

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Set Python shell in Org edit buffer

[Liu Hui]

Ihor Radchenko <yantar92@posteo.net> 于2023年12月9日周六 18:29写道：
>
> Liu Hui <liuhui1610@gmail.com> writes:
>
> >> 2. Check the new variable and attempt to run
> >>    `org-babel-<lang>-associate-session' in org-src-mode definition.
> >
> > I think associating the edit buffer with some session doesn't require
> > starting the session, which is at least feasible for ob-python. When
> > editing python src block, users can use C-c C-p to start the session
> > themselves for evaluating code. So it would be nice to allow a value
> > of 'associate in the customization, which means just running
> > `org-babel-<lang>-associate-session'.
>
> I think we have a misunderstanding here.
>
> Didn't we just discuss that C-c C-p in python is not equivalent to
> `org-babel-python-initiate-session'?
>
> Also, by "start a new session" I meant "start a new session if there is
> none running already; if a session is already running, unconditionally
> associate Org Src buffer with that running session".

I just want to set 'python-shell-buffer-name' in the edit buffer
according to the :session header and don't need to start the session
even if the session doesn't exist. It may be not a real association by
your definition, so my previous comment is not relevant any more.

Re: [PATCH] Set Python shell in Org edit buffer

[Jack Kamm]

Liu Hui <liuhui1610@gmail.com> writes:

> I just want to set 'python-shell-buffer-name' in the edit buffer
> according to the :session header and don't need to start the session
> even if the session doesn't exist.

Sorry that I missed this thread.

I agree that `python-shell-buffer-name' should be set according to the
:session header, and that Liu's patch fixes a problem in ob-python.

Is there any objection if I go ahead and apply it?

I think the question of what to do when a session doesn't exist is a
separate issue. Liu's patch doesn't change the status quo on this
matter (C-c C-c still tells you to call run-python either way). And
I'm not convinced there is a problem with the status quo anyways,
though additional customization could be nice.

Re: [PATCH] Set Python shell in Org edit buffer

[Jack Kamm]

Ihor Radchenko <yantar92@posteo.net> writes:

> I think we have a misunderstanding here.
>
> Didn't we just discuss that C-c C-p in python is not equivalent to
> `org-babel-python-initiate-session'?

ob-python works fine with sessions started externally by `run-python'.
And I have preserved this functionality, as I enjoy the flexibility of
working this way.

In particular, ob-python will detect when it needs to run
`org-babel-python--setup-session', even if the session was started
externally by `run-python'.

`org-babel-python-initiate-session' might be misnamed, which may be
causing some confusion here. It is run whenever a Python block is
executed, even if the session already exists.

Re: [PATCH] Set Python shell in Org edit buffer

[Ihor Radchenko]

Jack Kamm <jackkamm@gmail.com> writes:

> Liu Hui <liuhui1610@gmail.com> writes:
>
>> I just want to set 'python-shell-buffer-name' in the edit buffer
>> according to the :session header and don't need to start the session
>> even if the session doesn't exist.
>
> Sorry that I missed this thread.
>
> I agree that `python-shell-buffer-name' should be set according to the
> :session header, and that Liu's patch fixes a problem in ob-python.
>
> Is there any objection if I go ahead and apply it?

Because I am still thinking about the idea with global customization and
`org-babel-<lang>-associate-session'.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Set Python shell in Org edit buffer

[Ihor Radchenko]

Jack Kamm <jackkamm@gmail.com> writes:

> Ihor Radchenko <yantar92@posteo.net> writes:
>
>> I think we have a misunderstanding here.
>>
>> Didn't we just discuss that C-c C-p in python is not equivalent to
>> `org-babel-python-initiate-session'?
>
> ob-python works fine with sessions started externally by `run-python'.
> And I have preserved this functionality, as I enjoy the flexibility of
> working this way.
>
> In particular, ob-python will detect when it needs to run
> `org-babel-python--setup-session', even if the session was started
> externally by `run-python'.

As long as it remains undocumented, we can break this in future (maybe
years from now, but still...). What might be more robust is to provide
an explicit "start session from Org Src buffer" command for ob-python
and re-bind `run-python' to this command in Org Src buffers.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Set Python shell in Org edit buffer

[Jack Kamm]

Ihor Radchenko <yantar92@posteo.net> writes:

> As long as it remains undocumented, we can break this in future (maybe
> years from now, but still...).

Fair enough, I've had to fix this feature from time to time due to
breakage in the past.

I just pushed d0d838b02 which should hopefully prevent future breakage:

https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=d0d838b02e44a40adca14d6eae39fd4c364730da

It adds a unit test for this feature, and makes the docstrings for
org-babel-python-initiate-session(-by-key) more thorough. It also
improves robustness if the source block is executed before run-python
finishes initializing.

> What might be more robust is to provide an explicit "start session
> from Org Src buffer" command for ob-python and re-bind `run-python' to
> this command in Org Src buffers.

We could refactor `org-babel-python-initiate-session-by-key' to call a
separate `org-babel-python-run-python' interactive command that wraps
`run-python', and then rebind `run-python' to it in a local minor mode
for ob-python Src buffers, which could be started in
`org-babel-edit-prep:python'.

But I'm not sure if it's worth the hassle, or if d0d838b02 already
addresses the concern sufficiently?

Note that I'd like ob-python to keep working with run-python, even if
it's invoked outside of an Org Src buffer.

Re: [PATCH] Set Python shell in Org edit buffer

[Jack Kamm]

Liu Hui <liuhui1610@gmail.com> writes:

> But it is indeed possible that two sessions are inconsistent, if users
> intend to have different org-babel-python-command and
> python-shell-interpreter, which are used by
> `org-babel-python-initiate-session' and `run-python', respectively.

I have just proposed this patch, which will reduce this discrepancy by
having ob-python sessions use the same interpreter as `run-python' by
default:

https://list.orgmode.org/87edf41yeb.fsf@gmail.com/T/#u

Additionally, we could set `python-shell-interpreter(-args)' in
`org-babel-edit-prep:python' to make `run-python' in Org Src buffer use
the same interpreter as :python header or customized
`org-babel-python-command'. See the changes to
`org-babel-python-initiate-session-by-key' in the linked patch for an
example -- specifically the part using `split-string-and-unquote' to get
the 2 parts for `python-shell-interpreter' and
`python-shell-interpreter-args', respectively.

Re: [PATCH] Set Python shell in Org edit buffer

[Liu Hui]

Jack Kamm <jackkamm@gmail.com> 于2023年12月30日周六 06:20写道：
>
> Liu Hui <liuhui1610@gmail.com> writes:
>
> > But it is indeed possible that two sessions are inconsistent, if users
> > intend to have different org-babel-python-command and
> > python-shell-interpreter, which are used by
> > `org-babel-python-initiate-session' and `run-python', respectively.
>
> I have just proposed this patch, which will reduce this discrepancy by
> having ob-python sessions use the same interpreter as `run-python' by
> default:
>
> https://list.orgmode.org/87edf41yeb.fsf@gmail.com/T/#u

Thanks! It is a nice change IMO.

> Additionally, we could set `python-shell-interpreter(-args)' in
> `org-babel-edit-prep:python' to make `run-python' in Org Src buffer use
> the same interpreter as :python header or customized
> `org-babel-python-command'.

Agree. I think it can address the concern about inconsistent sessions.

Re: [PATCH] Set Python shell in Org edit buffer

[Ihor Radchenko]

Jack Kamm <jackkamm@gmail.com> writes:

>> What might be more robust is to provide an explicit "start session
>> from Org Src buffer" command for ob-python and re-bind `run-python' to
>> this command in Org Src buffers.
>
> We could refactor `org-babel-python-initiate-session-by-key' to call a
> separate `org-babel-python-run-python' interactive command that wraps
> `run-python', and then rebind `run-python' to it in a local minor mode
> for ob-python Src buffers, which could be started in
> `org-babel-edit-prep:python'.
>
> But I'm not sure if it's worth the hassle, or if d0d838b02 already
> addresses the concern sufficiently?

For ob-python, there is little difference. I am thinking in a more
general context of all the babel backends that support sessions.

python.el is convenient as it allows setting the session buffer name in
advance via buffer-local variable. Then, all the normal python-mode
commands, including `run-python' or `python-shell-send-region' will work
as expected. However, other babel backends like ob-R do not have such a
luxury (see the dance with renaming R process buffer in
`org-babel-R-initiate-session').

That's why I thought about remapping `run-python' and all the equivalent
commands for other backends.

However, as in the above example, `run-python' is not the only mean to
initiate session in various major modes - sending a line or region at
point is another common command; and they might be others. Not to
mention that re-binding is not panacea - people may still run the
original commands via M-x.

So, re-binding is probably not the best idea.
In other words, `org-babel-python-run-python' is not necessary.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Set Python shell in Org edit buffer

[Jack Kamm]

Ihor Radchenko <yantar92@posteo.net> writes:

> python.el is convenient as it allows setting the session buffer name in
> advance via buffer-local variable. Then, all the normal python-mode
> commands, including `run-python' or `python-shell-send-region' will work
> as expected. However, other babel backends like ob-R do not have such a
> luxury (see the dance with renaming R process buffer in
> `org-babel-R-initiate-session').

For R specifically, using `ess-gen-proc-buffer-name-function' might
simplify things. Though of course this doesn't solve the problem more
generally for other languages.

Re: [PATCH] Set Python shell in Org edit buffer

[Jack Kamm]

Ihor Radchenko <yantar92@posteo.net> writes:

>> I agree that `python-shell-buffer-name' should be set according to the
>> :session header, and that Liu's patch fixes a problem in ob-python.
>>
>> Is there any objection if I go ahead and apply it?
>
> Because I am still thinking about the idea with global customization and
> `org-babel-<lang>-associate-session'.

It's great that you're thinking about this -- it would be nice to have
better consistency between ob-R, ob-python, etc, and to have better
configurability on this. In particular, ob-R's behavior to automatically
start a session is annoying for me, especially when editing on my HPC's
login node which forbids starting R, python, etc outside of slurm.

However, the most recent version of Liu's patch is very small, and the
changes should be easy to modify in future, whatever your conclusion on
`org-babel-<lang>-associate-session'. So I would suggest not to let it
be blocked by this for too long.

Re: [PATCH] Set Python shell in Org edit buffer

[Ihor Radchenko]

Jack Kamm <jackkamm@gmail.com> writes:

>> Because I am still thinking about the idea with global customization and
>> `org-babel-<lang>-associate-session'.
>
> It's great that you're thinking about this -- it would be nice to have
> better consistency between ob-R, ob-python, etc, and to have better
> configurability on this. In particular, ob-R's behavior to automatically
> start a session is annoying for me, especially when editing on my HPC's
> login node which forbids starting R, python, etc outside of slurm.
>
> However, the most recent version of Liu's patch is very small, and the
> changes should be easy to modify in future, whatever your conclusion on
> `org-babel-<lang>-associate-session'. So I would suggest not to let it
> be blocked by this for too long.

I just do not want to do double work.

I now reverted the obsoletion of org-babel-<lang>-associate-session.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=99c9cae25
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=b4687fcd6

I also fixed the problem where org-babel-<lang>-associate-session was
never actually executed due to `org-src--babel-info' being set after
`org-src-mode-hook'.

Now, the question is what to do with the existing implementation of
`org-src-associate-babel-session'. It only runs
org-babel-<lang>-associate-session when

(and session (not (string= session "none"))
	 (org-babel-comint-buffer-livep session)
	 (let ((f (intern (format "org-babel-%s-associate-session"
                                  (nth 0 info)))))
           (and (fboundp f) (funcall f session))))

The questionable check here is (org-babel-comint-buffer-livep session) -
it only triggers when session is already initiated, while ob-python and
some other backends do not necessarily need to start a new session to
"associate" it with Org Src buffer.

I am tentatively inclined to change this check to

(or (org-babel-comint-buffer-livep session)
    (eq org-src-auto-initiate-session t)
    (alist-get (nth 0 info) org-src-auto-initiate-session)
    (alist-get 'default org-src-auto-initiate-session))

With `org-src-auto-initiate-session' being a customization that controls
whether to associate session for a given babel backend.

We may set the default value to something like

((default . t) ("R" . nil))

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Set Python shell in Org edit buffer

[Ihor Radchenko]

Jack Kamm <jackkamm@gmail.com> writes:

>> python.el is convenient as it allows setting the session buffer name in
>> advance via buffer-local variable. Then, all the normal python-mode
>> commands, including `run-python' or `python-shell-send-region' will work
>> as expected. However, other babel backends like ob-R do not have such a
>> luxury (see the dance with renaming R process buffer in
>> `org-babel-R-initiate-session').
>
> For R specifically, using `ess-gen-proc-buffer-name-function' might
> simplify things. Though of course this doesn't solve the problem more
> generally for other languages.

For now, I changed ob-R and ob-julia to use
`ess-gen-proc-buffer-name-function'. This is indeed cleaner.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=b6643884c

With this, I am wondering what would be more appropriate place to set
`ess-gen-proc-buffer-name-function' - `org-babel-edit-prep:R' or
`org-babel-R-associate-session'. Same for python and other backends.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Set Python shell in Org edit buffer

[Christopher M. Miles]

[-- Attachment #1: Type: text/plain, Size: 1707 bytes --]


Ihor Radchenko <yantar92@posteo.net> writes:

> Now, the question is what to do with the existing implementation of
> `org-src-associate-babel-session'. It only runs
> org-babel-<lang>-associate-session when
>
> (and session (not (string= session "none"))
> 	 (org-babel-comint-buffer-livep session)
> 	 (let ((f (intern (format "org-babel-%s-associate-session"
>                                   (nth 0 info)))))
>            (and (fboundp f) (funcall f session))))
>
> The questionable check here is (org-babel-comint-buffer-livep session) -
> it only triggers when session is already initiated, while ob-python and
> some other backends do not necessarily need to start a new session to
> "associate" it with Org Src buffer.
>
> I am tentatively inclined to change this check to
>
> (or (org-babel-comint-buffer-livep session)
>     (eq org-src-auto-initiate-session t)
>     (alist-get (nth 0 info) org-src-auto-initiate-session)
>     (alist-get 'default org-src-auto-initiate-session))
>
> With `org-src-auto-initiate-session' being a customization that controls
> whether to associate session for a given babel backend.
>
> We may set the default value to something like
>
> ((default . t) ("R" . nil))

I think this customization is reasonable. Agree to make it as a
customization option, The ob-clojure CIDER session which managed by
"sesman" seems also requires buffer associate with session.

-- 

[ stardiviner ]
I try to make every word tell the meaning that I want to express without misunderstanding.

Blog: https://stardiviner.github.io/
IRC(libera.chat, freenode): stardiviner, Matrix: stardiviner
GPG: F09F650D7D674819892591401B5DF1C95AE89AC3

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: [PATCH] Set Python shell in Org edit buffer

[Jack Kamm]

Ihor Radchenko <yantar92@posteo.net> writes:

> Now, the question is what to do with the existing implementation of
> `org-src-associate-babel-session'. It only runs
> org-babel-<lang>-associate-session when
>
> (and session (not (string= session "none"))
> 	 (org-babel-comint-buffer-livep session)
> 	 (let ((f (intern (format "org-babel-%s-associate-session"
>                                   (nth 0 info)))))
>            (and (fboundp f) (funcall f session))))
>
> The questionable check here is (org-babel-comint-buffer-livep session) -
> it only triggers when session is already initiated, while ob-python and
> some other backends do not necessarily need to start a new session to
> "associate" it with Org Src buffer.
>
> I am tentatively inclined to change this check to
>
> (or (org-babel-comint-buffer-livep session)
>     (eq org-src-auto-initiate-session t)
>     (alist-get (nth 0 info) org-src-auto-initiate-session)
>     (alist-get 'default org-src-auto-initiate-session))
>
> With `org-src-auto-initiate-session' being a customization that controls
> whether to associate session for a given babel backend.
>
> We may set the default value to something like
>
> ((default . t) ("R" . nil))

I think there are 2 aspects of the session+editing behavior that I'd
like to disentangle:

1. Whether to create session when editing (if it doesn't exist yet)
2. If session exists, whether to "associate" it so that evaluation
   commands (e.g. python-shell-send-region, ess-eval-region) and
   autocompletion use it.

Personally, I prefer to disable #1 while enabling #2. For ob-R, it
seems like #1 happens in `org-babel-edit-prep:R' while #2 happens in
`org-babel-R-associate-session', so adding an option to disable the
latter isn't useful for me, and it's not clear to me whether it'd be
useful generally for others either.

(I realize #2 hasn't worked properly for some time now until you fixed
it in this thread. I wasn't too badly affected because I usually only
use one session at a time, and ess-eval-region is able to figure out
the session in this case. But I did sometimes have to manually call
`ess-force-buffer-current' to get completion working, which it seems I
no longer have to do now that you've fixed this).

As an aside, I just noticed some inconsistent behavior in ob-R. It seems
it only auto-creates the session when ":session *foo*" (i.e. the session
name has "earmuffs"). But when ":session foo" (no earmuffs), ob-R
doesn't autostart the session. While this is probably accidental, it
doesn't seem to cause any problems, which makes me question whether it's
really needful for ob-R to initiate a session on edit.  In particular,
if ":session foo" already exists, then ess-eval-region still works fine
in the src block. Which is exactly my desired behavior -- don't create
the session if it doesn't exist yet, but if it already exists then play
nicely with it.

Another thing to note is that ob-R works fine with sessions externally
created with "M-x R".  (similar to how ob-python works fine with "M-x
run-python"). In fact, I sometimes use ob-R with manual "M-x R" sessions
when I need to use a different R binary/environment than my usual
one. So, it is not necessary for the R session to be started through ob-R.

As for ob-python; I think it's best to set `python-shell-buffer-name'
in `org-babel-edit-prep:python' rather than
`org-babel-python-associate-session'. While both work for
`python-shell-send-region' if the session already exists,
`org-babel-edit-prep:python' has the advantage that it will run even
if the session doesn't exist yet, so then "M-x run-python" in the src
block will create a session with the correct name.

Re: [PATCH] Set Python shell in Org edit buffer

[Ihor Radchenko]

Jack Kamm <jackkamm@gmail.com> writes:

>> Now, the question is what to do with the existing implementation of
>> `org-src-associate-babel-session'. It only runs
>> org-babel-<lang>-associate-session when
>>
>> (and session (not (string= session "none"))
>> 	 (org-babel-comint-buffer-livep session)
>> 	 (let ((f (intern (format "org-babel-%s-associate-session"
>>                                   (nth 0 info)))))
>>            (and (fboundp f) (funcall f session))))
>> ...
>> I am tentatively inclined to change this check to
>>
>> (or (org-babel-comint-buffer-livep session)
>>     (eq org-src-auto-initiate-session t)
>>     (alist-get (nth 0 info) org-src-auto-initiate-session)
>>     (alist-get 'default org-src-auto-initiate-session))
>
> I think there are 2 aspects of the session+editing behavior that I'd
> like to disentangle:
>
> 1. Whether to create session when editing (if it doesn't exist yet)
> 2. If session exists, whether to "associate" it so that evaluation
>    commands (e.g. python-shell-send-region, ess-eval-region) and
>    autocompletion use it.
>
> Personally, I prefer to disable #1 while enabling #2. For ob-R, it
> seems like #1 happens in `org-babel-edit-prep:R' while #2 happens in
> `org-babel-R-associate-session', so adding an option to disable the
> latter isn't useful for me, and it's not clear to me whether it'd be
> useful generally for others either.
> ...
> As an aside, I just noticed some inconsistent behavior in ob-R. It seems
> it only auto-creates the session when ":session *foo*" (i.e. the session
> name has "earmuffs"). But when ":session foo" (no earmuffs), ob-R
> doesn't autostart the session. While this is probably accidental, it
> doesn't seem to cause any problems, which makes me question whether it's
> really needful for ob-R to initiate a session on edit.  In particular,
> if ":session foo" already exists, then ess-eval-region still works fine
> in the src block. Which is exactly my desired behavior -- don't create
> the session if it doesn't exist yet, but if it already exists then play
> nicely with it.

I imagine that both #1 and #2 should happen in
org-babel-<lang>-associate-session. #1 should probably be discouraged,
and it looks like even for ob-R creating new session is not really
necessary.

So, a good option could be
(1) removing (org-babel-comint-buffer-livep session) from
    `org-src-associate-babel-session'
(2) Removing `org-babel-edit-prep:R'

Then, the existence of org-babel-<lang>-associate-session will simply be
a reminder for babel backends to implement session support inside babel
edit buffers.

> As for ob-python; I think it's best to set `python-shell-buffer-name'
> in `org-babel-edit-prep:python' rather than
> `org-babel-python-associate-session'. While both work for
> `python-shell-send-region' if the session already exists,
> `org-babel-edit-prep:python' has the advantage that it will run even
> if the session doesn't exist yet, so then "M-x run-python" in the src
> block will create a session with the correct name.

With the above, we can use `org-babel-python-associate-session'. Just
for consistency - org-babel-<lang>-associate-session is somewhat
redundant as org-babel-edit-prep:<lang> can be used every time the
format is used, except that org-babel-<lang>-associate-session function
does not need to check if :session argument is present.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Set Python shell in Org edit buffer

[Jack Kamm]

Ihor Radchenko <yantar92@posteo.net> writes:

> So, a good option could be
> (1) removing (org-babel-comint-buffer-livep session) from
>     `org-src-associate-babel-session'
> (2) Removing `org-babel-edit-prep:R'
>
> With the above, we can use `org-babel-python-associate-session'

Sounds good to me.

> I imagine that both #1 and #2 should happen in
> org-babel-<lang>-associate-session. #1 should probably be discouraged,
> and it looks like even for ob-R creating new session is not really
> necessary.

It looks like ob-R and ob-julia are the only languages that start
sessions on edit (based on grepping for "edit-prep" and
"associate-session").

I think their behavior is peculiar enough to have an ob-R/julia-specific
option on whether to initiate session on edit, with options nil, t, and
earmuffs. Earmuffs is the current behavior, but it's surprising enough
(IMO) that it might be worth changing the default to nil or t. But still
worth keeping the earmuffs option since this behavior seems to go back
to the original implementation (30931bfe1). If it helps, I can prepare a
patch for this after you've made the changes for
org-babel-<lang>-associate-session.

In my notebooks I generally define my ob-R sessions to have earmuffs
(like ":session *R:project-name*") so that they can easily work with
"M-x R" (which names sessions as such by default). Until now I did not
realize this was the culprit for the annoying (and undocumented) startup
behavior I was experiencing.

Re: [PATCH] Set Python shell in Org edit buffer

[William Denton]

On Sunday, January 7th, 2024 at 14:06, Jack Kamm <jackkamm@gmail.com> wrote:

> It looks like ob-R and ob-julia are the only languages that start
> sessions on edit (based on grepping for "edit-prep" and
> "associate-session").
> 
> I think their behavior is peculiar enough to have an ob-R/julia-specific
> option on whether to initiate session on edit, with options nil, t, and
> earmuffs. Earmuffs is the current behavior, but it's surprising enough
> (IMO) that it might be worth changing the default to nil or t. But still
> worth keeping the earmuffs option since this behavior seems to go back
> to the original implementation (30931bfe1).

I'm a regular R user and think this is a great idea.  A while back I had a bunch of problems with Org unexpectedly opening R buffers unexpectedly and I had no idea why it was happening.  (In fact I never really did until reading this.)  I fixed it by simplifying my R setup and removing some hooks and such, and now things work reasonably and I'm familiar with what happens when.  Having an option to control this would be a helpful addition.


Bill

--
William Denton
https://www.miskatonic.org/
Librarian, artist and licensed private investigator.
Toronto, Canada

Re: [PATCH] Set Python shell in Org edit buffer

[Ihor Radchenko]

Jack Kamm <jackkamm@gmail.com> writes:

> Ihor Radchenko <yantar92@posteo.net> writes:
>
>> So, a good option could be
>> (1) removing (org-babel-comint-buffer-livep session) from
>>     `org-src-associate-babel-session'
>> (2) Removing `org-babel-edit-prep:R'
>>
>> With the above, we can use `org-babel-python-associate-session'
>
> Sounds good to me.

Note that I proposed to remove auto-starting session completely, which
is in odds to what you propose below.

> I think their behavior is peculiar enough to have an ob-R/julia-specific
> option on whether to initiate session on edit, with options nil, t, and
> earmuffs. Earmuffs is the current behavior, but it's surprising enough
> (IMO) that it might be worth changing the default to nil or t. But still
> worth keeping the earmuffs option since this behavior seems to go back
> to the original implementation (30931bfe1). If it helps, I can prepare a
> patch for this after you've made the changes for
> org-babel-<lang>-associate-session.

IMHO, it might be enough to adjust org-babel-R-associate-session as the
following

(defun org-babel-R-associate-session (session)
  "Associate R code buffer with an R session.
Make SESSION be the inferior ESS process associated with the
current code buffer."
  (setq ess-local-process-name
	(process-name (get-buffer-process session)))
  (when ess-local-process-name (ess-make-buffer-current))
  (setq-local ess-gen-proc-buffer-name-function (lambda (_) session)))

From the user point of view, it should then make no difference.
If we follow the advice from
https://orgmode.org/worg/org-contrib/babel/languages/ob-doc-R.html,

    Use ESS to step through evaluation line-by-line
      - Use C-c ' to visit the edit buffer for your code block
      - Use ess-eval-line-and-step to evaluate each line in turn

ess-eval-line-and-step should still work regardless whether ESS session
is already initiated or not before opening Org Src buffer.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Set Python shell in Org edit buffer

[Jack Kamm]

Ihor Radchenko <yantar92@posteo.net> writes:

> Note that I proposed to remove auto-starting session completely, which
> is in odds to what you propose below.

Sure, I'm fine with that -- it seems like a reasonable change.

> IMHO, it might be enough to adjust org-babel-R-associate-session as the
> following
>
> (defun org-babel-R-associate-session (session)
>   "Associate R code buffer with an R session.
> Make SESSION be the inferior ESS process associated with the
> current code buffer."
>   (setq ess-local-process-name
> 	(process-name (get-buffer-process session)))
>   (when ess-local-process-name (ess-make-buffer-current))
>   (setq-local ess-gen-proc-buffer-name-function (lambda (_) session)))

I think you need to check that (get-buffer-process session) is non-nil
before calling process-name, otherwise you'll get (wrong-type-argument
processp nil).

Also it seems unnecessary to call `ess-make-buffer-current', as it's
already called by `ess-force-buffer-current' (which is called by
`ess-eval-region'). Though it doesn't hurt to call it, either.

Otherwise, this looks good to me.

Re: [PATCH] Set Python shell in Org edit buffer

[Jack Kamm]

Jack Kamm <jackkamm@gmail.com> writes:

> Also it seems unnecessary to call `ess-make-buffer-current', as it's
> already called by `ess-force-buffer-current' (which is called by
> `ess-eval-region'). Though it doesn't hurt to call it, either.

On reflection, maybe it's better to keep
`ess-make-buffer-current'. Maybe it's useful for other functionality
besides evaluation (e.g. completion), or on older versions of
ESS. And even if it isn't necessary anymore, it doesn't hurt either.

Re: [PATCH] Set Python shell in Org edit buffer

[Ihor Radchenko]

[-- Attachment #1: Type: text/plain, Size: 204 bytes --]

Jack Kamm <jackkamm@gmail.com> writes:

>> IMHO, it might be enough to adjust org-babel-R-associate-session as the
>> following
>...
> Otherwise, this looks good to me.

See the attached tentative patch.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ob-R-ob-julia-Do-not-force-start-session-in-Org-Src-.patch --]
[-- Type: text/x-patch, Size: 5451 bytes --]

From f6fd65f8e80b6efa0a5db084a9c9f94d46e67515 Mon Sep 17 00:00:00 2001
Message-ID: <f6fd65f8e80b6efa0a5db084a9c9f94d46e67515.1704824159.git.yantar92@posteo.net>
From: Ihor Radchenko <yantar92@posteo.net>
Date: Tue, 9 Jan 2024 19:08:44 +0100
Subject: [PATCH] ob-R, ob-julia: Do not force-start session in Org Src buffers

* lisp/ob-R.el (org-babel-R-associate-session): Set
`ess-gen-proc-buffer-name-function' to associate the right session
buffer if user requests session interaction from inside Org Src edit
buffer.
(org-babel-edit-prep:R): Remove in favor of
`org-babel-R-associate-session'.
* lisp/ob-julia.el (org-babel-julia-associate-session): Implement
analogous functionality.
(org-babel-edit-prep:julia): Remove in favor of
`org-babel-julia-associate-session'.
* etc/ORG-NEWS (~org-edit-special~ no longer force-starts session in R
and Julia source blocks): Document the change.

Link: https://orgmode.org/list/87r0ir2ln8.fsf@gmail.com
---
 etc/ORG-NEWS     | 11 +++++++++++
 lisp/ob-R.el     | 20 ++++++--------------
 lisp/ob-julia.el | 16 +++++++++-------
 3 files changed, 26 insertions(+), 21 deletions(-)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index b808357d8..245e595ff 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -13,6 +13,17 @@ Please send Org bug reports to mailto:emacs-orgmode@gnu.org.
 
 * Version 9.7 (not released yet)
 ** Important announcements and breaking changes
+*** ~org-edit-special~ no longer force-starts session in R and Julia source blocks
+
+Previously, when R/Julia source block had =:session= header argument
+set to a session name with "earmuffs" (like =*session-name*=),
+~org-edit-special~ always started a session, if it does not exist.
+
+Now, ~org-edit-special~ arranges that a new session with correct name
+is initiated only when user explicitly executes R/Julia-mode commands
+that trigger session interactions.  The same session will remain
+available in the context of Org babel.
+
 *** ~org-agenda-search-headline-for-time~ now ignores all the timestamp in headings
 
 Previously, ~org-agenda-search-headline-for-time~ made Org agenda
diff --git a/lisp/ob-R.el b/lisp/ob-R.el
index 3d13c55a7..4b5cbf7c9 100644
--- a/lisp/ob-R.el
+++ b/lisp/ob-R.el
@@ -91,17 +91,6 @@ (defcustom org-babel-R-command "R --slave --no-save"
   :version "24.1"
   :type 'string)
 
-(defvar ess-current-process-name) ; dynamically scoped
-(defvar ess-local-process-name)   ; dynamically scoped
-(defun org-babel-edit-prep:R (info)
-  "Initiate R session for Org edit buffer, as needed.
-The session name is taken from INFO."
-  (let ((session (cdr (assq :session (nth 2 info)))))
-    (when (and session
-	       (string-prefix-p "*"  session)
-	       (string-suffix-p "*" session))
-      (org-babel-R-initiate-session session nil))))
-
 ;; The usage of utils::read.table() ensures that the command
 ;; read.table() can be found even in circumstances when the utils
 ;; package is not in the search path from R.
@@ -292,13 +281,16 @@ (defun org-babel-R-initiate-session (session params)
 	      (ess-wait-for-process R-proc)))
 	  (current-buffer))))))
 
+(defvar ess-current-process-name) ; dynamically scoped
+(defvar ess-local-process-name)   ; dynamically scoped
 (defun org-babel-R-associate-session (session)
   "Associate R code buffer with an R session.
 Make SESSION be the inferior ESS process associated with the
 current code buffer."
-  (setq ess-local-process-name
-	(process-name (get-buffer-process session)))
-  (ess-make-buffer-current))
+  (when-let ((process (get-buffer-process session)))
+    (setq ess-local-process-name (process-name process))
+    (ess-make-buffer-current))
+  (setq-local ess-gen-proc-buffer-name-function (lambda (_) session)))
 
 (defvar org-babel-R-graphics-devices
   '((:bmp "bmp" "filename")
diff --git a/lisp/ob-julia.el b/lisp/ob-julia.el
index cddd25e79..10a331e54 100644
--- a/lisp/ob-julia.el
+++ b/lisp/ob-julia.el
@@ -70,12 +70,15 @@ (defvar ess-current-process-name) ; dynamically scoped
 (defvar ess-local-process-name)   ; dynamically scoped
 (defvar ess-eval-visibly-p)       ; dynamically scoped
 (defvar ess-local-customize-alist); dynamically scoped
-(defun org-babel-edit-prep:julia (info)
-  (let ((session (cdr (assq :session (nth 2 info)))))
-    (when (and session
-	       (string-prefix-p "*"  session)
-	       (string-suffix-p "*" session))
-      (org-babel-julia-initiate-session session nil))))
+(defvar ess-gen-proc-buffer-name-function) ; defined in ess-inf.el
+(defun org-babel-julia-associate-session (session)
+  "Associate R code buffer with an R session.
+Make SESSION be the inferior ESS process associated with the
+current code buffer."
+  (when-let ((process (get-buffer-process session)))
+    (setq ess-local-process-name (process-name process))
+    (ess-make-buffer-current))
+  (setq-local ess-gen-proc-buffer-name-function (lambda (_) session)))
 
 (defun org-babel-expand-body:julia (body params &optional _graphics-file)
   "Expand BODY according to PARAMS, return the expanded body."
@@ -178,7 +181,6 @@ (defun org-babel-julia-assign-elisp (name value)
     (format "%s = %s" name (org-babel-julia-quote-csv-field value))))
 
 (defvar ess-ask-for-ess-directory) ; dynamically scoped
-(defvar ess-gen-proc-buffer-name-function) ; defined in ess-inf.el
 (defun org-babel-julia-initiate-session (session params)
   "If there is not a current julia process then create one."
   (unless (string= session "none")
-- 
2.43.0


[-- Attachment #3: Type: text/plain, Size: 225 bytes --]



-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Set Python shell in Org edit buffer

[Jack Kamm]

Ihor Radchenko <yantar92@posteo.net> writes:

> See the attached tentative patch.

Thanks!

> ---
>  etc/ORG-NEWS     | 11 +++++++++++
>  lisp/ob-R.el     | 20 ++++++--------------
>  lisp/ob-julia.el | 16 +++++++++-------
>  3 files changed, 26 insertions(+), 21 deletions(-)

Not sure if you are doing this in a separate commit, but you also need
to make the change to org-src.el to remove

(org-babel-comint-buffer-livep session)

in `org-src-associate-babel-session'. Else,
`org-babel-R-associate-session' won't run if there's no live session,
and hence ESS won't create new session with the right name.

>  * Version 9.7 (not released yet)
>  ** Important announcements and breaking changes
> +*** ~org-edit-special~ no longer force-starts session in R and Julia source blocks
> +
> +Previously, when R/Julia source block had =:session= header argument
> +set to a session name with "earmuffs" (like =*session-name*=),
> +~org-edit-special~ always started a session, if it does not exist.
> +
> +Now, ~org-edit-special~ arranges that a new session with correct name
> +is initiated only when user explicitly executes R/Julia-mode commands
> +that trigger session interactions.  The same session will remain
> +available in the context of Org babel.

I tested the patch (plus the additional change to org-src.el), with an
Org file with following 2 blocks:

#+begin_src R :session foo :results output
  print('foo')
#+end_src

#+begin_src R :session *bar* :results output
  print('bar')
#+end_src

On block "foo", I did C-', and then ess-eval-line. It creates a session
named "foo", as expected.

On block "*bar*", I did the same. It does not create session named
"*bar*", instead evaluating in session "foo". It seems ESS will always
assume you want to evaluate in existing session if one exists, rather
than start a new associated session, and it seems there is no way to
tell it to behave otherwise.

However, calling "M-x R" while editing block "*bar*" does create session
"*bar*" with correct name.

After sessions "foo" and "*bar*" have both been created, doing C-' and
then ess-eval-line will evaluate in the correct session.

It's annoying there's no way to tell ESS to start new session instead of
evaluating in existing one. Here are a few alternatives we could
consider to deal with this:

1. Change the worg/ORG-NEWS, to suggest users make sure the session
exists (either by evaluating a source block or call "M-x R" in edit
block) before running ess-eval-line.

2. Add ob-R and ob-julia customization options (as previously suggested)
to explicitly control the startup behavior (either to auto-start, or not).

3. Submit PR to ESS to add a variable we could let-bind, to force it to
start an associated session rather than evaluate in an existing
non-associated sessions.

Currently I lean towards a combination of #1 and #3, but am not sure,
and happy to go with whatever you think is best.

[FR] Add buffer-local setting to request specific ESS process/session name (was: [PATCH] Set Python shell in Org edit buffer)

[Ihor Radchenko]

Hi,

I'd like to request a new ESS feature that will allow to choose which
session is created by ESS when no session is yet associated with a
buffer.

Currently, `ess-request-a-process' unconditionally re-uses an existing
ESS process with appropriate `ess-dialect', even when such process is
not consistent with `ess-gen-proc-buffer-name-function'.

This behavior puts Org mode's ESS support in somewhat difficult position
- Org mode allows multiple sessions in Org src blocks, and we want some
way to tell ESS which session process to use for any given buffer
without actually starting that process manually.
We had a hope that setting `ess-gen-proc-buffer-name-function' would
suffice and that ESS would start a new process according to
`ess-gen-proc-buffer-name-function' when no process is yet associated
with buffer. But it is not the case.

Some more context:
(full thread is in
<https://list.orgmode.org/CAOQTW-O+qs7xAeP7Bemu4ThM4-oGJYxxD+K2jAAw-w7RHtX2gQ@mail.gmail.com/T/#maa33e7c3f72b5c81deb91e1a1d27d57725097fa4>)

Jack Kamm <jackkamm@gmail.com> writes:

> I tested the patch (plus the additional change to org-src.el), with an
> Org file with following 2 blocks:
>
> #+begin_src R :session foo :results output
>   print('foo')
> #+end_src
>
> #+begin_src R :session *bar* :results output
>   print('bar')
> #+end_src

These are two R blocks that should be associated with two different ESS
R processes.

> On block "foo", I did C-', and then ess-eval-line. It creates a session
> named "foo", as expected.

When we edit the first block and when no ESS process is available,
`ess-eval-line' respects `ess-gen-proc-buffer-name-function' set by Org
mode, and creates a new ESS process "foo".

> On block "*bar*", I did the same. It does not create session named
> "*bar*", instead evaluating in session "foo". It seems ESS will always
> assume you want to evaluate in existing session if one exists, rather
> than start a new associated session, and it seems there is no way to
> tell it to behave otherwise.

But when the "foo" process is already running, despite different
`ess-gen-proc-buffer-function', `ess-eval-line' connects to the existing
"foo" process rather than "*bar*", as we anticipated.

> However, calling "M-x R" while editing block "*bar*" does create session
> "*bar*" with correct name.
>
> After sessions "foo" and "*bar*" have both been created, doing C-' and
> then ess-eval-line will evaluate in the correct session.

The only workaround, which is not ideal, is to start ESS process
unconditionally. We'd like this to change.

> It's annoying there's no way to tell ESS to start new session instead of
> evaluating in existing one. Here are a few alternatives we could
> consider to deal with this:
>
> 1. Change the worg/ORG-NEWS, to suggest users make sure the session
> exists (either by evaluating a source block or call "M-x R" in edit
> block) before running ess-eval-line.
>
> 2. Add ob-R and ob-julia customization options (as previously suggested)
> to explicitly control the startup behavior (either to auto-start, or not).
>
> 3. Submit PR to ESS to add a variable we could let-bind, to force it to
> start an associated session rather than evaluate in an existing
> non-associated sessions.
>
> Currently I lean towards a combination of #1 and #3, but am not sure,
> and happy to go with whatever you think is best.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Set Python shell in Org edit buffer

[Ihor Radchenko]

Jack Kamm <jackkamm@gmail.com> writes:

>> ---
>>  etc/ORG-NEWS     | 11 +++++++++++
>>  lisp/ob-R.el     | 20 ++++++--------------
>>  lisp/ob-julia.el | 16 +++++++++-------
>>  3 files changed, 26 insertions(+), 21 deletions(-)
>
> Not sure if you are doing this in a separate commit, but you also need
> to make the change to org-src.el to remove
>
> (org-babel-comint-buffer-livep session)

My oversight. Thanks for the heads up.

> It's annoying there's no way to tell ESS to start new session instead of
> evaluating in existing one. Here are a few alternatives we could
> consider to deal with this:
>
> 1. Change the worg/ORG-NEWS, to suggest users make sure the session
> exists (either by evaluating a source block or call "M-x R" in edit
> block) before running ess-eval-line.
>
> 2. Add ob-R and ob-julia customization options (as previously suggested)
> to explicitly control the startup behavior (either to auto-start, or not).
>
> 3. Submit PR to ESS to add a variable we could let-bind, to force it to
> start an associated session rather than evaluate in an existing
> non-associated sessions.
>
> Currently I lean towards a combination of #1 and #3, but am not sure,
> and happy to go with whatever you think is best.

We can also advice `ess-request-a-process' as a temporary workaround.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [FR] Add buffer-local setting to request specific ESS process/session name (was: [PATCH] Set Python shell in Org edit buffer)

[Sparapani, Rodney]

[-- Attachment #1: Type: text/plain, Size: 5574 bytes --]

Hi Jack:

Do you have a patch?  I’m not an org-mode user so I can’t test this myself.
Thanks

--
Rodney Sparapani, Associate Professor of Biostatistics, He/Him/His
Vice President, Wisconsin Chapter of the American Statistical Association
Institute for Health and Equity, Division of Biostatistics
Medical College of Wisconsin, Milwaukee Campus

From: ESS-core <ess-core-bounces@r-project.org> on behalf of Ihor Radchenko <yantar92@posteo.net>
Date: Wednesday, January 10, 2024 at 6:15 AM
To: Jack Kamm <jackkamm@gmail.com>, ESS-core@r-project.org <ESS-core@r-project.org>
Cc: Liu Hui <liuhui1610@gmail.com>, emacs-orgmode@gnu.org <emacs-orgmode@gnu.org>
Subject: [FR] Add buffer-local setting to request specific ESS process/session name (was: [PATCH] Set Python shell in Org edit buffer)
ATTENTION: This email originated from a sender outside of MCW. Use caution when clicking on links or opening attachments.
________________________________

Hi,

I'd like to request a new ESS feature that will allow to choose which
session is created by ESS when no session is yet associated with a
buffer.

Currently, `ess-request-a-process' unconditionally re-uses an existing
ESS process with appropriate `ess-dialect', even when such process is
not consistent with `ess-gen-proc-buffer-name-function'.

This behavior puts Org mode's ESS support in somewhat difficult position
- Org mode allows multiple sessions in Org src blocks, and we want some
way to tell ESS which session process to use for any given buffer
without actually starting that process manually.
We had a hope that setting `ess-gen-proc-buffer-name-function' would
suffice and that ESS would start a new process according to
`ess-gen-proc-buffer-name-function' when no process is yet associated
with buffer. But it is not the case.

Some more context:
(full thread is in
<https://urldefense.com/v3/__https://list.orgmode.org/CAOQTW-O*qs7xAeP7Bemu4ThM4-oGJYxxD*K2jAAw-w7RHtX2gQ@mail.gmail.com/T/*maa33e7c3f72b5c81deb91e1a1d27d57725097fa4__;Kysj!!H8mHWRdzp34!4Q2jIK3jwVcEOegQ2fIC5apK11fdrSkufT30OuWfzKJeRQ_nskag8V97kuVRUbqj41mRZtQD1vyJBoGvlQ$ >)

Jack Kamm <jackkamm@gmail.com> writes:

> I tested the patch (plus the additional change to org-src.el), with an
> Org file with following 2 blocks:
>
> #+begin_src R :session foo :results output
>   print('foo')
> #+end_src
>
> #+begin_src R :session *bar* :results output
>   print('bar')
> #+end_src

These are two R blocks that should be associated with two different ESS
R processes.

> On block "foo", I did C-', and then ess-eval-line. It creates a session
> named "foo", as expected.

When we edit the first block and when no ESS process is available,
`ess-eval-line' respects `ess-gen-proc-buffer-name-function' set by Org
mode, and creates a new ESS process "foo".

> On block "*bar*", I did the same. It does not create session named
> "*bar*", instead evaluating in session "foo". It seems ESS will always
> assume you want to evaluate in existing session if one exists, rather
> than start a new associated session, and it seems there is no way to
> tell it to behave otherwise.

But when the "foo" process is already running, despite different
`ess-gen-proc-buffer-function', `ess-eval-line' connects to the existing
"foo" process rather than "*bar*", as we anticipated.

> However, calling "M-x R" while editing block "*bar*" does create session
> "*bar*" with correct name.
>
> After sessions "foo" and "*bar*" have both been created, doing C-' and
> then ess-eval-line will evaluate in the correct session.

The only workaround, which is not ideal, is to start ESS process
unconditionally. We'd like this to change.

> It's annoying there's no way to tell ESS to start new session instead of
> evaluating in existing one. Here are a few alternatives we could
> consider to deal with this:
>
> 1. Change the worg/ORG-NEWS, to suggest users make sure the session
> exists (either by evaluating a source block or call "M-x R" in edit
> block) before running ess-eval-line.
>
> 2. Add ob-R and ob-julia customization options (as previously suggested)
> to explicitly control the startup behavior (either to auto-start, or not).
>
> 3. Submit PR to ESS to add a variable we could let-bind, to force it to
> start an associated session rather than evaluate in an existing
> non-associated sessions.
>
> Currently I lean towards a combination of #1 and #3, but am not sure,
> and happy to go with whatever you think is best.

--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://urldefense.com/v3/__https://orgmode.org/__;!!H8mHWRdzp34!4Q2jIK3jwVcEOegQ2fIC5apK11fdrSkufT30OuWfzKJeRQ_nskag8V97kuVRUbqj41mRZtQD1vxMkIQ3lQ$ >.
Support Org development at <https://urldefense.com/v3/__https://liberapay.com/org-mode__;!!H8mHWRdzp34!4Q2jIK3jwVcEOegQ2fIC5apK11fdrSkufT30OuWfzKJeRQ_nskag8V97kuVRUbqj41mRZtQD1vzCeD9KoQ$ >,
or support my work at <https://urldefense.com/v3/__https://liberapay.com/yantar92__;!!H8mHWRdzp34!4Q2jIK3jwVcEOegQ2fIC5apK11fdrSkufT30OuWfzKJeRQ_nskag8V97kuVRUbqj41mRZtQD1vykpYx2Qw$ >

_______________________________________________
ESS-core list: https://urldefense.com/v3/__https://stat.ethz.ch/mailman/listinfo/ess-core__;!!H8mHWRdzp34!4Q2jIK3jwVcEOegQ2fIC5apK11fdrSkufT30OuWfzKJeRQ_nskag8V97kuVRUbqj41mRZtQD1vycT2vdFw$<https://urldefense.com/v3/__https:/stat.ethz.ch/mailman/listinfo/ess-core__;!!H8mHWRdzp34!4Q2jIK3jwVcEOegQ2fIC5apK11fdrSkufT30OuWfzKJeRQ_nskag8V97kuVRUbqj41mRZtQD1vycT2vdFw$>

[-- Attachment #2: Type: text/html, Size: 9252 bytes --]

Re: [FR] Add buffer-local setting to request specific ESS process/session name (was: [PATCH] Set Python shell in Org edit buffer)

[Sparapani, Rodney]

[-- Attachment #1: Type: text/plain, Size: 6425 bytes --]

Oops!  Sorry, I misread the signature.  That should be…

Hi Ihor:

Do you have a patch?  I’m not an org-mode user so I can’t test this myself.
Thanks

--
Rodney Sparapani, Associate Professor of Biostatistics, He/Him/His
Vice President, Wisconsin Chapter of the American Statistical Association
Institute for Health and Equity, Division of Biostatistics
Medical College of Wisconsin, Milwaukee Campus

From: Sparapani, Rodney <rsparapa@mcw.edu>
Date: Wednesday, January 10, 2024 at 1:14 PM
To: Ihor Radchenko <yantar92@posteo.net>, Jack Kamm <jackkamm@gmail.com>, ESS-core@r-project.org <ESS-core@r-project.org>
Cc: Liu Hui <liuhui1610@gmail.com>, emacs-orgmode@gnu.org <emacs-orgmode@gnu.org>
Subject: Re: [FR] Add buffer-local setting to request specific ESS process/session name (was: [PATCH] Set Python shell in Org edit buffer)
Hi Jack:

Do you have a patch?  I’m not an org-mode user so I can’t test this myself.
Thanks

--
Rodney Sparapani, Associate Professor of Biostatistics, He/Him/His
Vice President, Wisconsin Chapter of the American Statistical Association
Institute for Health and Equity, Division of Biostatistics
Medical College of Wisconsin, Milwaukee Campus

From: ESS-core <ess-core-bounces@r-project.org> on behalf of Ihor Radchenko <yantar92@posteo.net>
Date: Wednesday, January 10, 2024 at 6:15 AM
To: Jack Kamm <jackkamm@gmail.com>, ESS-core@r-project.org <ESS-core@r-project.org>
Cc: Liu Hui <liuhui1610@gmail.com>, emacs-orgmode@gnu.org <emacs-orgmode@gnu.org>
Subject: [FR] Add buffer-local setting to request specific ESS process/session name (was: [PATCH] Set Python shell in Org edit buffer)
ATTENTION: This email originated from a sender outside of MCW. Use caution when clicking on links or opening attachments.
________________________________

Hi,

I'd like to request a new ESS feature that will allow to choose which
session is created by ESS when no session is yet associated with a
buffer.

Currently, `ess-request-a-process' unconditionally re-uses an existing
ESS process with appropriate `ess-dialect', even when such process is
not consistent with `ess-gen-proc-buffer-name-function'.

This behavior puts Org mode's ESS support in somewhat difficult position
- Org mode allows multiple sessions in Org src blocks, and we want some
way to tell ESS which session process to use for any given buffer
without actually starting that process manually.
We had a hope that setting `ess-gen-proc-buffer-name-function' would
suffice and that ESS would start a new process according to
`ess-gen-proc-buffer-name-function' when no process is yet associated
with buffer. But it is not the case.

Some more context:
(full thread is in
<https://urldefense.com/v3/__https://list.orgmode.org/CAOQTW-O*qs7xAeP7Bemu4ThM4-oGJYxxD*K2jAAw-w7RHtX2gQ@mail.gmail.com/T/*maa33e7c3f72b5c81deb91e1a1d27d57725097fa4__;Kysj!!H8mHWRdzp34!4Q2jIK3jwVcEOegQ2fIC5apK11fdrSkufT30OuWfzKJeRQ_nskag8V97kuVRUbqj41mRZtQD1vyJBoGvlQ$ >)

Jack Kamm <jackkamm@gmail.com> writes:

> I tested the patch (plus the additional change to org-src.el), with an
> Org file with following 2 blocks:
>
> #+begin_src R :session foo :results output
>   print('foo')
> #+end_src
>
> #+begin_src R :session *bar* :results output
>   print('bar')
> #+end_src

These are two R blocks that should be associated with two different ESS
R processes.

> On block "foo", I did C-', and then ess-eval-line. It creates a session
> named "foo", as expected.

When we edit the first block and when no ESS process is available,
`ess-eval-line' respects `ess-gen-proc-buffer-name-function' set by Org
mode, and creates a new ESS process "foo".

> On block "*bar*", I did the same. It does not create session named
> "*bar*", instead evaluating in session "foo". It seems ESS will always
> assume you want to evaluate in existing session if one exists, rather
> than start a new associated session, and it seems there is no way to
> tell it to behave otherwise.

But when the "foo" process is already running, despite different
`ess-gen-proc-buffer-function', `ess-eval-line' connects to the existing
"foo" process rather than "*bar*", as we anticipated.

> However, calling "M-x R" while editing block "*bar*" does create session
> "*bar*" with correct name.
>
> After sessions "foo" and "*bar*" have both been created, doing C-' and
> then ess-eval-line will evaluate in the correct session.

The only workaround, which is not ideal, is to start ESS process
unconditionally. We'd like this to change.

> It's annoying there's no way to tell ESS to start new session instead of
> evaluating in existing one. Here are a few alternatives we could
> consider to deal with this:
>
> 1. Change the worg/ORG-NEWS, to suggest users make sure the session
> exists (either by evaluating a source block or call "M-x R" in edit
> block) before running ess-eval-line.
>
> 2. Add ob-R and ob-julia customization options (as previously suggested)
> to explicitly control the startup behavior (either to auto-start, or not).
>
> 3. Submit PR to ESS to add a variable we could let-bind, to force it to
> start an associated session rather than evaluate in an existing
> non-associated sessions.
>
> Currently I lean towards a combination of #1 and #3, but am not sure,
> and happy to go with whatever you think is best.

--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://urldefense.com/v3/__https://orgmode.org/__;!!H8mHWRdzp34!4Q2jIK3jwVcEOegQ2fIC5apK11fdrSkufT30OuWfzKJeRQ_nskag8V97kuVRUbqj41mRZtQD1vxMkIQ3lQ$ >.
Support Org development at <https://urldefense.com/v3/__https://liberapay.com/org-mode__;!!H8mHWRdzp34!4Q2jIK3jwVcEOegQ2fIC5apK11fdrSkufT30OuWfzKJeRQ_nskag8V97kuVRUbqj41mRZtQD1vzCeD9KoQ$ >,
or support my work at <https://urldefense.com/v3/__https://liberapay.com/yantar92__;!!H8mHWRdzp34!4Q2jIK3jwVcEOegQ2fIC5apK11fdrSkufT30OuWfzKJeRQ_nskag8V97kuVRUbqj41mRZtQD1vykpYx2Qw$ >

_______________________________________________
ESS-core list: https://urldefense.com/v3/__https://stat.ethz.ch/mailman/listinfo/ess-core__;!!H8mHWRdzp34!4Q2jIK3jwVcEOegQ2fIC5apK11fdrSkufT30OuWfzKJeRQ_nskag8V97kuVRUbqj41mRZtQD1vycT2vdFw$<https://urldefense.com/v3/__https:/stat.ethz.ch/mailman/listinfo/ess-core__;!!H8mHWRdzp34!4Q2jIK3jwVcEOegQ2fIC5apK11fdrSkufT30OuWfzKJeRQ_nskag8V97kuVRUbqj41mRZtQD1vycT2vdFw$>

[-- Attachment #2: Type: text/html, Size: 11717 bytes --]

Re: [FR] Add buffer-local setting to request specific ESS process/session name (was: [PATCH] Set Python shell in Org edit buffer)

[Ihor Radchenko]

"Sparapani, Rodney" <rsparapa@mcw.edu> writes:

> Hi Ihor:
>
> Do you have a patch?  I’m not an org-mode user so I can’t test this myself.
> Thanks

Well. I am not exactly ESS user, so I wanted to get a general feedback
first before trying anything blindly.

I think I can demonstrate the problem we are facing without forcing you
to use Org mode though:

1. create and open test.R file containing
   x = "foo"
2. M-: (setq-local ess-gen-proc-buffer-name-function (lambda (_) "session1"))
3. M-x ess-eval-line
4. Observe "session1" R comint buffer displayed
5. create and open test2.R file containing
   y = "bar"
6. M-: (setq-local ess-gen-proc-buffer-name-function (lambda (_) "session2"))
7. M-x ess-eval-line
8. Observe that the line still goes to "session2"

As a non-user, I am not sure if the above is a bug or just some kind of
missing feature. However, for comparison, python.el allows setting
`python-shell-buffer-name' and doing steps similar to the above will
yield two independent python shells to interact with.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [FR] Add buffer-local setting to request specific ESS process/session name (was: [PATCH] Set Python shell in Org edit buffer)

[Sparapani, Rodney]

[-- Attachment #1: Type: text/plain, Size: 2791 bytes --]

I see.  But, I assume that you meant…
8. Observe that the line still goes to "session1"

I usually launch another emacs for “session2”.
There’s probably a way to do this manually,
but, I take your point.  However, if you are a
non-user, then why do you care?

--
Rodney Sparapani, Associate Professor of Biostatistics, He/Him/His
Vice President, Wisconsin Chapter of the American Statistical Association
Institute for Health and Equity, Division of Biostatistics
Medical College of Wisconsin, Milwaukee Campus

From: Ihor Radchenko <yantar92@posteo.net>
Date: Wednesday, January 10, 2024 at 1:28 PM
To: Sparapani, Rodney <rsparapa@mcw.edu>
Cc: Jack Kamm <jackkamm@gmail.com>, ESS-core@r-project.org <ESS-core@r-project.org>, Liu Hui <liuhui1610@gmail.com>, emacs-orgmode@gnu.org <emacs-orgmode@gnu.org>
Subject: Re: [FR] Add buffer-local setting to request specific ESS process/session name (was: [PATCH] Set Python shell in Org edit buffer)
ATTENTION: This email originated from a sender outside of MCW. Use caution when clicking on links or opening attachments.
________________________________

"Sparapani, Rodney" <rsparapa@mcw.edu> writes:

> Hi Ihor:
>
> Do you have a patch?  I’m not an org-mode user so I can’t test this myself.
> Thanks

Well. I am not exactly ESS user, so I wanted to get a general feedback
first before trying anything blindly.

I think I can demonstrate the problem we are facing without forcing you
to use Org mode though:

1. create and open test.R file containing
   x = "foo"
2. M-: (setq-local ess-gen-proc-buffer-name-function (lambda (_) "session1"))
3. M-x ess-eval-line
4. Observe "session1" R comint buffer displayed
5. create and open test2.R file containing
   y = "bar"
6. M-: (setq-local ess-gen-proc-buffer-name-function (lambda (_) "session2"))
7. M-x ess-eval-line
8. Observe that the line still goes to "session2"

As a non-user, I am not sure if the above is a bug or just some kind of
missing feature. However, for comparison, python.el allows setting
`python-shell-buffer-name' and doing steps similar to the above will
yield two independent python shells to interact with.

--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://urldefense.com/v3/__https://orgmode.org/__;!!H8mHWRdzp34!-wEqGdchA5j_KbsspJFtmyWNRaCHt2I9O995IbqLTDFlr_k-WFr8eKVdxmlYby3I--FsLJJplSjQzj5f4w$ >.
Support Org development at <https://urldefense.com/v3/__https://liberapay.com/org-mode__;!!H8mHWRdzp34!-wEqGdchA5j_KbsspJFtmyWNRaCHt2I9O995IbqLTDFlr_k-WFr8eKVdxmlYby3I--FsLJJplSi71O7Wsw$ >,
or support my work at <https://urldefense.com/v3/__https://liberapay.com/yantar92__;!!H8mHWRdzp34!-wEqGdchA5j_KbsspJFtmyWNRaCHt2I9O995IbqLTDFlr_k-WFr8eKVdxmlYby3I--FsLJJplSjYCog7Mw$ >

[-- Attachment #2: Type: text/html, Size: 6548 bytes --]

Re: [FR] Add buffer-local setting to request specific ESS process/session name (was: [PATCH] Set Python shell in Org edit buffer)

[Ihor Radchenko]

"Sparapani, Rodney" <rsparapa@mcw.edu> writes:

> I see.  But, I assume that you meant…
> 8. Observe that the line still goes to "session1"

Yup.

> I usually launch another emacs for “session2”.
> There’s probably a way to do this manually,
> but, I take your point.  However, if you are a
> non-user, then why do you care?

Org mode supports multiple ESS sessions in Org mode src blocks
(https://orgmode.org/worg/org-contrib/babel/languages/ob-doc-R.html#org2a21acd)

However, we currently have to use a workaround that does not always work
well to assign separate sessions to R source blocks in Org mode.
ESS is the only comint mode that requires such workaround and does not
allow setting a specific process buffer name for a given R buffer.

We recently discovered `ess-gen-proc-buffer-name-function', but it does
not work as we expected, unfortunately. (which is either because we
misunderstand something or because there is a bug).

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [External] [FR] Add buffer-local setting to request specific ESS process/session name (was: [PATCH] Set Python shell in Org edit buffer)

[Richard M. Heiberger]

This idiscussion s reminding me of the following ESS functions

ess-add-ess-process           M-x ... RET
   Execute this command from within a buffer running a process to add
ess-request-a-process         M-x ... RET
   Ask for a process, and make it the current ESS process.
ess-switch-process            M-x ... RET
   Force a switch to a new underlying process.

Are these relevant to the question?

Rich

> On Jan 10, 2024, at 15:15, Ihor Radchenko <yantar92@posteo.net> wrote:
>
> "Sparapani, Rodney" <rsparapa@mcw.edu> writes:
>
>> I see.  But, I assume that you meant…
>> 8. Observe that the line still goes to "session1"
>
> Yup.
>
>> I usually launch another emacs for “session2”.
>> There’s probably a way to do this manually,
>> but, I take your point.  However, if you are a
>> non-user, then why do you care?
>
> Org mode supports multiple ESS sessions in Org mode src blocks
> (https://orgmode.org/worg/org-contrib/babel/languages/ob-doc-R.html#org2a21acd)
>
> However, we currently have to use a workaround that does not always work
> well to assign separate sessions to R source blocks in Org mode.
> ESS is the only comint mode that requires such workaround and does not
> allow setting a specific process buffer name for a given R buffer.
>
> We recently discovered `ess-gen-proc-buffer-name-function', but it does
> not work as we expected, unfortunately. (which is either because we
> misunderstand something or because there is a bug).
>
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at <https://orgmode.org/>.
> Support Org development at <https://liberapay.com/org-mode>,
> or support my work at <https://liberapay.com/yantar92>
>
> _______________________________________________
> ESS-core list: https://stat.ethz.ch/mailman/listinfo/ess-core

Re: [External] [FR] Add buffer-local setting to request specific ESS process/session name (was: [PATCH] Set Python shell in Org edit buffer)

[Ihor Radchenko]

"Richard M. Heiberger" <rmh@temple.edu> writes:

> This idiscussion s reminding me of the following ESS functions
>
> ess-request-a-process         M-x ... RET
>    Ask for a process, and make it the current ESS process.

AFAIK, this is the function deciding whether to run a new ESS process or
re-use the existing one. It is responsible for the observed behaviour.

> ess-switch-process            M-x ... RET
>    Force a switch to a new underlying process.

Uses `ess-request-a-process'.

> ess-add-ess-process           M-x ... RET
>    Execute this command from within a buffer running a process to add

Does not look relevant - it seems to be for the process (comint) buffer,
not for the R/other ESS source buffer.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

[ob-clojure] Clojure sessions in Org Src buffers (was: [PATCH] Set Python shell in Org edit buffer)

[Ihor Radchenko]

"Christopher M. Miles" <numbchild@gmail.com> writes:

>> (or (org-babel-comint-buffer-livep session)
>>     (eq org-src-auto-initiate-session t)
>>     (alist-get (nth 0 info) org-src-auto-initiate-session)
>>     (alist-get 'default org-src-auto-initiate-session))
>>
>> With `org-src-auto-initiate-session' being a customization that controls
>> whether to associate session for a given babel backend.
>>
>> We may set the default value to something like
>>
>> ((default . t) ("R" . nil))
>
> I think this customization is reasonable.

Note that we are trying an alternative route with asking upstream ESS
devs to provide a way to associate session without starting it.

>  ... Agree to make it as a
> customization option, The ob-clojure CIDER session which managed by
> "sesman" seems also requires buffer associate with session.

AFAIK, ob-clojure does not support sessions inside Org Src buffers -
there is no `org-babel-associate-clojure-session' or
`org-babel-edit-prep:clojure'.

I am not sure if it is possible to interact with all the supported
Clojure REPLs from inside Org Src blocks.

CCing Daniel - the ob-clojure maintainer.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Set Python shell in Org edit buffer

[Jack Kamm]

Ihor Radchenko <yantar92@posteo.net> writes:

>> It's annoying there's no way to tell ESS to start new session instead of
>> evaluating in existing one. Here are a few alternatives we could
>> consider to deal with this:
>>
>> 1. Change the worg/ORG-NEWS, to suggest users make sure the session
>> exists (either by evaluating a source block or call "M-x R" in edit
>> block) before running ess-eval-line.
>>
>> 2. Add ob-R and ob-julia customization options (as previously suggested)
>> to explicitly control the startup behavior (either to auto-start, or not).
>>
>> 3. Submit PR to ESS to add a variable we could let-bind, to force it to
>> start an associated session rather than evaluate in an existing
>> non-associated sessions.
>>
>> Currently I lean towards a combination of #1 and #3, but am not sure,
>> and happy to go with whatever you think is best.
>
> We can also advice `ess-request-a-process' as a temporary workaround.

My concern is that advising `ess-request-a-process' would cause
maintenance burden on ob-R. It would require some knowledge about the
ESS internals to maintain properly.

Reading through `ess-request-a-process' is rather daunting, and it
doesn't look straightforward to patch it to behave as we want. I think
the reason is because ESS allows you to call `rename-buffer' on the
inferior R session, and still have it remain associated with its editing
buffers. Which is quite a different model than the way python.el works.

After some more thought, I'm now leaning towards #2 as the best and
simplest option. In particular, I think ob-R should have an option
`org-babel-R-start-session-on-edit', which would have 2 options, nil
(default) or t. No need to add an option for the current "earmuffs"
behavior unless we find someone who actually wants it. Then, update Worg
documentation to suggest manually calling "M-x R" if session hasn't
already started yet, or to set `org-babel-R-start-session-on-edit' if
the user would like that behavior.

The alternative (trying to make `ess-eval-line' smart enough to start
new session without user intervention) seems overcomplicated from Org
development/maintenance POV, for too little gain. Clear documentation
and configuration options on Org side are good enough. This is a bit of
an edge case anyways, and unless there is interest from the ESS
developers, I think it is too much struggle to try and change how ESS
works for this. (But thank you for emailing ESS to ask).

Re: [PATCH] Set Python shell in Org edit buffer

[Ihor Radchenko]

Jack Kamm <jackkamm@gmail.com> writes:

>>> Currently I lean towards a combination of #1 and #3, but am not sure,
>>> and happy to go with whatever you think is best.
>>
>> We can also advice `ess-request-a-process' as a temporary workaround.
>
> My concern is that advising `ess-request-a-process' would cause
> maintenance burden on ob-R. It would require some knowledge about the
> ESS internals to maintain properly.

Not really. I only meant writing an advice iff our request is accepted
by ESS devs. Then, all we need is to advice the earlier versions of ESS
and remove the advice after the new ESS release (we only support the
latest release of the optional third-party packages:
https://orgmode.org/worg/org-maintenance.html#emacs-compatibility). No
changes to advice will be needed in future.

I plan to propose a patch for ESS soon and see if it is going to be
accepted.

> Reading through `ess-request-a-process' is rather daunting, and it
> doesn't look straightforward to patch it to behave as we want. I think
> the reason is because ESS allows you to call `rename-buffer' on the
> inferior R session, and still have it remain associated with its editing
> buffers. Which is quite a different model than the way python.el works.

We can simply let-bind `ess-process-name-list' to alter what
`ess-request-a-process' returns. I do not see major problems here.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Set Python shell in Org edit buffer

[Jack Kamm]

Ihor Radchenko <yantar92@posteo.net> writes:

>> My concern is that advising `ess-request-a-process' would cause
>> maintenance burden on ob-R. It would require some knowledge about the
>> ESS internals to maintain properly.
>
> Not really. I only meant writing an advice iff our request is accepted
> by ESS devs. Then, all we need is to advice the earlier versions of ESS
> and remove the advice after the new ESS release (we only support the
> latest release of the optional third-party packages:
> https://orgmode.org/worg/org-maintenance.html#emacs-compatibility). No
> changes to advice will be needed in future.
>
> I plan to propose a patch for ESS soon and see if it is going to be
> accepted.

In that case, this sounds great. Hope the ESS devs are receptive!

[PATCH] Add buffer-local setting to request specific ESS process/session name (was: [PATCH] Set Python shell in Org edit buffer)

[Ihor Radchenko]

[-- Attachment #1: Type: text/plain, Size: 358 bytes --]

Ihor Radchenko <yantar92@posteo.net> writes:

> We recently discovered `ess-gen-proc-buffer-name-function', but it does
> not work as we expected, unfortunately. (which is either because we
> misunderstand something or because there is a bug).

I am attaching tentative patch that will make `ess-request-a-process'
obey `ess-gen-proc-buffer-name-function'.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ess-request-a-process-Honor-ess-gen-proc-buffer-name.patch --]
[-- Type: text/x-patch, Size: 2084 bytes --]

From 68984a5a0a1df5a5a2619b579f23f70128e979cd Mon Sep 17 00:00:00 2001
Message-ID: <68984a5a0a1df5a5a2619b579f23f70128e979cd.1705837631.git.yantar92@posteo.net>
From: Ihor Radchenko <yantar92@posteo.net>
Date: Sun, 21 Jan 2024 12:44:32 +0100
Subject: [PATCH] ess-request-a-process: Honor
 ess-gen-proc-buffer-name-function

* lisp/ess-inf.el (ess-request-a-process): Do not make processes not
matching `ess-gen-proc-buffer-name-function' current.
---
 lisp/ess-inf.el | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/lisp/ess-inf.el b/lisp/ess-inf.el
index 9ca3f455..aaff314c 100644
--- a/lisp/ess-inf.el
+++ b/lisp/ess-inf.el
@@ -820,7 +820,7 @@ (defun ess-request-a-process (message &optional noswitch ask-if-1)
                            (delete-dups (list "R" "S+" (or (bound-and-true-p S+-dialect-name) "S+")
                                               "stata" (or (bound-and-true-p STA-dialect-name) "stata")
                                               "julia" "SAS")))))
-         (pname-list (delq nil ;; keep only those matching dialect
+         (pname-list (delq nil ;; keep only those matching dialect and `ess-gen-proc-buffer-name-function'
                            (append
                             (mapcar (lambda (lproc)
                                       (and (equal ess-dialect
@@ -828,6 +828,8 @@ (defun ess-request-a-process (message &optional noswitch ask-if-1)
                                                    'ess-dialect
                                                    (process-buffer (get-process (car lproc)))))
                                            (not (equal ess-local-process-name (car lproc)))
+                                           (equal (buffer-name (process-buffer (get-process (car lproc))))
+                                                  (funcall ess-gen-proc-buffer-name-function (car lproc)))
                                            (car lproc)))
                                     ess-process-name-list)
                             ;; append local only if running
-- 
2.43.0


[-- Attachment #3: Type: text/plain, Size: 224 bytes --]


-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Add buffer-local setting to request specific ESS process/session name (was: [PATCH] Set Python shell in Org edit buffer)

[Sparapani, Rodney]

[-- Attachment #1: Type: text/plain, Size: 1844 bytes --]

Hi Idor:

I’ve test that and it works for me per your prescription below.
And, I have committed it to the ESS git repo.  Thanks!

1. create and open test.R file containing
   x = "foo"
2. M-: (setq-local ess-gen-proc-buffer-name-function (lambda (_) "session1"))
3. M-x ess-eval-line
4. Observe "session1" R comint buffer displayed
5. create and open test2.R file containing
   y = "bar"
6. M-: (setq-local ess-gen-proc-buffer-name-function (lambda (_) "session2"))
7. M-x ess-eval-line
8. Observe that the line still goes to "session2"

--
Rodney Sparapani, Associate Professor of Biostatistics, He/Him/His
Vice President, Wisconsin Chapter of the American Statistical Association
Institute for Health and Equity, Division of Biostatistics
Medical College of Wisconsin, Milwaukee Campus

If this is outside of working hours, then please respond when convenient.

From: Ihor Radchenko <yantar92@posteo.net>
Date: Sunday, January 21, 2024 at 5:45 AM
To: Sparapani, Rodney <rsparapa@mcw.edu>
Cc: Jack Kamm <jackkamm@gmail.com>, ESS-core@r-project.org <ESS-core@r-project.org>, Liu Hui <liuhui1610@gmail.com>, emacs-orgmode@gnu.org <emacs-orgmode@gnu.org>
Subject: [PATCH] Add buffer-local setting to request specific ESS process/session name (was: [PATCH] Set Python shell in Org edit buffer)
ATTENTION: This email originated from a sender outside of MCW. Use caution when clicking on links or opening attachments.
________________________________

Ihor Radchenko <yantar92@posteo.net> writes:

> We recently discovered `ess-gen-proc-buffer-name-function', but it does
> not work as we expected, unfortunately. (which is either because we
> misunderstand something or because there is a bug).

I am attaching tentative patch that will make `ess-request-a-process'
obey `ess-gen-proc-buffer-name-function'.

[-- Attachment #2: Type: text/html, Size: 5253 bytes --]

Re: [PATCH] Add buffer-local setting to request specific ESS process/session name (was: [PATCH] Set Python shell in Org edit buffer)

[Ihor Radchenko]

"Sparapani, Rodney" <rsparapa@mcw.edu> writes:

> I’ve test that and it works for me per your prescription below.
> And, I have committed it to the ESS git repo.

Thank you!
May I know which version of ESS will have this commit?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Add buffer-local setting to request specific ESS process/session name (was: [PATCH] Set Python shell in Org edit buffer)

[Martin Maechler]

>>>>> Ihor Radchenko 
>>>>>     on Mon, 22 Jan 2024 12:13:15 +0000 writes:

    > "Sparapani, Rodney" <rsparapa@mcw.edu> writes:
    >> I’ve test that and it works for me per your prescription
    >> below.  And, I have committed it to the ESS git repo.

    > Thank you!  May I know which version of ESS will have this
    > commit?

The Melpa version that was just released, does.

As you probably know more than me about emacs package
maintenance and release cycles: 
Do you think we should also try to get a  Melpa-stable release?

Is the wording we have about MELPA in
   https://ess.r-project.org/Manual/readme.html#Installing-from-a-third_002dparty-repository
satisfactory?

Thank you, Ihor, for your contributions!

Best,
Martin

--
Martin Maechler
ETH Zurich, Switzerland

Re: [PATCH] Add buffer-local setting to request specific ESS process/session name (was: [PATCH] Set Python shell in Org edit buffer)

[Ihor Radchenko]

Martin Maechler <maechler@stat.math.ethz.ch> writes:

>     > Thank you!  May I know which version of ESS will have this
>     > commit?
>
> The Melpa version that was just released, does.

Noted. I did not know that ESS is distributed via MELPA.
I am wondering why not ELPA/non-GNU ELPA - ELPA is built-in and users
can simply M-x package-install packages from ELPA repositories.

> As you probably know more than me about emacs package
> maintenance and release cycles: 
> Do you think we should also try to get a  Melpa-stable release?

That's up to you to decide if ESS is in stable state or not.
You may find https://orgmode.org/worg/org-maintenance.html#release-types
useful to see how Org mode handles releases.

> Is the wording we have about MELPA in
>    https://ess.r-project.org/Manual/readme.html#Installing-from-a-third_002dparty-repository
> satisfactory?

Looks fine.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Add buffer-local setting to request specific ESS process/session name (was: [PATCH] Set Python shell in Org edit buffer)

[Sparapani, Rodney]

[-- Attachment #1: Type: text/plain, Size: 3532 bytes --]

Hi Idor:

I have never had much success with either MELPA or ELPA.  Something about
my TLS setup that I have not figured how to fix.  But, I did receive an email that
we are available at ELPA too (however, I can’t test the veracity of that myself).
See the ESS-help thread here….
https://www.mail-archive.com/ess-help@r-project.org/msg01043.html

--
Rodney Sparapani, Associate Professor of Biostatistics, He/Him/His
Vice President, Wisconsin Chapter of the American Statistical Association
Institute for Health and Equity, Division of Biostatistics
Medical College of Wisconsin, Milwaukee Campus

If this is outside of working hours, then please respond when convenient.

From: Ihor Radchenko <yantar92@posteo.net>
Date: Thursday, January 25, 2024 at 7:05 AM
To: Martin Maechler <maechler@stat.math.ethz.ch>
Cc: Sparapani, Rodney <rsparapa@mcw.edu>, ESS-core@r-project.org <ESS-core@r-project.org>, Jack Kamm <jackkamm@gmail.com>, emacs-orgmode@gnu.org <emacs-orgmode@gnu.org>, Liu Hui <liuhui1610@gmail.com>
Subject: Re: [PATCH] Add buffer-local setting to request specific ESS process/session name (was: [PATCH] Set Python shell in Org edit buffer)
ATTENTION: This email originated from a sender outside of MCW. Use caution when clicking on links or opening attachments.
________________________________

Martin Maechler <maechler@stat.math.ethz.ch> writes:

>     > Thank you!  May I know which version of ESS will have this
>     > commit?
>
> The Melpa version that was just released, does.

Noted. I did not know that ESS is distributed via MELPA.
I am wondering why not ELPA/non-GNU ELPA - ELPA is built-in and users
can simply M-x package-install packages from ELPA repositories.

> As you probably know more than me about emacs package
> maintenance and release cycles:
> Do you think we should also try to get a  Melpa-stable release?

That's up to you to decide if ESS is in stable state or not.
You may find https://urldefense.com/v3/__https://orgmode.org/worg/org-maintenance.html*release-types__;Iw!!H8mHWRdzp34!9qd7lXuuGaLyWEAPOcHUmf_Fr_oYuyym5rkIQSSfGycheI8pfc2DqZqFLvG8jwINxn5hSssSSxjLZi_mEg$<https://urldefense.com/v3/__https:/orgmode.org/worg/org-maintenance.html*release-types__;Iw!!H8mHWRdzp34!9qd7lXuuGaLyWEAPOcHUmf_Fr_oYuyym5rkIQSSfGycheI8pfc2DqZqFLvG8jwINxn5hSssSSxjLZi_mEg$>
useful to see how Org mode handles releases.

> Is the wording we have about MELPA in
>    https://urldefense.com/v3/__https://ess.r-project.org/Manual/readme.html*Installing-from-a-third_002dparty-repository__;Iw!!H8mHWRdzp34!9qd7lXuuGaLyWEAPOcHUmf_Fr_oYuyym5rkIQSSfGycheI8pfc2DqZqFLvG8jwINxn5hSssSSxhsgRpn0A$<https://urldefense.com/v3/__https:/ess.r-project.org/Manual/readme.html*Installing-from-a-third_002dparty-repository__;Iw!!H8mHWRdzp34!9qd7lXuuGaLyWEAPOcHUmf_Fr_oYuyym5rkIQSSfGycheI8pfc2DqZqFLvG8jwINxn5hSssSSxhsgRpn0A$>
> satisfactory?

Looks fine.

--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://urldefense.com/v3/__https://orgmode.org/__;!!H8mHWRdzp34!9qd7lXuuGaLyWEAPOcHUmf_Fr_oYuyym5rkIQSSfGycheI8pfc2DqZqFLvG8jwINxn5hSssSSxgmy_SMoA$ >.
Support Org development at <https://urldefense.com/v3/__https://liberapay.com/org-mode__;!!H8mHWRdzp34!9qd7lXuuGaLyWEAPOcHUmf_Fr_oYuyym5rkIQSSfGycheI8pfc2DqZqFLvG8jwINxn5hSssSSxjREUC_7Q$ >,
or support my work at <https://urldefense.com/v3/__https://liberapay.com/yantar92__;!!H8mHWRdzp34!9qd7lXuuGaLyWEAPOcHUmf_Fr_oYuyym5rkIQSSfGycheI8pfc2DqZqFLvG8jwINxn5hSssSSxiZiaCMNA$ >

[-- Attachment #2: Type: text/html, Size: 7465 bytes --]

Re: [PATCH] Add buffer-local setting to request specific ESS process/session name (was: [PATCH] Set Python shell in Org edit buffer)

[Ihor Radchenko]

"Sparapani, Rodney" <rsparapa@mcw.edu> writes:

> I have never had much success with either MELPA or ELPA.  Something about
> my TLS setup that I have not figured how to fix.  But, I did receive an email that
> we are available at ELPA too (however, I can’t test the veracity of that myself).
> See the ESS-help thread here….
> https://www.mail-archive.com/ess-help@r-project.org/msg01043.html

I see. ESS is indeed listed on ELPA https://elpa.gnu.org/packages/ess.html

Note that ELPA is the official package repository for GNU Emacs. So, it
is worth documenting that ESS is distributed via ELPA in
https://ess.r-project.org/Manual/readme.html#Installing-from-a-third_002dparty-repository

Also, unlike MELPA, ELPA uses stable (tagged) releases by default.
Unstable releases (latest commit) are only available on demand, via
ELPA-devel (https://elpa.gnu.org/devel/ess.html)

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Add buffer-local setting to request specific ESS process/session name (was: [PATCH] Set Python shell in Org edit buffer)

[Sparapani, Rodney]

[-- Attachment #1: Type: text/plain, Size: 1923 bytes --]

Hi Idor:

This last remark below can’t possibly be true.  Because I did not intend to trigger
a release, ESS 24.01.0 was not initially tagged.  However, after I received that email
from ELPA, it was a fait accompli.  So I fixed some cosmetic issues and tagged it post-haste,
i.e., ELPA’s announcement is definitely pre-tagging.  So I’m not sure where the disconnect is.
But, again, I don’t find the package mechanism at all convenient and will have to spend some
time debugging since it has never worked for me.  I will put these ELPA/MELPA issues on my
growing TODO list.  Thanks for pointing this out!

--
Rodney Sparapani, Associate Professor of Biostatistics, He/Him/His
Vice President, Wisconsin Chapter of the American Statistical Association
Institute for Health and Equity, Division of Biostatistics
Medical College of Wisconsin, Milwaukee Campus

If this is outside of working hours, then please respond when convenient.

From: Ihor Radchenko <yantar92@posteo.net>
Date: Thursday, January 25, 2024 at 9:30 AM
To: Sparapani, Rodney <rsparapa@mcw.edu>
Cc: Martin Maechler <maechler@stat.math.ethz.ch>, ESS-core@r-project.org <ESS-core@r-project.org>, Jack Kamm <jackkamm@gmail.com>, emacs-orgmode@gnu.org <emacs-orgmode@gnu.org>, Liu Hui <liuhui1610@gmail.com>
Subject: Re: [PATCH] Add buffer-local setting to request specific ESS process/session name (was: [PATCH] Set Python shell in Org edit buffer)
Also, unlike MELPA, ELPA uses stable (tagged) releases by default.
Unstable releases (latest commit) are only available on demand, via
ELPA-devel (https://urldefense.com/v3/__https://elpa.gnu.org/devel/ess.html__;!!H8mHWRdzp34!98PglwLJJtQrc63k3vrtr0K0MjYZSPEf3WV0GPD_wsxbsO3bM8AYCDRI0W69ZcBg5STqpaG7_h3rF8wlAQ$<https://urldefense.com/v3/__https:/elpa.gnu.org/devel/ess.html__;!!H8mHWRdzp34!98PglwLJJtQrc63k3vrtr0K0MjYZSPEf3WV0GPD_wsxbsO3bM8AYCDRI0W69ZcBg5STqpaG7_h3rF8wlAQ$> )

[-- Attachment #2: Type: text/html, Size: 5023 bytes --]

Re: [PATCH] Add buffer-local setting to request specific ESS process/session name (was: [PATCH] Set Python shell in Org edit buffer)

[Ihor Radchenko]

"Sparapani, Rodney" <rsparapa@mcw.edu> writes:

> This last remark below can’t possibly be true.  Because I did not intend to trigger
> a release, ESS 24.01.0 was not initially tagged.  However, after I received that email
> from ELPA, it was a fait accompli.  So I fixed some cosmetic issues and tagged it post-haste,
> i.e., ELPA’s announcement is definitely pre-tagging.  So I’m not sure
> where the disconnect is.

Hmm. You are right. The exact rule is following package Version:
comment:

https://git.savannah.gnu.org/cgit/emacs/elpa.git/plain/README

    *** Release and devel archives
    elpa.gnu.org serves the gnu and nongnu package collections (roughly,
    gnu requires FSF copyright assign, nongnu doesn't, but these terms are
    not fully defined here).
    
    In addition, elpa.gnu.org serves release and devel versions of each
    package. The release version is defined by a change in the =Version:=
    header of a package; the devel version is the latest commit.

So, you likely changed Version: in lisp/ess.el first and only then added
git tag.

(I always do this in one go, which is why I never noticed this detail)

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Set Python shell in Org edit buffer

[Ihor Radchenko]

Jack Kamm <jackkamm@gmail.com> writes:

> Ihor Radchenko <yantar92@posteo.net> writes:
>
>>> My concern is that advising `ess-request-a-process' would cause
>>> maintenance burden on ob-R. It would require some knowledge about the
>>> ESS internals to maintain properly.
>>
>> Not really. I only meant writing an advice iff our request is accepted
>> by ESS devs. Then, all we need is to advice the earlier versions of ESS
>> and remove the advice after the new ESS release (we only support the
>> latest release of the optional third-party packages:
>> https://orgmode.org/worg/org-maintenance.html#emacs-compatibility). No
>> changes to advice will be needed in future.
>>
>> I plan to propose a patch for ESS soon and see if it is going to be
>> accepted.
>
> In that case, this sounds great. Hope the ESS devs are receptive!

Applied, onto main, fixing the oversight in
org-src-associate-babel-session (now, it does not require session
running).
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=319563cef

Since ESS already released a new version with my patch for ESS included,
no advice is needed.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Set Python shell in Org edit buffer

[Ihor Radchenko]

[-- Attachment #1: Type: text/plain, Size: 530 bytes --]

Liu Hui <liuhui1610@gmail.com> writes:

> Yes, I have updated the text and you're welcome to improve it. Thanks!
> From c503b2ed5116e2abae25459b09abc919074ac54a Mon Sep 17 00:00:00 2001
> From: Liu Hui <liuhui1610@gmail.com>
> Date: Tue, 5 Dec 2023 11:40:38 +0800
> Subject: [PATCH] Set Python shell in Org edit buffer

Now, after amending `org-src-associate-babel-session' to execute even
when no session is active, we can use
`org-babel-python-associate-session'.

Attaching tentative patch that should be equivalent to yours.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ob-python-Set-Python-shell-in-Org-edit-buffer.patch --]
[-- Type: text/x-patch, Size: 2330 bytes --]

From f8a748aee619f9fa3f9104321b33212447e7f952 Mon Sep 17 00:00:00 2001
Message-ID: <f8a748aee619f9fa3f9104321b33212447e7f952.1706473998.git.yantar92@posteo.net>
From: Ihor Radchenko <yantar92@posteo.net>
Date: Sun, 28 Jan 2024 21:29:25 +0100
Subject: [PATCH] ob-python: Set Python shell in Org edit buffer

* lisp/ob-python.el (org-babel-python-associate-session): New function
setting `python-shell-buffer-name' in *Org Src* buffer according to
source block's :session parameter.  This function will be triggered by
`org-src-associate-babel-session'.
* etc/ORG-NEWS (ob-python now sets ~python-shell-buffer-name~ in Org
edit buffers): Announce the change.

Link: https://orgmode.org/list/CAOQTW-MdC=jiGf+3bEVtfww+izSZix7csBJ+mZ4eZ2BQHDR42w@mail.gmail.com
---
 etc/ORG-NEWS      | 7 +++++++
 lisp/ob-python.el | 6 ++++++
 2 files changed, 13 insertions(+)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index f9c916a9d..2163f00e1 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -13,6 +13,13 @@ Please send Org bug reports to mailto:emacs-orgmode@gnu.org.
 
 * Version 9.7 (not released yet)
 ** Important announcements and breaking changes
+*** ob-python now sets ~python-shell-buffer-name~ in Org edit buffers
+
+When editing a Python src block, the editing buffer is now associated
+with the Python shell specified by the src block's ~:session~ header,
+which means users can now send code directly from the edit buffer,
+e.g., using ~C-c C-c~, to the session specified in the Org buffer.
+
 *** ~org-edit-special~ no longer force-starts session in R and Julia source blocks
 
 Previously, when R/Julia source block had =:session= header argument
diff --git a/lisp/ob-python.el b/lisp/ob-python.el
index 4d7492e2b..2f62d7353 100644
--- a/lisp/ob-python.el
+++ b/lisp/ob-python.el
@@ -83,6 +83,12 @@ (defcustom org-babel-python-None-to 'hline
   :package-version '(Org . "8.0")
   :type 'symbol)
 
+(defun org-babel-python-associate-session (session)
+  "Associate Python code buffer with an Python session.
+Make SESSION without earmuffs be the Python buffer name."
+  (setq-local python-shell-buffer-name
+              (org-babel-python-without-earmuffs session)))
+
 (defun org-babel-execute:python (body params)
   "Execute Python BODY according to PARAMS.
 This function is called by `org-babel-execute-src-block'."
-- 
2.43.0


[-- Attachment #3: Type: text/plain, Size: 224 bytes --]


-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Set Python shell in Org edit buffer

[Jack Kamm]

Ihor Radchenko <yantar92@posteo.net> writes:

> Liu Hui <liuhui1610@gmail.com> writes:
>
>> Yes, I have updated the text and you're welcome to improve it. Thanks!
>> From c503b2ed5116e2abae25459b09abc919074ac54a Mon Sep 17 00:00:00 2001
>> From: Liu Hui <liuhui1610@gmail.com>
>> Date: Tue, 5 Dec 2023 11:40:38 +0800
>> Subject: [PATCH] Set Python shell in Org edit buffer
>
> Now, after amending `org-src-associate-babel-session' to execute even
> when no session is active, we can use
> `org-babel-python-associate-session'.
>
> Attaching tentative patch that should be equivalent to yours.

Thanks Ihor. I tested the patch and it seems to work well.

Re: [PATCH] Set Python shell in Org edit buffer

[Jack Kamm]

Ihor Radchenko <yantar92@posteo.net> writes:

> Applied, onto main, fixing the oversight in
> org-src-associate-babel-session (now, it does not require session
> running).
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=319563cef
>
> Since ESS already released a new version with my patch for ESS included,
> no advice is needed.

Thank you!

Re: [PATCH] Set Python shell in Org edit buffer

[Ihor Radchenko]

Jack Kamm <jackkamm@gmail.com> writes:

>> Now, after amending `org-src-associate-babel-session' to execute even
>> when no session is active, we can use
>> `org-babel-python-associate-session'.
>>
>> Attaching tentative patch that should be equivalent to yours.
>
> Thanks Ihor. I tested the patch and it seems to work well.

Thanks for testing!
Applied, onto main, adding Liu Hui as co-author.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=107cbc74a

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>