Evaluating python code blocks in python-mode

Evaluating python code blocks in python-mode

[Yuri D'Elia]

Hi everyone,

I'd like some advice about using python-mode with a python subprocess.
I always preferred to use emacs+file+external process as opposed to use
an ipython's notebook-like interface: it's just more convenient,
especially when debugging existing code.

I often use the same setup when doing some analysis, and this is when an
inotebook-like interface is more convenient for evaluation: I'd like to
evaluate code not line-by-line, or by defun, but by my custom-defined
blocks. I realized I was continuously selecting a region, and using
py-send-region over and over.

Right now I narrow to the region I'm editing, and use py-send-buffer
instead. It's ok-eish, but narrow/widen narrow/widen, while less common,
is still inconvenient.

I've seen people using org-mode for this kind of setup instead. Which
might be ok, but I'd like to know what other choices I could have as well.

I was thinking of defining a custom region using comments, and defining
my own py-send-custom-block using comment markers as boundaries. Surely,
something like this must already exists.

Re: Evaluating python code blocks in python-mode

[Andreas Röhler]

Am 20.06.2015 um 12:50 schrieb Yuri D'Elia:
> Hi everyone,
>
> I'd like some advice about using python-mode with a python subprocess.
> I always preferred to use emacs+file+external process as opposed to use
> an ipython's notebook-like interface: it's just more convenient,
> especially when debugging existing code.
>
> I often use the same setup when doing some analysis, and this is when an
> inotebook-like interface is more convenient for evaluation: I'd like to
> evaluate code not line-by-line, or by defun, but by my custom-defined
> blocks. I realized I was continuously selecting a region, and using
> py-send-region over and over.
>
> Right now I narrow to the region I'm editing, and use py-send-buffer
> instead. It's ok-eish, but narrow/widen narrow/widen, while less common,
> is still inconvenient.
>
> I've seen people using org-mode for this kind of setup instead. Which
> might be ok, but I'd like to know what other choices I could have as well.
>
> I was thinking of defining a custom region using comments, and defining
> my own py-send-custom-block using comment markers as boundaries. Surely,
> something like this must already exists.
>
>


IIUC in question are arbitrary chunks of code.
What about something like that:

;;;;

(defvar py-section-start "# {{")
(defvar py-section-end "# }}")

(defun py-send-section ()
   (interactive)
   (save-excursion
     (unless (looking-at py-section-start)
       (search-backward py-section-start)
       (set-mark (point))
       (if (and (looking-at py-section-start)(search-forward 
py-section-end))
       (py-execute-region (region-beginning) (region-end))
     (error "Can't see boundaries of py-section")))))

;;;;

BTW didn't call it block, as this is already used by forms heading others.

Re: Evaluating python code blocks in python-mode

[Andreas Röhler]

Am 20.06.2015 um 21:09 schrieb Andreas Röhler:
>
> Am 20.06.2015 um 12:50 schrieb Yuri D'Elia:
>> Hi everyone,
>>
>> I'd like some advice about using python-mode with a python subprocess.
>> I always preferred to use emacs+file+external process as opposed to use
>> an ipython's notebook-like interface: it's just more convenient,
>> especially when debugging existing code.
>>
>> I often use the same setup when doing some analysis, and this is when an
>> inotebook-like interface is more convenient for evaluation: I'd like to
>> evaluate code not line-by-line, or by defun, but by my custom-defined
>> blocks. I realized I was continuously selecting a region, and using
>> py-send-region over and over.
>>
>> Right now I narrow to the region I'm editing, and use py-send-buffer
>> instead. It's ok-eish, but narrow/widen narrow/widen, while less common,
>> is still inconvenient.
>>
>> I've seen people using org-mode for this kind of setup instead. Which
>> might be ok, but I'd like to know what other choices I could have as 
>> well.
>>
>> I was thinking of defining a custom region using comments, and defining
>> my own py-send-custom-block using comment markers as boundaries. Surely,
>> something like this must already exists.
>>
>>
>
>
> IIUC in question are arbitrary chunks of code.
> What about something like that:
>
> ;;;;
>
> (defvar py-section-start "# {{")
> (defvar py-section-end "# }}")
>
> (defun py-send-section ()
>   (interactive)
>   (save-excursion
>     (unless (looking-at py-section-start)
>       (search-backward py-section-start)
>       (set-mark (point))
>       (if (and (looking-at py-section-start)(search-forward 
> py-section-end))
>       (py-execute-region (region-beginning) (region-end))
>     (error "Can't see boundaries of py-section")))))
>
> ;;;;
>
> BTW didn't call it block, as this is already used by forms heading 
> others.
>
>
>

As python-mode sends a plain string to interpreter anyway, must not 
touch mark in buffer:

(defun py-send-section ()
   (interactive)
   (save-excursion
     (let ((start (progn (unless (looking-at py-section-start)
              (search-backward py-section-start)
              (point)))))
       (if (and (looking-at py-section-start)(search-forward 
py-section-end))
       (py-execute-region start (point))
     (error "Can't see boundaries of py-section")))))

Re: Evaluating python code blocks in python-mode

[Yuri D'Elia]

On 06/21/2015 08:20 AM, Andreas Röhler wrote:
>> IIUC in question are arbitrary chunks of code.
>> What about something like that:
<...>
> 
> (defun py-send-section ()
>    (interactive)
>    (save-excursion
>      (let ((start (progn (unless (looking-at py-section-start)
>               (search-backward py-section-start)
>               (point)))))
>        (if (and (looking-at py-section-start)(search-forward 
> py-section-end))
>        (py-execute-region start (point))
>      (error "Can't see boundaries of py-section")))))

Yes, this is what I had in mind, however I was hoping there was a more
general established approach for this sort of thing. Since I'm also
using many other languages with an interpreter (ess-mode, tuareg to
mention two that I regularly use), I'd like a general approach.

For example, if we change py-section-start/end to use syntax-table, we
can define a general approach to defining code "chunks" using delimited
comments.

I was hoping there was already a global minor mode that does something
similar to delimit code chunks.

Re: Evaluating python code blocks in python-mode

[Yuri D'Elia]

On 06/21/2015 08:20 AM, Andreas Röhler wrote:
>> IIUC in question are arbitrary chunks of code.
>> What about something like that:
<...>
> 
> (defun py-send-section ()
>    (interactive)
>    (save-excursion
>      (let ((start (progn (unless (looking-at py-section-start)
>               (search-backward py-section-start)
>               (point)))))
>        (if (and (looking-at py-section-start)(search-forward 
> py-section-end))
>        (py-execute-region start (point))
>      (error "Can't see boundaries of py-section")))))

Yes, this is what I had in mind, however I was hoping there was a more
general established approach for this sort of thing. Since I'm also
using many other languages with an interpreter (ess-mode, tuareg to
mention two that I regularly use), I'd like a general approach.

For example, if we change py-section-start/end to use syntax-table, we
can define a general approach to defining code "chunks" using delimited
comments.

I was hoping there was already a global minor mode that does something
similar to delimit code chunks.

Re: Evaluating python code blocks in python-mode

[Yuri D'Elia]

On 06/21/2015 06:18 PM, Yuri D'Elia wrote:
> Yes, this is what I had in mind, however I was hoping there was a more
> general established approach for this sort of thing. Since I'm also
> using many other languages with an interpreter (ess-mode, tuareg to
> mention two that I regularly use), I'd like a general approach.
> 
> For example, if we change py-section-start/end to use syntax-table, we
> can define a general approach to defining code "chunks" using delimited
> comments.
> 
> I was hoping there was already a global minor mode that does something
> similar to delimit code chunks.

Maybe the best approach would simply be to reuse `folding-mode' per-mode
marks (as in: py-send-current-fold).

Re: Evaluating python code blocks in python-mode

[Andreas Röhler]

Am 21.06.2015 um 18:35 schrieb Yuri D'Elia:
> On 06/21/2015 06:18 PM, Yuri D'Elia wrote:
>> Yes, this is what I had in mind, however I was hoping there was a more
>> general established approach for this sort of thing. Since I'm also
>> using many other languages with an interpreter (ess-mode, tuareg to
>> mention two that I regularly use), I'd like a general approach.
>>
>> For example, if we change py-section-start/end to use syntax-table, we
>> can define a general approach to defining code "chunks" using delimited
>> comments.
>>
>> I was hoping there was already a global minor mode that does something
>> similar to delimit code chunks.
> Maybe the best approach would simply be to reuse `folding-mode' per-mode
> marks (as in: py-send-current-fold).
>
>
>

It might be an option to use existing markers from folding-mode. OTOH 
don't see a command there how to copy the section. Do you?

Re: Evaluating python code blocks in python-mode

[Yuri D'Elia]

On 06/21/2015 08:52 PM, Andreas Röhler wrote:
> It might be an option to use existing markers from folding-mode. OTOH 
> don't see a command there how to copy the section. Do you?

So this is what I came up with and I currently use:

http://www.thregr.org/~wavexx/tmp/python-mode-extra.el

Sorry for not inlining the file, but I think it's more readable.

The idea is that "C-c C-c" sends the current fold you're in. If you're in a nested fold, subfolds are send too. If you're not in a fold, the buffer (or narrowed region) is sent instead.

As an added bonus, I wanted to define a generic block delimiter, which is # --- in my case. `python-shell-send-fold-or-section' will send the current fold if there's any, up to the closest block delimiter (if any).

So for example, in a file like:

 print "1"
 # ---
 print "2"

"C-c C-c" on either print statement will be limited to that statement. Block delimiters can be nested inside folds, and work as you'd expect: if you're in the same fold as a delimiter, the delimiter is used. If you're upwards, the delimiter is ignored.

This makes it very easy to delimit "computation blocks", as you'd normally do in an interactive notebook, without the burden to define folds for each.

To top that, the region being sent is added with a volatile highlight (volatile-highlights.el is required) so you see what's being evaluated.

I love it.

It works in Fabián's python-mode, but should work with minimal changes with other modes too.

I actually redefine some bindings for python-mode to behave more like ESS (which IMHO has a better layout for interactive evaluation), resulting in something like:

(add-hook 'python-mode-hook
	  (lambda ()
	    (local-set-key (kbd "C-c C-j") 'python-shell-send-line)
	    (local-set-key (kbd "C-c C-n") 'python-shell-send-line and-step)
	    (local-set-key (kbd "C-c C-f") 'python-shell-send-defun)
	    (local-set-key (kbd "C-c C-b") 'python-shell-send-buffer)
	    (local-set-key (kbd "C-c C-c") 'python-shell-send-fold-or-section)))

`python-shell-send-fold-or-section' is what is defined in python-mode-extra.el, which was a little tricker to define than anticipated, but it seems to work correctly.

I re-use the markers defined in folding-mode, so if you want to customize the markers just use `folding-add-to-marks-list'. folding-mode doesn't need to be enabled. I actually don't use folding, I just wanted to re-use the same methods.

Re: Evaluating python code blocks in python-mode

[Yuri D'Elia]

On 06/21/2015 08:52 PM, Andreas Röhler wrote:
> It might be an option to use existing markers from folding-mode. OTOH 
> don't see a command there how to copy the section. Do you?

So this is what I came up with and I currently use:

http://www.thregr.org/~wavexx/tmp/python-mode-extra.el

Sorry for not inlining the file, but I think it's more readable.

The idea is that "C-c C-c" sends the current fold you're in. If you're in a nested fold, subfolds are send too. If you're not in a fold, the buffer (or narrowed region) is sent instead.

As an added bonus, I wanted to define a generic block delimiter, which is # --- in my case. `python-shell-send-fold-or-section' will send the current fold if there's any, up to the closest block delimiter (if any).

So for example, in a file like:

 print "1"
 # ---
 print "2"

"C-c C-c" on either print statement will be limited to that statement. Block delimiters can be nested inside folds, and work as you'd expect: if you're in the same fold as a delimiter, the delimiter is used. If you're upwards, the delimiter is ignored.

This makes it very easy to delimit "computation blocks", as you'd normally do in an interactive notebook, without the burden to define folds for each.

To top that, the region being sent is added with a volatile highlight (volatile-highlights.el is required) so you see what's being evaluated.

I love it.

It works in Fabián's python-mode, but should work with minimal changes with other modes too.

I actually redefine some bindings for python-mode to behave more like ESS (which IMHO has a better layout for interactive evaluation), resulting in something like:

(add-hook 'python-mode-hook
	  (lambda ()
	    (local-set-key (kbd "C-c C-j") 'python-shell-send-line)
	    (local-set-key (kbd "C-c C-n") 'python-shell-send-line and-step)
	    (local-set-key (kbd "C-c C-f") 'python-shell-send-defun)
	    (local-set-key (kbd "C-c C-b") 'python-shell-send-buffer)
	    (local-set-key (kbd "C-c C-c") 'python-shell-send-fold-or-section)))

`python-shell-send-fold-or-section' is what is defined in python-mode-extra.el, which was a little tricker to define than anticipated, but it seems to work correctly.

I re-use the markers defined in folding-mode, so if you want to customize the markers just use `folding-add-to-marks-list'. folding-mode doesn't need to be enabled. I actually don't use folding, I just wanted to re-use the same methods.

Re: Evaluating python code blocks in python-mode

[Andreas Röhler]

Am 25.06.2015 um 12:25 schrieb Yuri D'Elia:
> On 06/21/2015 08:52 PM, Andreas Röhler wrote:
>> It might be an option to use existing markers from folding-mode. OTOH
>> don't see a command there how to copy the section. Do you?
> So this is what I came up with and I currently use:
>
> http://www.thregr.org/~wavexx/tmp/python-mode-extra.el
>
> Sorry for not inlining the file, but I think it's more readable.
>
> The idea is that "C-c C-c" sends the current fold you're in. If you're in a nested fold, subfolds are send too. If you're not in a fold, the buffer (or narrowed region) is sent instead.
>
> As an added bonus, I wanted to define a generic block delimiter, which is # --- in my case. `python-shell-send-fold-or-section' will send the current fold if there's any, up to the closest block delimiter (if any).
>
> So for example, in a file like:
>
>   print "1"
>   # ---
>   print "2"
>
> "C-c C-c" on either print statement will be limited to that statement. Block delimiters can be nested inside folds, and work as you'd expect: if you're in the same fold as a delimiter, the delimiter is used. If you're upwards, the delimiter is ignored.
>
> This makes it very easy to delimit "computation blocks", as you'd normally do in an interactive notebook, without the burden to define folds for each.
>
> To top that, the region being sent is added with a volatile highlight (volatile-highlights.el is required) so you see what's being evaluated.
>
> I love it.
>
> It works in Fabián's python-mode, but should work with minimal changes with other modes too.
>
> I actually redefine some bindings for python-mode to behave more like ESS (which IMHO has a better layout for interactive evaluation), resulting in something like:
>
> (add-hook 'python-mode-hook
> 	  (lambda ()
> 	    (local-set-key (kbd "C-c C-j") 'python-shell-send-line)
> 	    (local-set-key (kbd "C-c C-n") 'python-shell-send-line and-step)
> 	    (local-set-key (kbd "C-c C-f") 'python-shell-send-defun)
> 	    (local-set-key (kbd "C-c C-b") 'python-shell-send-buffer)
> 	    (local-set-key (kbd "C-c C-c") 'python-shell-send-fold-or-section)))
>
> `python-shell-send-fold-or-section' is what is defined in python-mode-extra.el, which was a little tricker to define than anticipated, but it seems to work correctly.
>
> I re-use the markers defined in folding-mode, so if you want to customize the markers just use `folding-add-to-marks-list'. folding-mode doesn't need to be enabled. I actually don't use folding, I just wanted to re-use the same methods.
>

Thanks for the inspiration. The feature made it into python-mode.el 
meanwhile

https://bugs.launchpad.net/python-mode/+bug/1467217

Kept it independent from folder-mode, as python-mode.el doesn't require 
any extern stuff for now.

BTW see in your file unknown "vhl/add-range"

Cheers,

Andreas

Re: Evaluating python code blocks in python-mode

[Yuri D'Elia]

On 06/25/2015 01:21 PM, Andreas Röhler wrote:
> BTW see in your file unknown "vhl/add-range"

Yes, this is from 'volatile-highlights.el' (from melpa).