python.el: improve sexp-based navigation or make it optional?

python.el: improve sexp-based navigation or make it optional?

[immerrr again]

Hi everyone,

I'm a frequent user of python.el, and I have a qualm about its "smart"
sexp navigation. As a long-time Emacs user, I have come to certain
expectations about how Emacs behaves, and I cannot wrap my hand around
some of the idiosyncrasies there.

Below you can find a couple of examples. Where forward-sexp from 1
takes point to 2, and backward-sexp from 2 takes point back to 1.

A.
<1>x = 'foo'<2>

B.
<1>class Test:
    X = 'foo'
    Y = 'bar'<2>

C.
<1>def calc_foo(...):
    ...<X lines of code>
    return foo<2>

It seems logical to be able to skip an entire statement, or an entire
defun/defclass, but what if I want to go select just the last literal
in a statement? The problem here is that there is no way to
unambiguously tell python-mode what symbol/literal/block do I want to
skip. To clarify, let's pretend Python had parentheses around its
syntactical blocks:

<1>(<2>def<2'> my_func(x):
    <3>(if (x == 'foo'):
        <4>(Y = <5>'bar'<5'>)<4'>)<3'>)<1'>

In this case it's easy to predict that forward/backward-sexp should
establish the following pairs: 1 with 1', 2 with 2', 3 with 3' and so
on. But without the parentheses, there is no way to distinguish
between 1 and 2, or between 1', 3', 4', and 5'. Python.el essentially
tries to make a guess, and curiously, this explicitly goes against one
of the principles listed in the Zen of Python,

> In the face of ambiguity, refuse the temptation to guess.

Some kind of a visual-sexp mode for this type of navigation would
reduce the uncertainty. A mode that would augment the code with some
kind of an overlay, showing parentheses around each sexp composed of
more than 1 symbol/literal, and provide visual guidance as to where
will point end up after a sexp-based navigation command. However, IIRC
overlays don't scale well, and it might require implementing yet
another parser or hacking all the structural information to the
pre-existing one, which is a quite tedious task.

In absence of such mode, I wonder if it would be a welcome addition to
introduce a configuration parameter that at least disables the smart
navigation, making sexp skip one symbol or explicitly
parenthesized/bracketed expression at a time.

What do you think?

Re: python.el: improve sexp-based navigation or make it optional?

[Stefan Kangas]

Hi,

immerrr again <immerrr@gmail.com> writes:

>                          I wonder if it would be a welcome addition to
> introduce a configuration parameter that at least disables the smart
> navigation, making sexp skip one symbol or explicitly
> parenthesized/bracketed expression at a time.

I don't see any reason not to welcome such a contribution.

I happen to think the sexp-navigation in python-mode does a decent job
(read: it never bothered me), but why not give an option to users who
find it too unpredictable.

Best regards,
Stefan Kangas

Re: python.el: improve sexp-based navigation or make it optional?

[Andreas Röhler]

On 16.05.20 15:16, immerrr again wrote:
>
> In absence of such mode, I wonder if it would be a welcome addition to
> introduce a configuration parameter that at least disables the smart
> navigation, making sexp skip one symbol or explicitly
> parenthesized/bracketed expression at a time.

Maybe that does it?: (from python.el)

If you prefer `cc-mode'-like
;; `forward-sexp' movement, setting `forward-sexp-function' to nil is
;; enough, You can do that using the `python-mode-hook':

;; (add-hook 'python-mode-hook
;;           (lambda () (setq forward-sexp-function nil)))

Re: python.el: improve sexp-based navigation or make it optional?

[immerrr again]

On Sat, May 16, 2020 at 9:04 PM Andreas Röhler
<andreas.roehler@online.de> wrote:
>
>
> On 16.05.20 15:16, immerrr again wrote:
> >
> > In absence of such mode, I wonder if it would be a welcome addition to
> > introduce a configuration parameter that at least disables the smart
> > navigation, making sexp skip one symbol or explicitly
> > parenthesized/bracketed expression at a time.
>
> Maybe that does it?: (from python.el)
>
> If you prefer `cc-mode'-like
> ;; `forward-sexp' movement, setting `forward-sexp-function' to nil is
> ;; enough, You can do that using the `python-mode-hook':
>
> ;; (add-hook 'python-mode-hook
> ;;           (lambda () (setq forward-sexp-function nil)))
>

Yep, it works, thanks! I have forgotten about that variable.

I wonder if it would be hard to implement the sexp visualisation mode
that would show "virtual parentheses".

Overlays perhaps would be  a tad slow, if there was a parenthesis pair
shown for every sexp out there. Maybe I could limit it only to the
sexps that immediately surround point...

Properties would be much better performance wise, but looks like
"display" characters are added as a block, and one cannot enter a
sequence of virtual parentheses with point. I could try adding spaces
or whatnot, but then I'd have to be extra careful that they are
cleaned up before saving or copying buffer contents.

Any recommendations on what could be the starting point here?

Re: python.el: improve sexp-based navigation or make it optional?

[Stefan Monnier]

> <1>class Test:
>     X = 'foo'
>     Y = 'bar'<2>

SMIE navigation faces similar problems and solves them differently: from
(1) is would jump to (2) but from (2) it would jump back to just before
'bar' instead of jumping back to (1).

IOW it jumps over "the nearest/smallest" sub-expression.


        Stefan

Re: python.el: improve sexp-based navigation or make it optional?

[Stefan Kangas]

immerrr again <immerrr@gmail.com> writes:

> In absence of such mode, I wonder if it would be a welcome addition to
> introduce a configuration parameter that at least disables the smart
> navigation, making sexp skip one symbol or explicitly
> parenthesized/bracketed expression at a time.
>
> What do you think?

Andreas Röhler filed this bug:

    Fwd: Re: python.el: improve sexp-based navigation or make it optional?
    https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41361

It would be most welcome if you worked on it.

Best regards,
Stefan Kangas