bug#11974: 24.1.50; python-nav-forward-sexp-function doesn't skip over strings

[Stefan Monnier <monnier@IRO.UMontreal.CA>]

> (let* ((bos (python-info-ppss-context 'string)))
>   (goto-char bos)
>   (forward-sexp 1)
>   (narrow-to-region bos (point)))

> This is broken now since forward-sexp goes to the end of the block, far
> past the end of the string (or not moving at all).  If you have any
> undindented code at the end of your buffer, then forward-sexp (and up-list)
> will refuse to move past it e.g.

There's a similar problem in SMIE's implementation of forward-sexp.
Basically, the issue is that there are several "sexp"s that start at
point and forward-sexp has to choose which one to skip over.
Every choice is valid, but since there's no way for the user to say
which choice she wants, any choice is bound to disappoint and/or
surprise the user sometimes.

That doesn't mean that it's a hopeless problem and that we shouldn't try
to improve the behavior, tho: in practice (maybe just for historical
reasons), I find that I generally prefer "the smallest sexp", especially
since I can repeat forward-sexp if it didn't skip enough.

I.e. Python's forward-sexp should only try to skip over
indentation-delimited blocks when it has a clear indication that only
skipping the current line/statement would be wrong, or that there's an
easy/intuitive way for the user to get the other behavior (e.g. by
starting the forward-sexp from elsewhere).

For example, with SMIE if I have
   a + b * c
and I'm before "b", forward-sexp will only skip over "b", but if I move
to just before the "+", then forward-sexp will skip to after "c".
I'm not sure if Python's indentation-delimited blocks can find a similar
"convention" for the user to express her intention.

> What I would expect form a "perfect" python forward-sexp is to skip over
> a block if and only if you are at the very beginning of the block.

You mean that forward-sexp would generally skip over just a statement,
except if it's the first statement of a block in which case it skips the
whole block?  That sounds a bit arbitrary, making it difficult for the
user to skip over just the first statement.

[ Bear in mind, I never write Python.  ]
How 'bout

  def foo():
      toto
      bar

if I'm before `toto' on the same line, forward-sexp should skip over
"toto", but if I'm right after the ":" (and before the \n) then
forward-sexp should skip to after "bar".


        Stefan