Re: Mysterious fontification/C++ context issue - Patch for beginning-of-defun-raw.

[Stefan Monnier <monnier@iro.umontreal.ca>]

>> >> I still haven't seen any explanation for why this code needs to be
>> >> in beginning-of-defun-raw.
 
> This isn't true, for any normal value of "needs".  I think it's more
> accurate to say that you don't accept that the explanation that has been
> given is a strong enough justification for the extra code in b-d-d-raw,
> which is fair enough.

By "need" I meant "technical need" rather than a philosophical need to make
something work according to some interpretation of the doc.
So you seem to agree that it's not "needed" in the technical sense.
Good.
 
> No, it's deeper than that.  It's to make the function match its
> documentation (see "Left Margin Paren" in the Emacs manual).  It's to
> make things work consistently when opic0ids is nil.  Everybody in this
> discussion whose native language is English agrees on this point.  ;-)

I think it's a bit misleading: your original motivation was to get your code
working in cc-mode, not to fit the code to the doc.

> (i) The meaning of "beginning-of-defun" in a source file is, and always
> has been, a paren at top level.  This proved too slow in early versions
> of Emacs, and so the "column 0" heuristic was adopted.  This has been
> documented in the Emacs manual since time immemorial; see page "Left
> Margin Paren" in the Emacs 21 manual (i.e. before I made any changes to
> it.  ;-)

Other than the last historical paragraph, the emacs21 node only talks about
the fact that open-parens-in-column-0 should only be used when they mean
a beginning-of-defun.
I.e. open-paren-in-column-0-is-defun-start means "if you see a paren in
column-0, it's a defun" and not "all defuns should start at column-0".
So setting it to nil means "a paren in column-0 may be something else than
a defun" rather than "defuns may start in any column".

Furthermore beginning-of-defun is a command bound to C-M-a and thus
available in every major mode, so its meaning can't be nearly as limited as
you make it out to be since in many programming languages there is no
open-paren at the beginning of a defun.

> (iii) This column 0 heuristic is just that - a heuristic, not a
> definition of b-o-d.

Agreed.  Which is why I'd agree that b-o-d could be made to check
syntax-ppss *after finding a defun-like thingy* to check that it's not
inside a comment or string.

If you look at the current definition of b-o-d, you shoul dbe shocked at the
lack of consistency in its behavior: if defun-prompt-regexp is nil and
open-paren-in-column-0-is-defun-start is nil (i.e. in the new case you
added), then it works at the level of the parse tree, whereas for every
other setting it only looks at the level of the lex'd tokens (without paying
attention to any parsing state).  There's clearly something wrong there.

> (iv) The variable open-paren-in-column-0-is-defun-start was introduced
> in the murky past, and there doesn't seem to be a record of why.  I
> can't find any discussion of it in the emacs-devel archives.  However,
> it's doc-string states: "Non-nil means an open paren in column 0 denotes
> the start of a defun".

And the contrapositive of "Non-nil means an open paren in column 0 denotes
the start of a defun" is "nil means that an open paren in column 0 does not
necessarily denote the start of a defun".

> Otherwise put, "non-nil means that the column 0 heuristic applies".
> (v) By pure logic, this MUST extend to "nil means the column 0 heuristic
> does NOT apply".  Otherwise the variable would be meaningless.

Applying pure logic after adding a fuzzy (natural language) interpretation
doesn't seem very appealing to me.

> Without that code, when opic0ids is nil, beginning-of-defun doesn't do
> the right thing: it finds a column 0 parenthesis rather than one at top
> level.  That has been the state up till now, and is _very_ confusing for
> Elisp hackers.

Then let's fix it right for all uses (including those using
defun-prompt-regexp and beginning-of-defun-function) by checking syntax-ppss
after the fact and retrying if we find out we've been confused by some
defun-looking comment or string.


        Stefan