Semantics of font-lock-beginning-of-syntax-function

Semantics of font-lock-beginning-of-syntax-function

[Martin Stjernholm]

The docstring for font-lock-beginning-of-syntax-function says:

    Non-nil means use this function to move back outside of a
    syntactic block.  When called with no args it should leave point
    at the beginning of any enclosing syntactic block.

I'd like to know what - exactly - "syntactic block" means here. How is
this function used, and what assumptions does the code that uses it
have on the target position? A couple of guesses I've made more or
less out of the blue are:

o  The point should be outside any comment or string literal.
o  The point should be no later than the beginning of the current
   line.

The word "block" suggests that it also should be at the beginning of a
programmatic block, i.e. a paren sexp in C and similar languages. If
so, why does font-lock need that? Afaics it doesn't operate on the
structural level at all. Not knowing all the details, I thought it'd
be enough to go to any preceding position that's outside comments and
string literals.

If I interpret the second sentence correctly, it should move to (at
least) the beginning of the outermost enclosing block. When and why is
it necessary to move that far? Many times (e.g. in Java code) that's
almost as bad as the beginning of the buffer since there might very
well be a single class surrounding all the code in a file.

Re: Semantics of font-lock-beginning-of-syntax-function

[Stefan Monnier]

> The docstring for font-lock-beginning-of-syntax-function says:
> 
>     Non-nil means use this function to move back outside of a
>     syntactic block.  When called with no args it should leave point
>     at the beginning of any enclosing syntactic block.
> 
> I'd like to know what - exactly - "syntactic block" means here. How is
> this function used, and what assumptions does the code that uses it
> have on the target position? A couple of guesses I've made more or
> less out of the blue are:
> 
> o  The point should be outside any comment or string literal.
> o  The point should be no later than the beginning of the current
>    line.

That looks right, although I think it's usually (always?) called from
the beginning of a line, so the second constraint is just "no later than
point".

> The word "block" suggests that it also should be at the beginning of a
> programmatic block, i.e. a paren sexp in C and similar languages. If
> so, why does font-lock need that? Afaics it doesn't operate on the
> structural level at all. Not knowing all the details, I thought it'd
> be enough to go to any preceding position that's outside comments and
> string literals.

Note that the details have slightly changed in Emacs-21.4 when I added
syntax-ppss and made font-lock use it.  `font-lock' only cares about
comments and strings, but syntax-ppss returns the same info as
`parse-partial-sexp' and might be used by any other code.
Also font-lock-beginning-of-syntax-function might be ignored if syntax-ppss
feels like the result is wrong (e.g. if the `face' property says
`font-lock-comment-face') or if it expects that its own cache of parse data
will be quicker.  This last optimization was also present in font-lock
before, although it doesn't work exactly in the same way (the current
caching in syntax-ppss is much more aggressive).

> If I interpret the second sentence correctly, it should move to (at
> least) the beginning of the outermost enclosing block. When and why is
> it necessary to move that far? Many times (e.g. in Java code) that's
> almost as bad as the beginning of the buffer since there might very
> well be a single class surrounding all the code in a file.

`syntax-ppss' currently assumes that `syntax-begin-function' (which
defaults to the value of `font-lock-beginning-of-syntax-function')
returns a position POS such that (parse-partial-sexp (point-min) POS)
is equivalent to nil (i.e. such that
(equal (pps POS (+ POS N)) (pps (point-min) (+ POS N)) modulo minor
discrepancies.

It's OK for `syntax-begin-function' (font-lock-beginning-of-syntax-function
should be obsoleted) not to move to the very top-level as long as users
of syntax-ppss don't depend on it.

Note also that the caching done in syntax-ppss makes `syntax-begin-function'
much less important.


	Stefan

Re: Semantics of font-lock-beginning-of-syntax-function

[Thien-Thi Nguyen]

"Stefan Monnier" <monnier+gnu/emacs@RUM.cs.yale.edu> writes:

   It's OK for `syntax-begin-function' (font-lock-beginning-of-syntax-function
   should be obsoleted) not to move to the very top-level as long as users of
   syntax-ppss don't depend on it.

would you recommend something like `backward-up-list' for this?

thi

Re: Semantics of font-lock-beginning-of-syntax-function

[Stefan Monnier]

>    It's OK for `syntax-begin-function' (font-lock-beginning-of-syntax-function
>    should be obsoleted) not to move to the very top-level as long as users of
>    syntax-ppss don't depend on it.
> 
> would you recommend something like `backward-up-list' for this?

No.  `backward-up-list' and most other sexp functions blindly
assume that point is outside of any string/comment, so they're
generally not appropriate.


	Stefan