Re: font-lock-syntactic-keywords obsolet?

[Alan Mackenzie <acm@muc.de>]

Hello, Noam.

On Sat, Jun 18, 2016 at 11:07:03AM -0400, Noam Postavsky wrote:
> On Sat, Jun 18, 2016 at 3:03 AM, Andreas Röhler
> <andreas.roehler@online.de> wrote:
> > Anyone who looks how syntax-propertize-function is introduced in source,
> > --given some decent understanding of Emacs Lisp-- will see the bunch of
> > issues. There is nothing to explain here.

> Ah, is this like The Emperor's New Clothes? Anyone who doesn't see the
> issues lacks a decent understand of Emacs Lisp? I'll play the kid who
> doesn't know any better. Here's the source where
> syntax-propertize-function is introduced:

I see the issues here, and I avoid syntax-propertize and syntax-ppss
wherever possible.

> ;;; Applying syntax-table properties where needed.

> (defvar syntax-propertize-function nil
>   ;; Rather than a -functions hook, this is a -function because it's easier
>   ;; to do a single scan than several scans: with multiple scans,
> [more commentary explaining issues with multiple scans that have been
> avoided...]

> Looks like a normal defvar to me.

Let's have that excerpt in full:

  ;; Rather than a -functions hook, this is a -function because it's easier
  ;; to do a single scan than several scans: with multiple scans, one cannot
  ;; assume that the text before point has been propertized, so syntax-ppss
  ;; gives unreliable results (and stores them in its cache to boot, so we'd
  ;; have to flush that cache between each function, and we couldn't use
  ;; syntax-ppss-flush-cache since that would not only flush the cache but also
  ;; reset syntax-propertize--done which should not be done in this case).

The issue is not caused by multiple scans.  CC Mode (in particular C++
Mode) does several scans to apply syntax-table text properties, simply
because that's a natural thing to do (there being several distinct
reasons for these properties being applied).  That way, it's easier to
debug, easier to understand, and less error prone.

It is not "easier to do a single scan".  It is simply that a single scan
is virtually forced if one uses the syntax-ppss/syntax-propertize
mechanism, because:
(i) parse-partial-sexp is very likely to be needed for calculating the
  syntax-table text properties.  Because only some s-t properties will
  have been applied, precise control of p-p-s is needed.  I think users
  of syntax-ppss hold that using parse-partial-sexp directly is a Bad
  Thing.
(ii) The only way syntax-propertize gets called is by calling
  syntax-ppss for some position below where one needs the properties
  applying.

Additionally, syntax-ppss hasn't conformed with its specification w.r.t.
narrowed buffers for a long time, if ever.  (See bug #22983.)

Additionally 2, when syntax-propertize-function is non-nil, all
syntax-table text properties beyond point are wiped out by a change,
causing the need to regenerate them.  This is not always necessary, and
might be expensive in run time (see, again, C++ Mode).

In summary, the syntax-ppss/syntax-propertize mechanism might be a good
choice for implementing syntax-table text properties, but it is not
necessarily so.

-- 
Alan Mackenzie (Nuremberg, Germany).