Re: APL mode

[Stefan Monnier <monnier@IRO.UMontreal.CA>]

>>> What does sexp mean for non-lisp languages like C etc?
>>> (Needed for functions like forward-sexp)
>> It means "a subtree in the abstract syntax tree".
> In principle, that's fine. However in practice there are things like
> a. comments which do not exist in the AST

comments should normally be skipped by forward-sexp.

> b. preprocessor commands that appear and disappear before ASTs are on the
> scene

Indeed, there are cases where the source code does not correspond to
a tree (and not even a DAG either).  In that cases, the expected
behavior of forward-sexp is undefined (i.e. total freedom).

> c. (most important) emacs doesnt really do a full-scale context free
> grammar analysis does it?

Not sure what you man by "Emacs" here: Emacs just runs the code provided
by the major mode author, so it's the major mode's author's
responsibility to make sure forward-sexp works as it should.

> So I guess I am asking: Emacs uses regular exps to fudge a semblance of
> context free structure. How does it do this?

There's `forward-sexp-function' which lets you write your parser by
hand.

Else, there's a very simplistic builtin parser, which doesn't know about
infix operators and only know how to skip "normal" comments, how to
recognize "normal" strings, and how to skip parentheses (and related
pairs, with restrictions such as: the opening and closing markers are
single chars).  The "grammar" for this parser is given in the form of
what we call a "syntax-table".

In between, there's SMIE which does precedence parsing.  It's a very
weak grammar, but since we want to parse backward (for backward-sexp),
it's a good option since, contrary to LL/LR and friends it's fully
symmetric.  We use it for various different modes: Coq, SML, OCaml,
Prolog, sh, Ruby, Modula-2, Octave, css.


        Stefan "who doesn't know much about APL's syntax"