Re: Handling extensions of programming languages

[haj@posteo.de (Harald Jörg)]

Matt Armstrong <matt@rfc20.org> writes:

> I'm not an expert in this topic it pertains to Emacs itself, but I've
> always editor and development tools interesting and so have paid
> attention to these issues over the years.

Thanks for sharing your insights!

> [...]
> Very good Emacs support for languages with flexible syntax, which have a
> high level of faithfulness to the language, or even "perfect"
> faithfulness, all seem to rely on tools native to the language and
> external to Emacs, usually by way of some sort of external server.
> [...] The common theme seems to be using the interpreter/compiler
> itself to parse, without relying on the editor to understand the code
> deeply.

This is fine.  For Perl, this has some limitations since you actually
need to run parts of the code to find out whether it compiles (or, more
precise, whether it can be interpreted correctly).  This might be
undesired, e.g. for security reasons with "unknown" code.

> For a different approach, you have examples of complete or nearly
> complete parsers written in Emacs Lisp.
> [...]
> The drawback here is that, by design, any syntax extensions and
> local mini-DSLs, etc., must also have parsers written in Emacs Lisp.

Exactly!  "How hard can that be?" -- Damian Conway, in a presentation
which shows, among other tricks, a ~2000-line Perl regular expression
which matches (not actually parses) Perl code.

I *guess* that Emacs Lisp is well suited for a pragmatic/heuristic
approach, and I want to give it a try.

> (info "(ccmode)Custom Macros") is an example of how cc-mode supports a
> limited form of syntax extension.

Many thanks!  This is the sort of pointers I'm after.  I'll take a look
how this is implemented.

> I think most modes in Emacs Lisp take a pragmatic approach, using
> heuristics that get the job done most of the time without being too
> computationally expensive.  The SMIE package is a generalization of this
> idea, see (info "(elisp)SMIE").

> I am not aware of anything like SMIE that allows for languages
> extensions to be "plugged in" in a general way.

Well, I have my doubts that Perl is a good candidate for SMIE, and
trying to use SMIE in CPerl mode would be a major rewrite anyway.  I
guess the Emacs Losp basics (font-lock-add-keyword, hooks) will have to
do the job.

-- 
Cheers, and again thanks for your time,
haj