Parsing beyond sexps

Parsing beyond sexps

[Damien Cassou]

Hi,

I'm trying to write a general-purpose linting engine for Emacs Lisp. It
currently uses a generator which repeatedly calls `(read buffer)` and
yields the result. It also uses a generator to iterate over the sexps of
a sexp, recursively. Here is the code:

https://gitlab.com/lintel/lintel/blob/2c98a02b17e34140f576e0bf2a9a7c47920dc1ce/lintel.el#L283

This is not enough though as I want comments and line/column positions
in the yielded values. Is there anything I could use from Emacs core
that would make my life easier?

I thought about using `parse-partial-sexp` and `syntax-ppss` but don't
know which iteration strategy to use to get information on the whole
buffer.  I could also use `forward-sexp` and check for the presence of a
comment at point each time this function returns.

Any piece of advice would be appreciated.

Please keep me in CC when answering.

Thank you.

-- 
Damien Cassou

"Success is the ability to go from one failure to another without
losing enthusiasm." --Winston Churchill

Re: Parsing beyond sexps

[Mattias Engdegård]

19 jan. 2020 kl. 09.11 skrev Damien Cassou <damien@cassou.me>:

> This is not enough though as I want comments and line/column positions
> in the yielded values. Is there anything I could use from Emacs core
> that would make my life easier?

Nothing in Emacs core that I know of, but you could have a look at how the 'relint' package does it: keep track of the position before each top-level form, and then maintain a path of list indices taken as it descends into each form. Example:

(defun select-weapon (opponent)
  "A Matter of Life and Death."
  (if (fencing-master-p opponent)
      'duelling-pistols
    'rapier))

The path to duelling-pistols would be (1 2 4), since it is built up backwards as a stack during descent: first take element 4 of the top-level form to the 'if'-expression, then element 2 of that form to the (quote duelling-pistols), and finally element 1 to get past the quote. Maintaining the path during traversal turns out to be fairly cheap.

The tuple <top-level-position, path> can then be transformed into <line, column> (or just a buffer position) when required for diagnostics; see 'relint--pos-line-col-from-toplevel-pos-path'. It's somewhat slow but not time-critical.