From: Stefan Monnier <monnier@iro.umontreal.ca>
To: Stephen Leake <stephen_leake@member.fsf.org>
Cc: emacs-devel <emacs-devel@gnu.org>
Subject: Re: access to parser stack in SMIE
Date: Sat, 06 Oct 2012 08:37:40 -0400 [thread overview]
Message-ID: <jwvipan23ok.fsf-monnier+emacs@gnu.org> (raw)
In-Reply-To: <85pq4wgrho.fsf@member.fsf.org> (Stephen Leake's message of "Sat, 06 Oct 2012 00:21:39 -0400")
Your problem is one I also bumped into for the Modula-2 and Pascal modes
(can't remember how I "solved" them nor to what extent the "solution"
works).
> When all the tokens are properly disambiguated, SMIE can traverse
> correctly from package "begin" to "package". But we can't do that while
> disambiguating "begin"; that's circular.
Actually, in some cases, it can be made to work: to disambiguate "begin"
setup a loop that calls smie-backward-sexp repeatedly (starting from the
position just before the "begin", of course) checking after each call
whether the result lets us decide which of the two begins we're
dealing with.
Depending on the particular language's grammar this may not work because
"checking whether the result lets us decide" may not be possible.
But the recursion does terminate at least.
> The solution I found is to deliberately call `smie-forward-sexp'
> starting at the beginning of the buffer.
Right: the only way the parsing-stack can be used reliably is if we
always parse from the beginning.
> It might also make sense to incorporate the refined keyword cache
> mechanism into smie.
Right, if we want to make the stack visible, then we also need to
implement the cache.
Note that such a "forward full-parse with cache" approach has several
downsides:
- potential performance impact on long buffers.
- risk of the cache going out of sync.
- parse errors far away in an unrelated (earlier) part of the buffer
can prevent proper local indentation. Parse errors can occur for lots
of reasons (e.g. temporarily incorrect code, incomplete parser, or
use of "exotic" language extension, use of a preprocessor, ...).
That doesn't mean it's not workable, but those downsides had better come
with some significant upside. One significant upside is that by only
parsing forward we can use any other parsing technology, such as LALR,
GLR, PEG, ...
I.e. I think it's an interesting direction but it would be another package.
Stefan
next prev parent reply other threads:[~2012-10-06 12:37 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-10-06 4:21 access to parser stack in SMIE Stephen Leake
2012-10-06 12:37 ` Stefan Monnier [this message]
2012-10-06 18:55 ` Stephen Leake
2012-10-07 19:04 ` Stefan Monnier
2012-10-07 23:18 ` Stephen Leake
2012-10-08 1:00 ` Stefan Monnier
2012-10-08 1:28 ` Stephen Leake
2012-10-08 22:58 ` Stephen Leake
2012-10-09 2:26 ` Stefan Monnier
2012-10-09 3:29 ` Stephen Leake
2012-10-09 4:20 ` Stefan Monnier
2012-10-09 11:23 ` Stephen Leake
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=jwvipan23ok.fsf-monnier+emacs@gnu.org \
--to=monnier@iro.umontreal.ca \
--cc=emacs-devel@gnu.org \
--cc=stephen_leake@member.fsf.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.