Re: parsing (was tokenizing)

[Stephen Leake <stephen_leake@stephe-leake.org>]

Eric Ludlam <ericludlam@gmail.com> writes:

>  On 09/20/2014 12:40 PM, Vladimir Kazanov wrote:
>>> Tokenizing the whole buffer after any change is easily fast enough (on
>>> modern hardware), even on a 7000 line buffer. Semantic parsing gets a
>>> lot slower.
>>>
>> This is what I do right now in my prototype of a smarter Python mode.
>> The tokenizing process itself is usually fast enough. But parsing is
>> more complicated, and may take some time to rebuild the parse tree.
>> Incremental approach is a natural step here.
>>
>
> I caught only the tail of this thread, so I apologize if I refer to
> the incorrect thing.
>
> A year ago or so we were talking about ada-mode, a modified parser,
> and how it might integrate with CEDET/Semantic on the CEDET mailing
> list.  Is it still 'wisi', a different flavor of 'wisent' ?

Yes.

> If calls into your parser are being handled by parts of CEDET/Semantic
> for creating tags, then there is an incremental parser that you can
> enable that works with two of the other parser types included.  

No, I'm not using the CEDET front end UI. Partly because that
incremental parser does not handle the Ada language (or I did not want
to put in the effort to modify my grammar to make it work).

The wisi parser is generalized LALR, and it does not lend itself to
incremental parsing.

I also didn't need the full flexibility of the Semantic lexer; the wisi
lexer relies solely on the elisp syntax properties, and is faster.

On the other hand, the wisi parser is unacceptably slow for 7000 line
Ada files (and I have customers complaining about files 10 times that
size (or they are using really slow machines)), so I'm looking into
ways to use compiled Ada code instead of elisp to run the parser.

> I'll be happy to help
> you figure out how to make your parser work in that framework.

Once I get the speed acceptable, I intend to look again at the CEDET UI,
and try to use as much as possible with Ada mode. In particular, smart
completion would be a nice feature.

-- 
-- Stephe