Re: forward-sexp when on a floating point number

[Marcin Borkowski <mbork@mbork.pl>]

On 2016-01-18, at 14:30, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> PS: Lexing via FSM is harder than I make it out to be, of course, since
> multi-char tokens introduce the question of how to figure out with which
> state to start lexing (e.g. if we start a command from the middle of
> a token), as well as how to "tokenize backward" ("single-char tokens"
> (like we have now) can trivially be parsed with the same FSM going
> forward and backward, but that's not true for the more general case).

FWIW, I did something like this for (La)TeX here:
https://github.com/mbork/tex-plus.el (note: this is still WiP, and
contains more than just tokenizing TeX).  Either TeX token syntax is
weird and/or difficult, either I really suck at writing lexers/parsers,
or this is indeed a (potentially) difficult problem.  (Probably all
three.)

>         Stefan

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University