From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Marcin Borkowski Newsgroups: gmane.emacs.devel Subject: Re: forward-sexp when on a floating point number Date: Wed, 20 Jan 2016 23:15:53 +0100 Message-ID: <87powv6go6.fsf@mbork.pl> References: <87y4bvf59p.fsf@gmail.com> <871t9e8usz.fsf@mbork.pl> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1453328199 2861 80.91.229.3 (20 Jan 2016 22:16:39 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 20 Jan 2016 22:16:39 +0000 (UTC) Cc: emacs-devel@gnu.org To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed Jan 20 23:16:29 2016 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1aM13F-0002Dz-H1 for ged-emacs-devel@m.gmane.org; Wed, 20 Jan 2016 23:16:29 +0100 Original-Received: from localhost ([::1]:45268 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aM13E-0002Ai-SR for ged-emacs-devel@m.gmane.org; Wed, 20 Jan 2016 17:16:28 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:34054) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aM12w-00027h-55 for emacs-devel@gnu.org; Wed, 20 Jan 2016 17:16:11 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aM12r-0001Id-Qq for emacs-devel@gnu.org; Wed, 20 Jan 2016 17:16:10 -0500 Original-Received: from mail.mojserwer.eu ([2a01:5e00:2:52::8]:33938) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aM12r-0001IG-It for emacs-devel@gnu.org; Wed, 20 Jan 2016 17:16:05 -0500 Original-Received: from localhost (localhost [127.0.0.1]) by mail.mojserwer.eu (Postfix) with ESMTP id 308508F2020; Wed, 20 Jan 2016 23:16:04 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at mail.mojserwer.eu Original-Received: from mail.mojserwer.eu ([127.0.0.1]) by localhost (mail.mojserwer.eu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ear7hSkMTtZR; Wed, 20 Jan 2016 23:15:54 +0100 (CET) Original-Received: from localhost (unknown [109.232.24.28]) by mail.mojserwer.eu (Postfix) with ESMTPSA id 355D28F201F; Wed, 20 Jan 2016 23:15:54 +0100 (CET) User-agent: mu4e 0.9.13; emacs 25.1.50.1 In-reply-to: X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2a01:5e00:2:52::8 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:198465 Archived-At: On 2016-01-18, at 22:34, Stefan Monnier wrote: >> FWIW, I did something like this for (La)TeX here: >> https://github.com/mbork/tex-plus.el (note: this is still WiP, and >> contains more than just tokenizing TeX). Either TeX token syntax is >> weird and/or difficult, either I really suck at writing lexers/parsers, >> or this is indeed a (potentially) difficult problem. (Probably all >> three.) > > Don't know about the middle one, but the other two indeed apply. In the > general case, the esiest way to solve this problem is probably to go > back to a safe earlier state and then lex forward from there. Yes, in case of TeX, it is relatively easy (assuming the user did not play around with catcodes - in such a case, /anything/ can happen (see e.g. https://tug.org/TUGboat/tb19-4/tb61carl.pdf - try it!). > A "safe earlier state" could be "right after a character which can only > appear at the end of a token" [ of course, there's no guarantee that such > a character exists ]. In TeX (under normal catcode regime), it's just like that: newline. > Also "going back and then lex forward" implies a potential serious > performance problem, so it would require some form of caching. Exactly, and I wanted to avoid that. Frankly, I had no idea how to do this anyway; now I'm a bit wiser, and I guess I could run an idle timer to update the cache, and have a global variable for keeping the state (cache synchronized or not), set to "not in sync" after any text-changing command. Of course, any command that actually needs a cache would have to recreate it, too. Would that be a good plan? > Stefan Best, -- Marcin Borkowski http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski Faculty of Mathematics and Computer Science Adam Mickiewicz University