From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Lennart Borgman Newsgroups: gmane.emacs.devel Subject: Re: "Font-lock is limited to text matching" is a myth Date: Tue, 11 Aug 2009 21:48:09 +0200 Message-ID: References: <7b501d5c0908091634ndfba631vd9db6502db301097@mail.gmail.com> <200908101335.24002.danc@merrillprint.com> <87my67s8mr.fsf@randomsample.de> <1249942011.29022.15.camel@projectile.siege-engine.com> <1249955428.29022.186.camel@projectile.siege-engine.com> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Trace: ger.gmane.org 1250020262 1545 80.91.229.12 (11 Aug 2009 19:51:02 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 11 Aug 2009 19:51:02 +0000 (UTC) Cc: Daniel Colascione , David Engster , Daniel Colascione , emacs-devel@gnu.org, Steve Yegge , Stefan Monnier , Deniz Dogan , Leo , Miles Bader To: eric@siege-engine.com Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Tue Aug 11 21:50:53 2009 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1MaxMz-0006V8-AA for ged-emacs-devel@m.gmane.org; Tue, 11 Aug 2009 21:50:53 +0200 Original-Received: from localhost ([127.0.0.1]:48118 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1MaxMx-0003Rc-OU for ged-emacs-devel@m.gmane.org; Tue, 11 Aug 2009 15:50:51 -0400 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1MaxKS-0002Xn-W4 for emacs-devel@gnu.org; Tue, 11 Aug 2009 15:48:17 -0400 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1MaxKO-0002Vu-Id for emacs-devel@gnu.org; Tue, 11 Aug 2009 15:48:16 -0400 Original-Received: from [199.232.76.173] (port=59897 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1MaxKO-0002Vo-6D for emacs-devel@gnu.org; Tue, 11 Aug 2009 15:48:12 -0400 Original-Received: from an-out-0708.google.com ([209.85.132.251]:33873) by monty-python.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1MaxKM-000524-2y; Tue, 11 Aug 2009 15:48:10 -0400 Original-Received: by an-out-0708.google.com with SMTP id b6so1549696ana.21 for ; Tue, 11 Aug 2009 12:48:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=uudh8MsnldxhT0t2UnQqNb/mZgGqiP4d1vNAZ34cCLU=; b=R2XsrlM99RCDesnV9HA8XkFCYaU6U6z0HcOqjl8bvaouh6iW5s6Wp3Ztobz2WyxYA5 rypEmRJDnVDSiAyWNra5wdGCCgYjbwbY7ek5uqwebALie3T+zOeRjro3iOK/wcM9YQQt vW6g0HotfgkM56se9C8uQh4PxyIN4Vz7ldoXA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=NquxQRvLIJOeMOk6oxnyTaOpVM6U6guZG6G9dWIABqjDlz8Ty3juCy3xZcNwHLBRce /xUCHz8dVQZ7Xg0CYq3HrYxLKXluE9WvZ5AtG/E4RRCJ8CDDHySdbBbgpBBX731+9L1F iAIr2KyIw1aQ7JIVrzzQmEQ8jK+mwVTwPUxj0= Original-Received: by 10.100.96.4 with SMTP id t4mr5842160anb.170.1250020089435; Tue, 11 Aug 2009 12:48:09 -0700 (PDT) In-Reply-To: <1249955428.29022.186.camel@projectile.siege-engine.com> X-detected-operating-system: by monty-python.gnu.org: GNU/Linux 2.6 (newer, 2) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:114096 Archived-At: On Tue, Aug 11, 2009 at 3:50 AM, Eric M. Ludlam wrot= e: > On Tue, 2009-08-11 at 00:19 +0200, Lennart Borgman wrote: >> On Tue, Aug 11, 2009 at 12:06 AM, Eric M. Ludlam = wrote: >> >> Hi Eric, >> >> > =C2=A0The concept of using the Semantic parser/generator framework for >> > improving font-locking accuracy has come up many times. =C2=A0No-one t= o my >> > knowledge has attempted to mix the two. >> >> >> Maybe that can easier be done if Semantic parser use >> font-lock/JIT-lock timers and marking to keep track of what need to be >> reparsed? (It is just a wild idea perhaps.) > > I'm not certain of how the font/jit lock works. =C2=A0Semantic works by > tracking edits (after-change-functions) and then on it's own timer, it > coalesces the changes into parsable units. =C2=A0It then reparses those > units. This is precisely what font/jit lock does too. More precisely this is done by setting the text property 'fontified to nil (if I remember correctly). JIT-lock will then pick up this and refontify those parts. > Font lock can refontify based on fairly small subsections of a buffer, > such as a single code line, or a comment section. =C2=A0Semantic's > subsections are the size of functions, variables, and datatypes (ie, the > tags it creates.) Font/jit lock may also need bigger sections. In the case of mumamo it does, for example. A change like adding a new major mode chunk boundary may make it necessary to refontify the whole rest of the buffer. >> So you do a first pass with coarse parsing and then you look in the >> sub-sections for details? Is this strictly necessary? I guess you are >> looking for top level definitions in the first pass? >> >> Could that pass have its own state and continue upon demand (when an >> item is not recognized) or is such a logic impossible? > > It could, but I have not done so. =C2=A0Tagging information is not genera= lly > needed right away, so just waiting for the user to either ask for it, or > sit idle for a while works pretty well. =C2=A0The overhead of such an > incremental parser isn't really needed. The benefits coudl be that Emacs behaved much more smoothly. Both because the background parsing could be interrupted and restarted easier and because it could be easier to handle precomutation (perhaps). The overhead might perhaps be rather small if you could say "please give me a JIT framework that computes these values: ... (list of functions)... and runs according to scheme ...(JIT computation style)..."? Just loose ideas of course, but... > The needs between the tagging parser and the font-lock parser are > different. =C2=A0Font lock needs to colorize arbitrary blocks of text, an= d a > tagging parser needs to parse everything, but only needs the data > periodically. A very good point that maybe can be translated to strucures in a JIT framew= ork. > Converting a tagging parser to a colorizing parser would be challenging > because of these different uses. They might use similar JIT frameworks but not the same timers (but maybe cooperating dito). >> > =C2=A0I would imagine that the parsing engine in Semantic, if it is de= emed >> > critical by the maintainers, will get faster if key components are >> > integrated into the C code. >> >> Is that part stable? > > Yes. =C2=A0Not much is going on there. Stefan seems positive to getting this kind of things into C code. To me it seems very reasonable too. >> > =C2=A0Lastly, as David Engster stated, CEDET has decoration tools that >> > decorate entire tags in some way, such as putting a line on top of >> > functions. =C2=A0This is a separate decoration activity not related to= font >> > lock, and something font lock would not be able to do reliably. >> >> Why not if it asks the parser? > > Font lock runs long before the parser bothers trying to catch up. =C2=A0F= ont > lock would needs hooks for after the parser runs. > problems. Perhaps it can be handled like this: - The parser puts a tag on the parts where it wants special faces. - It also sets 'fontified to nil on those regions. This will tell JIT lock to refontify those parts. - Then font-lock keywords functions supplied by the parser can set the actual fontification when called by font/JIT-lock. This will avoid the problem that the special parser fontification gets thrown away when scrolling that Joakim mentioned. > While font lock and semantic share a need for a parsing infrastructure, > the where/when of the parsing is quite different. =C2=A0It is possible to > conceptually mix and match the parsers vs the schedulers. =C2=A0In practi= ce, > the two tools have their own lengthy histories that will make that > challenging. =C2=A0Before tackling such a project, it would be wise to ta= ke > multi-mode (or similar tool) into account. Yes. Thanks for the good explanations. > Eric >