From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: David Koppelman Newsgroups: gmane.emacs.bugs Subject: bug#192: regexp does not work as documented Date: Sun, 11 May 2008 14:09:31 -0500 Message-ID: References: <87k5i8ukq8.fsf@stupidchicken.com> <200805061335.11379.bruno@clisp.org> <48204B3D.6000500@gmx.at> <4826A303.3030002@gmx.at> <87abiwoqzd.fsf@stupidchicken.com> Reply-To: David Koppelman , 192@emacsbugs.donarmstrong.com NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1210544332 22612 80.91.229.12 (11 May 2008 22:18:52 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 11 May 2008 22:18:52 +0000 (UTC) Cc: Chong Yidong , 192@emacsbugs.donarmstrong.com, Bruno Haible , emacs-devel@gnu.org To: Stefan Monnier Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Mon May 12 00:19:27 2008 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1JvJt4-0007jq-GT for geb-bug-gnu-emacs@m.gmane.org; Mon, 12 May 2008 00:19:22 +0200 Original-Received: from localhost ([127.0.0.1]:55531 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1JvJsL-00055v-ON for geb-bug-gnu-emacs@m.gmane.org; Sun, 11 May 2008 18:18:37 -0400 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1JvHGp-0003X3-2f for bug-gnu-emacs@gnu.org; Sun, 11 May 2008 15:31:43 -0400 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1JvHGo-0003Wa-9X for bug-gnu-emacs@gnu.org; Sun, 11 May 2008 15:31:42 -0400 Original-Received: from [199.232.76.173] (port=34373 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1JvHGo-0003WX-3V for bug-gnu-emacs@gnu.org; Sun, 11 May 2008 15:31:42 -0400 Original-Received: from rzlab.ucr.edu ([138.23.92.77]:52331) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.60) (envelope-from ) id 1JvHGn-0004jm-OG for bug-gnu-emacs@gnu.org; Sun, 11 May 2008 15:31:42 -0400 Original-Received: from rzlab.ucr.edu (rzlab.ucr.edu [127.0.0.1]) by rzlab.ucr.edu (8.13.8/8.13.8/Debian-3) with ESMTP id m4BJVdAP027809; Sun, 11 May 2008 12:31:39 -0700 Original-Received: (from debbugs@localhost) by rzlab.ucr.edu (8.13.8/8.13.8/Submit) id m4BJF3jL025348; Sun, 11 May 2008 12:15:03 -0700 X-Loop: don@donarmstrong.com Resent-From: David Koppelman Resent-To: bug-submit-list@donarmstrong.com Resent-CC: Emacs Bugs Resent-Date: Sun, 11 May 2008 19:15:03 +0000 Resent-Message-ID: Resent-Sender: don@donarmstrong.com X-Emacs-PR-Message: report 192 X-Emacs-PR-Package: emacs X-Emacs-PR-Keywords: Original-Received: via spool by 192-submit@emacsbugs.donarmstrong.com id=B192.121053297724704 (code B ref 192); Sun, 11 May 2008 19:15:03 +0000 Original-Received: (at 192) by emacsbugs.donarmstrong.com; 11 May 2008 19:09:37 +0000 Original-Received: from ecelsrv1.ece.lsu.edu (ecelsrv1.ece.lsu.edu [130.39.223.98]) by rzlab.ucr.edu (8.13.8/8.13.8/Debian-3) with ESMTP id m4BJ9Wuu024698 for <192@emacsbugs.donarmstrong.com>; Sun, 11 May 2008 12:09:34 -0700 Original-Received: from localhost (unknown [127.0.0.1]) by ecelsrv1.ece.lsu.edu (Postfix) with ESMTP id A955E28350; Sun, 11 May 2008 19:09:32 +0000 (UTC) X-Virus-Scanned: amavisd-new at ece.lsu.edu Original-Received: from ecelsrv1.ece.lsu.edu ([127.0.0.1]) by localhost (ecelsrv1.ece.lsu.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fr7PlMx0JlAw; Sun, 11 May 2008 14:09:32 -0500 (CDT) Original-Received: from nested.ece.lsu.edu (nested.ece.lsu.edu [130.39.222.143]) by ecelsrv1.ece.lsu.edu (Postfix) with ESMTP id C5F1D2833B; Sun, 11 May 2008 14:09:31 -0500 (CDT) In-Reply-To: (Stefan Monnier's message of "Sun, 11 May 2008 14:44:12 -0400") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.60 (gnu/linux) X-detected-kernel: by monty-python.gnu.org: Linux 2.6 (newer, 3) Resent-Date: Sun, 11 May 2008 15:31:42 -0400 X-Mailman-Approved-At: Sun, 11 May 2008 18:17:46 -0400 X-BeenThere: bug-gnu-emacs@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:17963 Archived-At: I've decided against having hi-lock turn on font-lock-multiline or even apply font-lock-multiline text properties, too much potential to slow things down to a crawl when an unsuspecting user enters a regexp. If I understand things correctly, the font-lock-multiline property is used to extend a region to be fontified, a region to be used for *all* keywords. This would have disastrous effects when multi-line patterns span, say, 100's of lines for modes with hundreds of keywords. I had been toying with the idea of limiting extended regions to something like 100 lines, but that still seems wasteful when most keywords are single line (I haven't benchmarked anything yet). A better solution would be to have font-lock use multi-line extended regions selectively. Perhaps a hint in the current keyword syntax (say, explicitly applying the font-lock-multiline property), or a separate method for providing multi-line keywords to font-lock. Such keywords would get the multi-line extended regions, the other just the whole-line extensions (or whatever the hooks do). Is this something the font-lock maintainers would consider? What I'll do now is just document the limitations for hi-lock and perhaps provide a warning when a multiline pattern is used. > That may not be enough. You'll probably want to do something like what > smerge does: > > (while (re-search-forward nil t) > (font-lock-fontify-region (match-beginning 0) (match-end 0))) I wouldn't do that without suppressing other keywords. > If someone wants that, I have a parser that takes a regexp and turns it > into something like `rx' syntax. It uses my lex.el library (which > takes an `rx'-like input syntax). That sounds useful, either E-mail it to me or let me know where to find it. Stefan Monnier writes: >> My latest plan is to do what Chong Yidong suggests, setting up text >> properties so that font-lock DTRT, though it doesn't seem as hard as he >> suggests (I'm still in the naive enthusiasm stage). > > Indeed, it shouldn't be that hard. > >> I tried adding the font-lock-multiline property to the face property >> list passed to font lock and that did the trick, even with the >> font-lock-multiline variable nil. > > That may not be enough. You'll probably want to do something like what > smerge does: > > (while (re-search-forward nil t) > (font-lock-fontify-region (match-beginning 0) (match-end 0))) > > this will find all the multiline elements. And the font-lock-multiline > property you add will make sure that those that were found will not > disappear accidentally because of some later refontification. > >> I rather do that than turn on font-lock-multiline because I'm assuming >> that font-lock-multiline is set to nil in most cases for >> a good reason. > > Setting the `font-lock-multiline' variable to t has a performance cost. > >> I actually thought about properly parsing the regexp, but the effort to >> do that could be spent on making multi-line patterns work properly, at >> least if they don't span too many lines. > > If someone wants that, I have a parser that takes a regexp and turns it > into something like `rx' syntax. It uses my lex.el library (which > takes an `rx'-like input syntax). > >> One more thing, multi-line regexp matches don't work properly even >> with font-lock-multiline t when jit-lock is being used in a buffer >> without syntactic fontification and using the default setting of >> jit-lock-contextually, setting it to t gets multi-line fontification >> to work. > > The `font-lock-multiline' variable only tells font-lock that if it ever > bumps into a multiline element, it should mark it (with the > font-lock-multiline property) so that it will not re-fontify it as > a whole if it ever needs to refontify it. > > So it doesn't solve the problem of "how do I make sure that font-lock > indeed finds the multiline element". Multiline elements can only be > found when font-locking a large enough piece of text, which tends to > only happen during the initial fontification, or during background or > contextual refontification, or during an explicit call such as in the > above while loop. > > > Stefan