* Bug in font-lock-syntactic-keywords handling?
@ 2014-10-06 11:42 immerrr again
2014-10-06 13:14 ` Stefan Monnier
0 siblings, 1 reply; 5+ messages in thread
From: immerrr again @ 2014-10-06 11:42 UTC (permalink / raw
To: emacs-devel
Hi all
I've run into an interesting issue with syntactic fontification with
lua-mode: it uses font-lock-syntactic-keywords to mark up long-strings
and long-comments as advised earlier in this list [1] and there has
been a report that this approach failed on a certain file [2]. It
looks like that initial fontification indeed fails on that
"connection.lua", but after-change refontification works fine.
In a nutshell, the problem is that font-lock-syntactically-fontified
is only moved forward when font-lock-syntactically-fontified < start.
Which means that when you first propertize [s0; s1) and then
propertize [s1, s2), font-lock-syntactically-fontified will still be
at s1 which is clearly inconsistent. Let's look at the initial
fontification of the file I've referenced above that exhibits that
behaviour (messages are annotated with their respective line numbers
according to [3]):
===========
1212: unfontify-region: [1; 520]
1217: font-lock-syntactically-fontified: 0
1217: start: 1
1218: moving start to 1
1219: set font-lock-syntactically-fontified: 520
1220: font-lock-fontify-syntactic-keywords-region: [1; 520]
============
1212: unfontify-region: [520; 1082]
1217: font-lock-syntactically-fontified: 520
1217: start: 520
1220: font-lock-fontify-syntactic-keywords-region: [520; 1082]
============
1212: unfontify-region: [1082; 1583]
1217: font-lock-syntactically-fontified: 520
1217: start: 1082
1218: moving start to 520
1219: set font-lock-syntactically-fontified: 1583
1220: font-lock-fontify-syntactic-keywords-region: [520; 1583]
As you can see, for the third iteration start moves back to the
beginning of the second chunk and it tries to re-propertize it without
unfontifying which causes an bug in lua-mode's syntactic keyword
handling. I do know that I (ab-)used a deprecated variable to maintain
compatibility with Emacs23 and using `syntax-propertize-function`
should fix that for Emacs24, but I'd be grateful if anyone pointed me
to a simple fix for Emacs23.
Cheers,
immerrr
1. http://comments.gmane.org/gmane.emacs.devel/153447
2. https://github.com/immerrr/lua-mode/issues/78
3. http://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/font-lock.el#n1212
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Bug in font-lock-syntactic-keywords handling?
2014-10-06 11:42 Bug in font-lock-syntactic-keywords handling? immerrr again
@ 2014-10-06 13:14 ` Stefan Monnier
2014-10-06 14:21 ` immerrr again
0 siblings, 1 reply; 5+ messages in thread
From: Stefan Monnier @ 2014-10-06 13:14 UTC (permalink / raw
To: immerrr again; +Cc: emacs-devel
> As you can see, for the third iteration start moves back to the
> beginning of the second chunk and it tries to re-propertize it without
> unfontifying which causes an bug in lua-mode's syntactic keyword
> handling. I do know that I (ab-)used a deprecated variable to maintain
> compatibility with Emacs23 and using `syntax-propertize-function`
> should fix that for Emacs24, but I'd be grateful if anyone pointed me
> to a simple fix for Emacs23.
That seems like a performance bug (it re-fontifies something that's
already been fontified), indeed, but I don't see why that would cause
something to "fail".
If it really causes a failure, I suspect that the failure is really
caused by an underlying bug in the specific syntactic-keywords rules,
which merely gets triggered by this performance bug.
Stefan
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Bug in font-lock-syntactic-keywords handling?
2014-10-06 13:14 ` Stefan Monnier
@ 2014-10-06 14:21 ` immerrr again
2014-10-06 16:28 ` Stefan Monnier
0 siblings, 1 reply; 5+ messages in thread
From: immerrr again @ 2014-10-06 14:21 UTC (permalink / raw
To: Stefan Monnier; +Cc: emacs-devel
On Mon, Oct 6, 2014 at 5:14 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>
> That seems like a performance bug (it re-fontifies something that's
> already been fontified), indeed, but I don't see why that would cause
> something to "fail".
>
> If it really causes a failure, I suspect that the failure is really
> caused by an underlying bug in the specific syntactic-keywords rules,
> which merely gets triggered by this performance bug.
>
>
> Stefan
The problem is that it doesn't unfontify the already processed chunk beforehand.
It this particular situation, when I encounter a "--[[" comment
starter I need to be sure that there are no dashes before
("---[[...\n" is a single-line comment by design) and that this
starter is not inside another comment. I did this the "smart" way: I
checked (elt ppss 4) of the *second* dash, so that when there's
something like "--X-[[" with X being the point, the point is already
inside a single-line comment. That single check covers both cases,
yet, as most "smart" things do, it breaks unexpectedly.
This one breaks when "--[[" is already propertized as a comment opener
and the second dash is already a part of that comment. This causes
comment-open matcher to fail and simple two-step propertize function
[1],
(lambda (limit) (or (try-close-comment limit) (try-open-comment limit)))
doesn't work, since there is no comment to be closed at the beginning
of the chunk, the opener is already propertized and thus skipped and
the closer is never invoked again.
Cheers,
immerrr
1. This lambda is a simplified version of
https://github.com/immerrr/lua-mode/blob/master/lua-mode.el#L868-L872
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Bug in font-lock-syntactic-keywords handling?
2014-10-06 14:21 ` immerrr again
@ 2014-10-06 16:28 ` Stefan Monnier
2014-10-06 23:27 ` immerrr again
0 siblings, 1 reply; 5+ messages in thread
From: Stefan Monnier @ 2014-10-06 16:28 UTC (permalink / raw
To: immerrr again; +Cc: emacs-devel
> The problem is that it doesn't unfontify the already processed
> chunk beforehand.
Oh, indeed, sorry for being dense.
That's indeed a bug. You could try to work around it with something like:
(defvar lua-font-lock-syntactic-keywords
'(((lambda (limit)
;; font-lock sometimes fails to erase the `syntax-table' property before
;; calling us. So let's make sure it's clean before moving on.
(remove-text-properties (point) limit '(syntax-table))
nil))
(lua-match-multiline-literal-bounds
(1 "!" nil noerror)
(2 "|" nil noerror))))
Also, I recommend you use syntax-propertize-function when available.
It comes with various advantages (e.g. it also works when font-lock is
disabled).
Stefan
PS: BTW, do you have any news about the copyright status of lua-mode
that would prevent including it in GNU ELPA?
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Bug in font-lock-syntactic-keywords handling?
2014-10-06 16:28 ` Stefan Monnier
@ 2014-10-06 23:27 ` immerrr again
0 siblings, 0 replies; 5+ messages in thread
From: immerrr again @ 2014-10-06 23:27 UTC (permalink / raw
To: Stefan Monnier; +Cc: emacs-devel
On Mon, Oct 6, 2014 at 8:28 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>
>
> Also, I recommend you use syntax-propertize-function when available.
> It comes with various advantages (e.g. it also works when font-lock is
> disabled).
>
>
Thanks a lot for your help, this should work. For some reason I
considered remove-text-properties an expensive operation and avoided
it, but a quick benchmark showed that syntax-ppss takes a lot more
time during syntactical fontification.
>
> PS: BTW, do you have any news about the copyright status of lua-mode
> that would prevent including it in GNU ELPA?
Sadly, no news. I haven't tried reaching those who hadn't signed the
assignment since the last time and I'm a lot more humble now in
estimates of my spare time to do actual rewriting. I will most likely
have another stab at it before Christmas.
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2014-10-06 23:27 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-10-06 11:42 Bug in font-lock-syntactic-keywords handling? immerrr again
2014-10-06 13:14 ` Stefan Monnier
2014-10-06 14:21 ` immerrr again
2014-10-06 16:28 ` Stefan Monnier
2014-10-06 23:27 ` immerrr again
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.