* Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. [not found] ` <E1YI1y9-0004eO-Re@vcs.savannah.gnu.org> @ 2015-02-02 18:50 ` Stefan Monnier 2015-02-02 19:27 ` Alan Mackenzie 2015-02-02 18:54 ` Stefan Monnier 2015-03-16 23:53 ` Daniel Colascione 2 siblings, 1 reply; 25+ messages in thread From: Stefan Monnier @ 2015-02-02 18:50 UTC (permalink / raw) To: emacs-devel; +Cc: Alan Mackenzie > + (when (boundp 'font-lock-extend-region-functions) > + (setq font-lock-extend-region-functions nil)) This breaks all other major modes's font-locking. Please make this change buffer-locally, as usual. Stefan ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. 2015-02-02 18:50 ` [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669 Stefan Monnier @ 2015-02-02 19:27 ` Alan Mackenzie 2015-02-03 2:12 ` Stefan Monnier 0 siblings, 1 reply; 25+ messages in thread From: Alan Mackenzie @ 2015-02-02 19:27 UTC (permalink / raw) To: Stefan Monnier; +Cc: emacs-devel Hello, Stefan. On Mon, Feb 02, 2015 at 01:50:35PM -0500, Stefan Monnier wrote: > > + (when (boundp 'font-lock-extend-region-functions) > > + (setq font-lock-extend-region-functions nil)) > This breaks all other major modes's font-locking. Please make this > change buffer-locally, as usual. font-lock-extend-region-functions is set to buffer local on L1155 of font-lock.el (in the trunk). If the above setq is breaking font-locking in other modes, then something strange is going on. Have you actually seen this breakage in other modes? If so, where and how? > Stefan -- Alan Mackenzie (Nuremberg, Germany). ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. 2015-02-02 19:27 ` Alan Mackenzie @ 2015-02-03 2:12 ` Stefan Monnier 0 siblings, 0 replies; 25+ messages in thread From: Stefan Monnier @ 2015-02-03 2:12 UTC (permalink / raw) To: Alan Mackenzie; +Cc: emacs-devel > font-lock-extend-region-functions is set to buffer local on L1155 of > font-lock.el (in the trunk). If the above setq is breaking font-locking > in other modes, then something strange is going on. Duh, sorry, I was blind: I did check with `C-h v font-lock-extend-region-functions' but I guess I saw what I expected to see rather than what was actually displayed. Stefan ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. [not found] ` <E1YI1y9-0004eO-Re@vcs.savannah.gnu.org> 2015-02-02 18:50 ` [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669 Stefan Monnier @ 2015-02-02 18:54 ` Stefan Monnier 2015-02-02 21:39 ` Alan Mackenzie 2015-03-16 23:53 ` Daniel Colascione 2 siblings, 1 reply; 25+ messages in thread From: Stefan Monnier @ 2015-02-02 18:54 UTC (permalink / raw) To: emacs-devel; +Cc: Alan Mackenzie > + ;; Prevent `font-lock-default-fontify-region' extending the region it will > + ;; fontify to whole lines by removing `font-lock-extend-region-whole-lines' > + ;; (and, coincidentally, `font-lock-extend-region-multiline' (which we do > + ;; not need)) from `font-lock-extend-region-functions'. (Emacs only). This > + ;; fixes Emacs bug #19669. My general understanding of font-lock's highlighting is that using a smaller region is only ever useful for performance reason, rather than for correctness (and this line of thinking influences the design of the code). So I'm really curious to understand how this counter-example works, since it might hint at further (latent) bugs elsewhere. Can you walk me through what happens in the scenario of bug#19669? Stefan ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. 2015-02-02 18:54 ` Stefan Monnier @ 2015-02-02 21:39 ` Alan Mackenzie 2015-02-03 2:26 ` Stefan Monnier 0 siblings, 1 reply; 25+ messages in thread From: Alan Mackenzie @ 2015-02-02 21:39 UTC (permalink / raw) To: Stefan Monnier; +Cc: emacs-devel Hello, Stefan. On Mon, Feb 02, 2015 at 01:54:20PM -0500, Stefan Monnier wrote: > > + ;; Prevent `font-lock-default-fontify-region' extending the region it will > > + ;; fontify to whole lines by removing `font-lock-extend-region-whole-lines' > > + ;; (and, coincidentally, `font-lock-extend-region-multiline' (which we do > > + ;; not need)) from `font-lock-extend-region-functions'. (Emacs only). This > > + ;; fixes Emacs bug #19669. > My general understanding of font-lock's highlighting is that using > a smaller region is only ever useful for performance reason, rather than > for correctness (and this line of thinking influences the design of the > code). So I'm really curious to understand how this counter-example > works, since it might hint at further (latent) bugs elsewhere. > Can you walk me through what happens in the scenario of bug#19669? Certainly! The pertinent file.h looks like this: ######################################################################### enum xxx_xxxxxxxxxx_x {XXX_XXXX_XXXX_XXXXX_XXXXX, XXX_XXXX_XXXX_XXXXX_XXXXX, XXX_XXXX_XXXXXX_XXXXXXXX_XXXXXXX, XXX_XXXX_XXXXXX_XXXXXXXX_XXXXXXX, XXX_XXXX_XXXX_XXXXXXXX, XXXXXX_XXXXXXX_XXXX_XXXXXXXX, XXXXX_XXXXXX_XXXX_XXXXXXXX, XXX_XXXXXXX, XXX_XXX_XXXXXXXX, XXX_XXX_XXX_XXXXXXXXX, XXXXXX_XXX_XXXXXXX, X_XXXXXXXXXX}; enum xxx_xxxx_xxxxxxxxxx_x {XXX_XXXXXX_XXXX_XXX, XXX_XXXXXX_XXXX_XX, XXX_XXXXXXXX_XXXX_XXX, XXX_XXXX_XXXXXXX, XXX_XX_XXXXX_XXXX_XXXXXXX, <================================== XXXXXX_XXXXXX_XXXX_XXXXXXXX, XXX_XXXX_XXXXXXX, // xxx XXXXXX_XXXXXXXX}; ######################################################################### 1. Visit the file. 2. Note that in the second enum, only the first 4 identifiers are fontified. 3. This is because only the first jit-lock-chunk-size bytes (rounded up to EOL) are fontified. 4. The second jit-lock-chunk begins inside enum braces. 5. Redisplay attempts to fontify the buffer by calling jit-lock-function for two chunks. The first chunk is fine. 6. jit-lock calls the value of font-lock-fontify-region-function, which is.. 7. ..c-font-lock-fontify-region function, with parameters 506, 631. (Here 506 is BOL marked, 631 is EOB). 8. Here, `new_beg' and `new_end' are initialised to 506, 631, then modified by calling the functions in c-before-context-fontification-functions. This contains just one function, c-context-set-fl-decl-start. 9. c-context-set-fl-decl-start just calls c-set-fl-decl-start.. 10. ..which ensures 506 is not in the middle of a declaration. In doing so, it returns 409, the new "safe" position. This is the position just after the "{". 11. (409 . 631) gets returned to c-font-lock-fontify-region, which calls (font-lock-default-fontify-region 409 631). 12. font-lock-default-fontify-region adjusts 409 to 404, since 409 is not at BOL. 404 is no longer a "syntactically safe" position wrt the original 506. 13. font-lock-default-fontify-region calls (font-lock-fontify-keywords-region 404 631)... 14. ...which eventually calls c-font-lock-enum-tail. This function was the solution to the problem of a jit-lock chunk starting in the middle of a long enum declaration. It fontifies the rest of the enum's identifiers (actually, if fontifies them all, up to limit; optimisation is called for, here). However... 15. .... point is no longer inside the enum's brace block, having been moved from 409 to 404 by font-lock-extend-region-whole-lines. So the condition for c-font-lock-enum-tail isn't met, and the fontification doesn't get triggered. > Stefan -- Alan Mackenzie (Nuremberg, Germany). ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. 2015-02-02 21:39 ` Alan Mackenzie @ 2015-02-03 2:26 ` Stefan Monnier 2015-02-07 12:27 ` Alan Mackenzie 0 siblings, 1 reply; 25+ messages in thread From: Stefan Monnier @ 2015-02-03 2:26 UTC (permalink / raw) To: Alan Mackenzie; +Cc: emacs-devel >> Can you walk me through what happens in the scenario of bug#19669? > Certainly! This looks specific to some assumptions made in CC-mode's font-lock code, rather than to assumptions made in font-lock itself, or assumptions often made in other major modes, so I think we're in the clear. Thanks. > 12. font-lock-default-fontify-region adjusts 409 to 404, since 409 is > not at BOL. 404 is no longer a "syntactically safe" position wrt the > original 506. What would be "the closest syntactically safe" position before 409? Stefan ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. 2015-02-03 2:26 ` Stefan Monnier @ 2015-02-07 12:27 ` Alan Mackenzie 2015-02-07 15:29 ` Stefan Monnier 0 siblings, 1 reply; 25+ messages in thread From: Alan Mackenzie @ 2015-02-07 12:27 UTC (permalink / raw) To: Stefan Monnier; +Cc: emacs-devel Hello, Stefan. On Mon, Feb 02, 2015 at 09:26:46PM -0500, Stefan Monnier wrote: > >> Can you walk me through what happens in the scenario of bug#19669? > > Certainly! > This looks specific to some assumptions made in CC-mode's font-lock > code, rather than to assumptions made in font-lock itself, or > assumptions often made in other major modes, so I think we're in the clear. > Thanks. > > 12. font-lock-default-fontify-region adjusts 409 to 404, since 409 is > > not at BOL. 404 is no longer a "syntactically safe" position wrt the > > original 506. > What would be "the closest syntactically safe" position before 409? In the sense meant, 409 is the earliest "syntactically safe" position, since (char-before 409) is "{" > Stefan -- Alan Mackenzie (Nuremberg, Germany). ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. 2015-02-07 12:27 ` Alan Mackenzie @ 2015-02-07 15:29 ` Stefan Monnier 2015-02-07 15:40 ` Alan Mackenzie 0 siblings, 1 reply; 25+ messages in thread From: Stefan Monnier @ 2015-02-07 15:29 UTC (permalink / raw) To: Alan Mackenzie; +Cc: emacs-devel >> What would be "the closest syntactically safe" position before 409? > In the sense meant, 409 is the earliest "syntactically safe" position, > since (char-before 409) is "{" So, even point-min wouldn't be safe? Stefan ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. 2015-02-07 15:29 ` Stefan Monnier @ 2015-02-07 15:40 ` Alan Mackenzie 0 siblings, 0 replies; 25+ messages in thread From: Alan Mackenzie @ 2015-02-07 15:40 UTC (permalink / raw) To: Stefan Monnier; +Cc: emacs-devel Hello, Stefan. On Sat, Feb 07, 2015 at 10:29:54AM -0500, Stefan Monnier wrote: > >> What would be "the closest syntactically safe" position before 409? > > In the sense meant, 409 is the earliest "syntactically safe" position, > > since (char-before 409) is "{" > So, even point-min wouldn't be safe? No. It has to be somewhere within the enum's braces to work. > Stefan -- Alan Mackenzie (Nuremberg, Germany). ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. [not found] ` <E1YI1y9-0004eO-Re@vcs.savannah.gnu.org> 2015-02-02 18:50 ` [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669 Stefan Monnier 2015-02-02 18:54 ` Stefan Monnier @ 2015-03-16 23:53 ` Daniel Colascione 2015-03-18 12:08 ` Alan Mackenzie 2015-03-30 16:44 ` Alan Mackenzie 2 siblings, 2 replies; 25+ messages in thread From: Daniel Colascione @ 2015-03-16 23:53 UTC (permalink / raw) To: emacs-devel, Alan Mackenzie [-- Attachment #1: Type: text/plain, Size: 793 bytes --] On 02/01/2015 01:22 PM, Alan Mackenzie wrote: > branch: emacs-24 > commit d69e9f1c24f37f36af33b0468c5a4e100dbb09b6 > Author: Alan Mackenzie <acm@muc.de> > Commit: Alan Mackenzie <acm@muc.de> > > CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. > > cc-mode.el (c-font-lock-init): Setq font-lock-extend-region-functions to > nil. This change is driving me nuts. Consider the following buffer, with @ representing point: void foo() { @ } If I type [f o r SPC], then depending on timing, I see only the "or", or only the "r", fontified in font-lock-keyword-face. A moment later, jit-lock kicks in and refontifies the entire chunk properly. Can we please extend the start of the region back to at least the last `}'? [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. 2015-03-16 23:53 ` Daniel Colascione @ 2015-03-18 12:08 ` Alan Mackenzie 2015-03-19 3:34 ` Daniel Colascione 2015-03-30 16:44 ` Alan Mackenzie 1 sibling, 1 reply; 25+ messages in thread From: Alan Mackenzie @ 2015-03-18 12:08 UTC (permalink / raw) To: Daniel Colascione; +Cc: emacs-devel Hello, Daniel. On Mon, Mar 16, 2015 at 04:53:28PM -0700, Daniel Colascione wrote: > On 02/01/2015 01:22 PM, Alan Mackenzie wrote: > > branch: emacs-24 > > commit d69e9f1c24f37f36af33b0468c5a4e100dbb09b6 > > Author: Alan Mackenzie <acm@muc.de> > > Commit: Alan Mackenzie <acm@muc.de> > > CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. > > cc-mode.el (c-font-lock-init): Setq font-lock-extend-region-functions to > > nil. > This change is driving me nuts. Consider the following buffer, with @ > representing point: > void foo() { > @ > } > If I type [f o r SPC], then depending on timing, I see only the "or", or > only the "r", fontified in font-lock-keyword-face. A moment later, > jit-lock kicks in and refontifies the entire chunk properly. Sorry about this. > Can we please extend the start of the region back to at least the last `}'? This could be a tricky one to solve. For the sake of your sanity, please consider applying the following temporary patch; it "solves" the current problem, but unfortunately leaves bug #19669's symptoms apparent again. diff -r cffcacba7345 cc-mode.el --- a/cc-mode.el Sun Feb 01 21:06:18 2015 +0000 +++ b/cc-mode.el Wed Mar 18 12:04:00 2015 +0000 @@ -1208,7 +1208,17 @@ ;; Type a space in the first blank line, and the fontification of the next ;; line was fouled up by context fontification. (let ((new-beg beg) (new-end end) new-region case-fold-search - open-paren-in-column-0-is-defun-start) + open-paren-in-column-0-is-defun-start + ;; Except in the after-change case, prevent + ;; `font-lock-default-fontify-region' extending the region it will + ;; fontify to whole lines. The following binding removes + ;; `font-lock-extend-region-whole-lines' from + ;; `font-lock-extend-region-functions'. (Emacs only). This fixes + ;; Emacs bug #19669. + (font-lock-extend-region-functions + (and (boundp 'font-lock-extend-region-functions) + c-in-after-change-fontification + font-lock-extend-region-functions))) (if c-in-after-change-fontification (setq c-in-after-change-fontification nil) (save-restriction @@ -1247,13 +1257,13 @@ (font-lock-mark-block-function . c-mark-function))) - ;; Prevent `font-lock-default-fontify-region' extending the region it will - ;; fontify to whole lines by removing `font-lock-extend-region-whole-lines' - ;; (and, coincidentally, `font-lock-extend-region-multiline' (which we do - ;; not need)) from `font-lock-extend-region-functions'. (Emacs only). This - ;; fixes Emacs bug #19669. - (when (boundp 'font-lock-extend-region-functions) - (setq font-lock-extend-region-functions nil)) + ;; ;; Prevent `font-lock-default-fontify-region' extending the region it will + ;; ;; fontify to whole lines by removing `font-lock-extend-region-whole-lines' + ;; ;; (and, coincidentally, `font-lock-extend-region-multiline' (which we do + ;; ;; not need)) from `font-lock-extend-region-functions'. (Emacs only). This + ;; ;; fixes Emacs bug #19669. + ;; (when (boundp 'font-lock-extend-region-functions) + ;; (setq font-lock-extend-region-functions nil)) (make-local-variable 'font-lock-fontify-region-function) (setq font-lock-fontify-region-function 'c-font-lock-fontify-region) -- Alan Mackenzie (Nuremberg, Germany). ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. 2015-03-18 12:08 ` Alan Mackenzie @ 2015-03-19 3:34 ` Daniel Colascione 2015-03-19 9:31 ` Alan Mackenzie 0 siblings, 1 reply; 25+ messages in thread From: Daniel Colascione @ 2015-03-19 3:34 UTC (permalink / raw) To: Alan Mackenzie; +Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 1463 bytes --] On 03/18/2015 05:08 AM, Alan Mackenzie wrote: > On Mon, Mar 16, 2015 at 04:53:28PM -0700, Daniel Colascione wrote: >> On 02/01/2015 01:22 PM, Alan Mackenzie wrote: >>> branch: emacs-24 >>> commit d69e9f1c24f37f36af33b0468c5a4e100dbb09b6 >>> Author: Alan Mackenzie <acm@muc.de> >>> Commit: Alan Mackenzie <acm@muc.de> > >>> CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. > >>> cc-mode.el (c-font-lock-init): Setq font-lock-extend-region-functions to >>> nil. > >> This change is driving me nuts. Consider the following buffer, with @ >> representing point: > >> void foo() { >> @ >> } > >> If I type [f o r SPC], then depending on timing, I see only the "or", or >> only the "r", fontified in font-lock-keyword-face. A moment later, >> jit-lock kicks in and refontifies the entire chunk properly. > > Sorry about this. > >> Can we please extend the start of the region back to at least the last `}'? > > This could be a tricky one to solve. For the sake of your sanity, please > consider applying the following temporary patch; it "solves" the current > problem, but unfortunately leaves bug #19669's symptoms apparent again. I've been doing the equivalent in a mode hook. I'm not sure that reverting to the previous behavior is the right thing either. Fundamentally, it just feels wrong that the solution to any fontification correctness bug should be to fontify less. [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. 2015-03-19 3:34 ` Daniel Colascione @ 2015-03-19 9:31 ` Alan Mackenzie 2015-03-19 13:49 ` Stefan Monnier 0 siblings, 1 reply; 25+ messages in thread From: Alan Mackenzie @ 2015-03-19 9:31 UTC (permalink / raw) To: Daniel Colascione; +Cc: emacs-devel Hello, Daniel. On Wed, Mar 18, 2015 at 08:34:19PM -0700, Daniel Colascione wrote: > On 03/18/2015 05:08 AM, Alan Mackenzie wrote: > > On Mon, Mar 16, 2015 at 04:53:28PM -0700, Daniel Colascione wrote: > >> On 02/01/2015 01:22 PM, Alan Mackenzie wrote: > >>> branch: emacs-24 > >>> commit d69e9f1c24f37f36af33b0468c5a4e100dbb09b6 > >>> Author: Alan Mackenzie <acm@muc.de> > >>> Commit: Alan Mackenzie <acm@muc.de> > >>> CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. > >>> cc-mode.el (c-font-lock-init): Setq font-lock-extend-region-functions to > >>> nil. > >> This change is driving me nuts. Consider the following buffer, with @ > >> representing point: > >> void foo() { > >> @ > >> } > >> If I type [f o r SPC], then depending on timing, I see only the "or", or > >> only the "r", fontified in font-lock-keyword-face. A moment later, > >> jit-lock kicks in and refontifies the entire chunk properly. > > Sorry about this. > >> Can we please extend the start of the region back to at least the last `}'? > > This could be a tricky one to solve. For the sake of your sanity, please > > consider applying the following temporary patch; it "solves" the current > > problem, but unfortunately leaves bug #19669's symptoms apparent again. > I've been doing the equivalent in a mode hook. I'm not sure that > reverting to the previous behavior is the right thing either. > Fundamentally, it just feels wrong that the solution to any > fontification correctness bug should be to fontify less. The problem in #19669 was that Font Lock's extension of the region to be font locked to the beginning of the line was changing the syntactic context, hence CC Mode's font locking didn't work in that case. The solution was to prevent that extension in the pertinent case. However, I got it wrong. The solution I'm looking at at the moment is to allow Font Lock to extend to BOL except where CC Mode itself has extended the region for syntactic reasons. As I said, it's tricky. Fundamentally, the problem is that Font Lock doesn't distinguish between the region which needs fontifying, and the region which supplies the syntactic context. -- Alan Mackenzie (Nuremberg, Germany). ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. 2015-03-19 9:31 ` Alan Mackenzie @ 2015-03-19 13:49 ` Stefan Monnier 2015-03-19 20:37 ` Alan Mackenzie 0 siblings, 1 reply; 25+ messages in thread From: Stefan Monnier @ 2015-03-19 13:49 UTC (permalink / raw) To: Alan Mackenzie; +Cc: Daniel Colascione, emacs-devel > The problem in #19669 was that Font Lock's extension of the region to be > font locked to the beginning of the line was changing the syntactic > context, hence CC Mode's font locking didn't work in that case. The > solution was to prevent that extension in the pertinent case. However, I > got it wrong. The way I see it, the core of the problem is that the way you handle partial re-fontification of a multi-line `union' construct has a dead spot. More specifically, if you have code like 1 union 2 { 3 foo, 4 bar 5 } you can handle fontification from 1,3, or 4 but not from 2. You need to refine the system you use to keep track of whether we're within a `union' so that it knows that position 2 is also "within a union". Stefan ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. 2015-03-19 13:49 ` Stefan Monnier @ 2015-03-19 20:37 ` Alan Mackenzie 2015-03-19 20:56 ` Stefan Monnier 0 siblings, 1 reply; 25+ messages in thread From: Alan Mackenzie @ 2015-03-19 20:37 UTC (permalink / raw) To: Stefan Monnier; +Cc: Daniel Colascione, emacs-devel Hello, Stefan. On Thu, Mar 19, 2015 at 09:49:13AM -0400, Stefan Monnier wrote: > > The problem in #19669 was that Font Lock's extension of the region to be > > font locked to the beginning of the line was changing the syntactic > > context, hence CC Mode's font locking didn't work in that case. The > > solution was to prevent that extension in the pertinent case. However, I > > got it wrong. > The way I see it, the core of the problem is that the way you handle > partial re-fontification of a multi-line `union' construct has > a dead spot. > More specifically, if you have code like > 1 union > 2 { > 3 foo, > 4 bar > 5 } > you can handle fontification from 1,3, or 4 but not from 2. You need to > refine the system you use to keep track of whether we're within > a `union' so that it knows that position 2 is also "within a union". Well, sort of. The problem I'm facing is that in Dima Kogan's bug #19669, the following construct appears: 1. enum xxx_xxxx_xxxxxxxxxx_x 2. {XXX_XXXXXX_XXXX_XXX, 3. XXX_XXXXXX_XXX_XXX, 4. XXXX_XXXXX_XXXX_XXX, Note that the brace on L2 is on the same line as the first XXX_.... When the user types on line 4, 5, ... here, CC Mode sets the fontification region start to JUST AFTER THE { ON L2. It is essential that Font Lock doesn't change this. When Daniel types the "r" of his "for", it is equally essential that Font Lock (or something) does adjust the fontification region start to BOL. This is the more normal case. How do we reconcile these? The pertinent adjustment is done by having, or not having, `font-lock-extend-region-wholelines' on `font-lock-extend-region-functions'. So, I have a solution which, in the "enum case" sets a flag which instructs c-font-lock-fontify-region later to bind `f-l-extend-r-f' to nil. Should work. Unfortunately, there is a dirty hack in `font-lock-extend-jit-lock-region-after-change' which illegitimately (i.e. at a time when it is forbidden, according to the contract on the variable) tests `font-lock-extend-region-functions' for 'f-l-extend-region-wholelines' and then fouls up the beginning of the "enum region" to BOL 2 before CC Mode's code has had a chance to bind `f-l-extend-region-functions' to nil. The purpose of this hack, made in 2006, is to prevent double redisplay, according to the comments in font-lock.el. I can't see how it works, right at the moment, but couldn't we somehow remove this dirty hack? > Stefan -- Alan Mackenzie (Nuremberg, Germany). ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. 2015-03-19 20:37 ` Alan Mackenzie @ 2015-03-19 20:56 ` Stefan Monnier 2015-03-20 16:30 ` Alan Mackenzie 0 siblings, 1 reply; 25+ messages in thread From: Stefan Monnier @ 2015-03-19 20:56 UTC (permalink / raw) To: Alan Mackenzie; +Cc: Daniel Colascione, emacs-devel >> you can handle fontification from 1,3, or 4 but not from 2. You need to >> refine the system you use to keep track of whether we're within >> a `union' so that it knows that position 2 is also "within a union". > Well, sort of. The problem I'm facing is that in Dima Kogan's bug > #19669, the following construct appears: > 1. enum xxx_xxxx_xxxxxxxxxx_x > 2. {XXX_XXXXXX_XXXX_XXX, > 3. XXX_XXXXXX_XXX_XXX, > 4. XXXX_XXXXX_XXXX_XXX, > Note that the brace on L2 is on the same line as the first XXX_.... > When the user types on line 4, 5, ... here, CC Mode sets the > fontification region start to JUST AFTER THE { ON L2. It is essential > that Font Lock doesn't change this. There's your problem: your current setup needs the starting point to be either before "union" or after the first open brace. It breaks down if it's between the two. That's the problem you need to fix. Stefan ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. 2015-03-19 20:56 ` Stefan Monnier @ 2015-03-20 16:30 ` Alan Mackenzie 2015-03-20 16:34 ` Daniel Colascione 2015-03-20 18:05 ` Stefan Monnier 0 siblings, 2 replies; 25+ messages in thread From: Alan Mackenzie @ 2015-03-20 16:30 UTC (permalink / raw) To: Stefan Monnier; +Cc: Daniel Colascione, emacs-devel Hello, Stefan. On Thu, Mar 19, 2015 at 04:56:16PM -0400, Stefan Monnier wrote: > >> you can handle fontification from 1,3, or 4 but not from 2. You need to > >> refine the system you use to keep track of whether we're within > >> a `union' so that it knows that position 2 is also "within a union". > > Well, sort of. The problem I'm facing is that in Dima Kogan's bug > > #19669, the following construct appears: > > 1. enum xxx_xxxx_xxxxxxxxxx_x > > 2. {XXX_XXXXXX_XXXX_XXX, > > 3. XXX_XXXXXX_XXX_XXX, > > 4. XXXX_XXXXX_XXXX_XXX, > > Note that the brace on L2 is on the same line as the first XXX_.... > > When the user types on line 4, 5, ... here, CC Mode sets the > > fontification region start to JUST AFTER THE { ON L2. It is essential > > that Font Lock doesn't change this. > There's your problem: your current setup needs the starting point to be > either before "union" or after the first open brace. > It breaks down if it's between the two. That's the problem you need to fix. Not really. CC Mode is quite capable of handling the Font Lock region starting at BOL2. The problem is, when that starting point needs to be after the brace on L2, Font Lock moves the starting point somewhere else, fouling up the font locking. This is proving surprisingly tough to fix. One place which is causing me puzzlement is in jit-lock-fontify-now. It's this bit: ;; The redisplay engine has already rendered the buffer up-to ;; `orig-start' and won't notice if the above jit-lock-functions ;; changed the appearance of any part of the buffer prior ;; to that. So if `start' is before `orig-start', we need to ;; cause a new redisplay cycle after this one so that any changes ;; are properly reflected on screen. ;; To make such repeated redisplay happen less often, we can ;; eagerly extend the refontified region with ;; jit-lock-after-change-extend-region-functions. (when (< start orig-start) (run-with-timer 0 nil #'jit-lock-force-redisplay (copy-marker start) (copy-marker orig-start))) The only way `start' can be before `orig-start' is on the first line of the fl region, and start is moved to BOL. Why is it necessary to go to all the expense of using `run-with-timer' rather than just fixing things inside `jit-lock-fontify-now'? What is the use case? The mechanism here, in particular the bit the attempts to avoid using this `run-with-timer' too often, is causing me pain. Could we not resolve the problem, whatever it is, by a less convoluted means? > Stefan -- Alan Mackenzie (Nuremberg, Germany). ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. 2015-03-20 16:30 ` Alan Mackenzie @ 2015-03-20 16:34 ` Daniel Colascione 2015-03-20 17:29 ` Alan Mackenzie 2015-03-20 18:05 ` Stefan Monnier 1 sibling, 1 reply; 25+ messages in thread From: Daniel Colascione @ 2015-03-20 16:34 UTC (permalink / raw) To: Alan Mackenzie, Stefan Monnier; +Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 1724 bytes --] On 03/20/2015 09:30 AM, Alan Mackenzie wrote: > Hello, Stefan. > > On Thu, Mar 19, 2015 at 04:56:16PM -0400, Stefan Monnier wrote: >>>> you can handle fontification from 1,3, or 4 but not from 2. You need to >>>> refine the system you use to keep track of whether we're within >>>> a `union' so that it knows that position 2 is also "within a union". > >>> Well, sort of. The problem I'm facing is that in Dima Kogan's bug >>> #19669, the following construct appears: > >>> 1. enum xxx_xxxx_xxxxxxxxxx_x >>> 2. {XXX_XXXXXX_XXXX_XXX, >>> 3. XXX_XXXXXX_XXX_XXX, >>> 4. XXXX_XXXXX_XXXX_XXX, > >>> Note that the brace on L2 is on the same line as the first XXX_.... > >>> When the user types on line 4, 5, ... here, CC Mode sets the >>> fontification region start to JUST AFTER THE { ON L2. It is essential >>> that Font Lock doesn't change this. > >> There's your problem: your current setup needs the starting point to be >> either before "union" or after the first open brace. >> It breaks down if it's between the two. That's the problem you need to fix. > > Not really. CC Mode is quite capable of handling the Font Lock region > starting at BOL2. The problem is, when that starting point needs to be > after the brace on L2, Font Lock moves the starting point somewhere > else, fouling up the font locking. This is proving surprisingly tough > to fix. I don't understand. Why shouldn't I be able to tell cc-mode to fontify *arbitrary* regions and expect it to correctly apply highlighting to these regions? It's the idea that the region "needs to be after brace" that I find confusing. Shouldn't jit-lock have the right to expand the region arbitrarily? [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. 2015-03-20 16:34 ` Daniel Colascione @ 2015-03-20 17:29 ` Alan Mackenzie 2015-03-20 21:25 ` Daniel Colascione 0 siblings, 1 reply; 25+ messages in thread From: Alan Mackenzie @ 2015-03-20 17:29 UTC (permalink / raw) To: Daniel Colascione; +Cc: Stefan Monnier, emacs-devel Hello, Daniel. On Fri, Mar 20, 2015 at 09:34:52AM -0700, Daniel Colascione wrote: > On 03/20/2015 09:30 AM, Alan Mackenzie wrote: > > On Thu, Mar 19, 2015 at 04:56:16PM -0400, Stefan Monnier wrote: > >>>> you can handle fontification from 1,3, or 4 but not from 2. You need to > >>>> refine the system you use to keep track of whether we're within > >>>> a `union' so that it knows that position 2 is also "within a union". > >>> Well, sort of. The problem I'm facing is that in Dima Kogan's bug > >>> #19669, the following construct appears: > >>> 1. enum xxx_xxxx_xxxxxxxxxx_x > >>> 2. {XXX_XXXXXX_XXXX_XXX, > >>> 3. XXX_XXXXXX_XXX_XXX, > >>> 4. XXXX_XXXXX_XXXX_XXX, > >>> Note that the brace on L2 is on the same line as the first XXX_.... > >>> When the user types on line 4, 5, ... here, CC Mode sets the > >>> fontification region start to JUST AFTER THE { ON L2. It is essential > >>> that Font Lock doesn't change this. > >> There's your problem: your current setup needs the starting point to be > >> either before "union" or after the first open brace. > >> It breaks down if it's between the two. That's the problem you need to fix. > > Not really. CC Mode is quite capable of handling the Font Lock region > > starting at BOL2. The problem is, when that starting point needs to be > > after the brace on L2, Font Lock moves the starting point somewhere > > else, fouling up the font locking. This is proving surprisingly tough > > to fix. > I don't understand. Why shouldn't I be able to tell cc-mode to fontify > *arbitrary* regions and expect it to correctly apply highlighting to > these regions? You can, as a CC Mode user. In its turn CC Mode needs to be able to select the region in which it does its analysis, so that that analysis starts at a neutral syntactic position. > It's the idea that the region "needs to be after brace" that I find > confusing. Shouldn't jit-lock have the right to expand the region > arbitrarily? Absolutely not. CC Mode needs to do syntactic analysis inside this region, so the region needs to start at the (syntactically) right place. To do this analysis CC Mode must first find the beginning of any definition the current position is in, otherwise context fontification can go wrong. An example of this going wrong is in this snippet: 1. template <typename T> 2. 3. 4. void myfunc(T* p) {} If a space is inserted on L3, this will trigger context fontification of L4. L4 is fontified differently in the presence of the template declaration (for reasons you, a C++ hacker, probably understand better than me). What the bug reporter saw was L4's fontification going wrong a half second after typing the space, something as irritating as what is happening to your "for". The solution I implemented for the problem was to extend the font lock region to include L1. That same region extension which includes that template line, in the enum example goes back to after the "{" on L2. Bug #19669 went wrong in c-font-lock-enum-tail, the function which handles a the start of a fontification region being within an enum brace block. c-font-lock-enum-tail will only trigger when the starting point is within that brace block. When Font Lock moves the starting point somewhere else (in particular, BOL), that condition no longer holds, and the font locking fails. -- Alan Mackenzie (Nuremberg, Germany). ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. 2015-03-20 17:29 ` Alan Mackenzie @ 2015-03-20 21:25 ` Daniel Colascione 2015-03-20 22:30 ` Alan Mackenzie 0 siblings, 1 reply; 25+ messages in thread From: Daniel Colascione @ 2015-03-20 21:25 UTC (permalink / raw) To: Alan Mackenzie; +Cc: Stefan Monnier, emacs-devel [-- Attachment #1: Type: text/plain, Size: 4436 bytes --] On 03/20/2015 10:29 AM, Alan Mackenzie wrote: > On Fri, Mar 20, 2015 at 09:34:52AM -0700, Daniel Colascione wrote: >> On 03/20/2015 09:30 AM, Alan Mackenzie wrote: >>> On Thu, Mar 19, 2015 at 04:56:16PM -0400, Stefan Monnier wrote: >>>>>> you can handle fontification from 1,3, or 4 but not from 2. You need to >>>>>> refine the system you use to keep track of whether we're within >>>>>> a `union' so that it knows that position 2 is also "within a union". > >>>>> Well, sort of. The problem I'm facing is that in Dima Kogan's bug >>>>> #19669, the following construct appears: > >>>>> 1. enum xxx_xxxx_xxxxxxxxxx_x >>>>> 2. {XXX_XXXXXX_XXXX_XXX, >>>>> 3. XXX_XXXXXX_XXX_XXX, >>>>> 4. XXXX_XXXXX_XXXX_XXX, > >>>>> Note that the brace on L2 is on the same line as the first XXX_.... > >>>>> When the user types on line 4, 5, ... here, CC Mode sets the >>>>> fontification region start to JUST AFTER THE { ON L2. It is essential >>>>> that Font Lock doesn't change this. > >>>> There's your problem: your current setup needs the starting point to be >>>> either before "union" or after the first open brace. >>>> It breaks down if it's between the two. That's the problem you need to fix. > >>> Not really. CC Mode is quite capable of handling the Font Lock region >>> starting at BOL2. The problem is, when that starting point needs to be >>> after the brace on L2, Font Lock moves the starting point somewhere >>> else, fouling up the font locking. This is proving surprisingly tough >>> to fix. > >> I don't understand. Why shouldn't I be able to tell cc-mode to fontify >> *arbitrary* regions and expect it to correctly apply highlighting to >> these regions? > > You can, as a CC Mode user. In its turn CC Mode needs to be able to > select the region in which it does its analysis, so that that analysis > starts at a neutral syntactic position. By itself, this statement makes sense, but isn't the right "neutral syntactic position" for a given fontification region purely a function of the region bounds and buffer state? cc-mode should be able to find the right position for any pair of buffer positions. If the right start position is before the region to be fontified, cc-mode can expand the region to encompass it. >> It's the idea that the region "needs to be after brace" that I find >> confusing. Shouldn't jit-lock have the right to expand the region >> arbitrarily? > > Absolutely not. CC Mode needs to do syntactic analysis inside this > region, so the region needs to start at the (syntactically) right place. Isn't it cc-mode's job to find a good position within or before the region font-lock gives it to fontify? cc-mode can nominate a good region using the extend-region functions, but as I see it, there's no contractual requirement that font-lock not arbitrarily enlarge this region. > To do this analysis CC Mode must first find the beginning of any > definition the current position is in, otherwise context fontification > can go wrong. An example of this going wrong is in this snippet: > > 1. template <typename T> > 2. > 3. > 4. void myfunc(T* p) {} > > If a space is inserted on L3, this will trigger context fontification of > L4. L4 is fontified differently in the presence of the template > declaration (for reasons you, a C++ hacker, probably understand better > than me). What the bug reporter saw was L4's fontification going wrong a > half second after typing the space, something as irritating as what is > happening to your "for". The solution I implemented for the problem was > to extend the font lock region to include L1. Extending the region to L1 is the right behavior. Why can't we perform this extension when asked to fontify any range in L1-L4? > That same region extension which includes that template line, in the enum > example goes back to after the "{" on L2. Bug #19669 went wrong in > c-font-lock-enum-tail, the function which handles a the start of a > fontification region being within an enum brace block. > c-font-lock-enum-tail will only trigger when the starting point is within > that brace block. When Font Lock moves the starting point somewhere else > (in particular, BOL), that condition no longer holds, and the font locking > fails. Why can't we check whether any character inside the region to fontify is an enum brace end? [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. 2015-03-20 21:25 ` Daniel Colascione @ 2015-03-20 22:30 ` Alan Mackenzie 0 siblings, 0 replies; 25+ messages in thread From: Alan Mackenzie @ 2015-03-20 22:30 UTC (permalink / raw) To: Daniel Colascione; +Cc: Stefan Monnier, emacs-devel Hello, Daniel. On Fri, Mar 20, 2015 at 02:25:09PM -0700, Daniel Colascione wrote: > On 03/20/2015 10:29 AM, Alan Mackenzie wrote: > > On Fri, Mar 20, 2015 at 09:34:52AM -0700, Daniel Colascione wrote: > >> On 03/20/2015 09:30 AM, Alan Mackenzie wrote: > >>> On Thu, Mar 19, 2015 at 04:56:16PM -0400, Stefan Monnier wrote: > >>>>>> you can handle fontification from 1,3, or 4 but not from 2. You need to > >>>>>> refine the system you use to keep track of whether we're within > >>>>>> a `union' so that it knows that position 2 is also "within a union". > >>>>> Well, sort of. The problem I'm facing is that in Dima Kogan's bug > >>>>> #19669, the following construct appears: > >>>>> 1. enum xxx_xxxx_xxxxxxxxxx_x > >>>>> 2. {XXX_XXXXXX_XXXX_XXX, > >>>>> 3. XXX_XXXXXX_XXX_XXX, > >>>>> 4. XXXX_XXXXX_XXXX_XXX, > >>>>> Note that the brace on L2 is on the same line as the first XXX_.... > >>>>> When the user types on line 4, 5, ... here, CC Mode sets the > >>>>> fontification region start to JUST AFTER THE { ON L2. It is essential > >>>>> that Font Lock doesn't change this. > >>>> There's your problem: your current setup needs the starting point to be > >>>> either before "union" or after the first open brace. > >>>> It breaks down if it's between the two. That's the problem you need to fix. > >>> Not really. CC Mode is quite capable of handling the Font Lock region > >>> starting at BOL2. The problem is, when that starting point needs to be > >>> after the brace on L2, Font Lock moves the starting point somewhere > >>> else, fouling up the font locking. This is proving surprisingly tough > >>> to fix. > >> I don't understand. Why shouldn't I be able to tell cc-mode to fontify > >> *arbitrary* regions and expect it to correctly apply highlighting to > >> these regions? > > You can, as a CC Mode user. In its turn CC Mode needs to be able to > > select the region in which it does its analysis, so that that analysis > > starts at a neutral syntactic position. > By itself, this statement makes sense, but isn't the right "neutral > syntactic position" for a given fontification region purely a function > of the region bounds and buffer state? Well, yes. And if it's a change triggered fontification, then a function also of the change data. > cc-mode should be able to find the right position for any pair of > buffer positions. If the right start position is before the region to > be fontified, cc-mode can expand the region to encompass it. That's precisely what is done. I'm not sure what you're trying to say here. > >> It's the idea that the region "needs to be after brace" that I find > >> confusing. Shouldn't jit-lock have the right to expand the region > >> arbitrarily? > > Absolutely not. CC Mode needs to do syntactic analysis inside this > > region, so the region needs to start at the (syntactically) right place. > Isn't it cc-mode's job to find a good position within or before the > region font-lock gives it to fontify? cc-mode can nominate a good region > using the extend-region functions, .... CC Mode does indeed find this good position, and communicates it to Font Lock via the specified interfaces. > .... but as I see it, there's no contractual requirement that font-lock > not arbitrarily enlarge this region. We differ here starkly. Think about it: the major mode is the absolute expert on where font-locking should start. Font Lock itself is merely an enthusiastic amateur, knowing nothing about the properties of the buffer. It is impudence of the highest order for Font Lock to override the major mode. In practice, Font Lock arbitrarily enlarging the region specified by the major mode has lead to problems, namely bug #19669, and thence to your problems with "for". > > To do this analysis CC Mode must first find the beginning of any > > definition the current position is in, otherwise context fontification > > can go wrong. An example of this going wrong is in this snippet: > > 1. template <typename T> > > 2. > > 3. > > 4. void myfunc(T* p) {} > > If a space is inserted on L3, this will trigger context fontification of > > L4. L4 is fontified differently in the presence of the template > > declaration (for reasons you, a C++ hacker, probably understand better > > than me). What the bug reporter saw was L4's fontification going wrong a > > half second after typing the space, something as irritating as what is > > happening to your "for". The solution I implemented for the problem was > > to extend the font lock region to include L1. > Extending the region to L1 is the right behavior. Why can't we perform > this extension when asked to fontify any range in L1-L4? We do. > > That same region extension which includes that template line, in the enum > > example goes back to after the "{" on L2. Bug #19669 went wrong in > > c-font-lock-enum-tail, the function which handles a the start of a > > fontification region being within an enum brace block. > > c-font-lock-enum-tail will only trigger when the starting point is within > > that brace block. When Font Lock moves the starting point somewhere else > > (in particular, BOL), that condition no longer holds, and the font locking > > fails. > Why can't we check whether any character inside the region to fontify is > an enum brace end? What do you mean by "enum brace end"? -- Alan Mackenzie (Nuremberg, Germany). ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. 2015-03-20 16:30 ` Alan Mackenzie 2015-03-20 16:34 ` Daniel Colascione @ 2015-03-20 18:05 ` Stefan Monnier 2015-03-20 21:12 ` Alan Mackenzie 1 sibling, 1 reply; 25+ messages in thread From: Stefan Monnier @ 2015-03-20 18:05 UTC (permalink / raw) To: Alan Mackenzie; +Cc: Daniel Colascione, emacs-devel >> There's your problem: your current setup needs the starting point to be >> either before "union" or after the first open brace. >> It breaks down if it's between the two. That's the problem you need to fix. > Not really. CC Mode is quite capable of handling the Font Lock region > starting at BOL2. You yourself told me that the region had to start after the brace. > The problem is, when that starting point needs to be > after the brace on L2, Font Lock moves the starting point somewhere > else, fouling up the font locking. I'm confused: "somewhere else" is typically BOL2. Here you say this fouls up font-locking, but above you said CC-Mode is quite capable of handling a font-lock region starting at BOL2. > One place which is causing me puzzlement is in jit-lock-fontify-now. > It's this bit: As you know, jit-lock is irrelevant to those kinds of font-lock problems (in the sense that it's always possible to trigger the same problems without the use of jit-lock, tho it may require some time to figure out a triggering sequence). > The only way `start' can be before `orig-start' is on the first line of > the fl region, and start is moved to BOL. Why is it necessary to go to > all the expense of using `run-with-timer' rather than just fixing things > inside `jit-lock-fontify-now'? What is the use case? It's explained in the comment. Which part of the comment don't you understand? By the time jit-lock is called the redisplay up to orig-start has already been done (you can assume it's already drawn on the screen). Stefan ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. 2015-03-20 18:05 ` Stefan Monnier @ 2015-03-20 21:12 ` Alan Mackenzie 2015-03-20 22:01 ` Stefan Monnier 0 siblings, 1 reply; 25+ messages in thread From: Alan Mackenzie @ 2015-03-20 21:12 UTC (permalink / raw) To: Stefan Monnier; +Cc: Daniel Colascione, emacs-devel Hello, Stefan. On Fri, Mar 20, 2015 at 02:05:48PM -0400, Stefan Monnier wrote: > >> There's your problem: your current setup needs the starting point to be > >> either before "union" or after the first open brace. > >> It breaks down if it's between the two. That's the problem you need to fix. > > Not really. CC Mode is quite capable of handling the Font Lock region > > starting at BOL2. > You yourself told me that the region had to start after the brace. In certain circumstances, this is necessary. When CC Mode sets a position using font-lock-extend-after-change-region-function, then it is necessary to start font-locking from that position. For example, if a line in a C file looks like: FOO, and you type an "L" before the comma, CC Mode, whilst font-locking, needs to know whether this identifier is (i) an element of an enum brace list; (ii) the name of a variable being declared (when the type will have been on a previous line); (iii) a parameter in a function call; or (iv) (maybe one or two other things). It does this by instructing Font Lock to start its processes from a specific earlier point in the file. In the case of FOO being inside a brace, without there being any semicolon before FOO, that position is the beginning of the first identifier after the brace. Starting from that position, c-font-lock-enum-tail will trigger, font-locking FOOL as font-lock-variable-name-face. Starting from outside the brace, c-font-lock-enum-tail will not trigger, leaving FOOL unfontified. When you, as a user, put point at BOL2 and type M-o M-o, then this will work too. Typing in L4, L5, .... triggering after-change fontification is an example of the first circumstance, where CC Mode sets BO font-lock region to just after the brace on L2. > > The problem is, when that starting point needs to be > > after the brace on L2, Font Lock moves the starting point somewhere > > else, fouling up the font locking. > I'm confused: "somewhere else" is typically BOL2. Yes. It is in a syntactically different context. > Here you say this fouls up font-locking, but above you said CC-Mode is > quite capable of handling a font-lock region starting at BOL2. See above. If necessary for its analysis (I'm not sure if it is), CC Mode would extend the region as required if asked to start fl'ing at BOL2. > > One place which is causing me puzzlement is in jit-lock-fontify-now. > > It's this bit: > As you know, jit-lock is irrelevant to those kinds of font-lock problems > (in the sense that it's always possible to trigger the same problems > without the use of jit-lock, tho it may require some time to figure > out a triggering sequence). No, it isn't. Jit lock is used, and has its imperfections, one of which I recently reported as a bug and fixed, another of which is causing me pain and I've asked you for help on. These problems don't exist in plain Font Lock mode (with f-l-support-mode nil). > > The only way `start' can be before `orig-start' is on the first line of > > the fl region, and start is moved to BOL. Why is it necessary to go to > > all the expense of using `run-with-timer' rather than just fixing things > > inside `jit-lock-fontify-now'? What is the use case? > It's explained in the comment. OK. I think I've got it now. jit-lock-fontify-now is expanding backwards into a region which is already fontified, but for which it doesn't "have permission", then reapplying face properties which are already there, then (just in case a face property might have changed) triggering a whole new expensive redisplay. To minimise this expense, a questionable short cut is applied in another function (font-lock-extend-jit-lock-region-after-change). It is this short cut which is causing me the pain. There's got to be a better way of doing things than this. The abstract cause of all these difficulties seems to be Font Lock's not distinguishing between a beginning of region to be used for analysis, and the beginning of region to which face properties will be applied. > Which part of the comment don't you understand? I'm not sure. I get it now, but it was extremely difficult arriving at this understanding. Perhaps if "any part of the buffer prior to that" had been "the initial part of the first line before `orig-start'", I would have got it sooner. > By the time jit-lock is called the redisplay up to orig-start has > already been done (you can assume it's already drawn on the screen). > Stefan -- Alan Mackenzie (Nuremberg, Germany). ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. 2015-03-20 21:12 ` Alan Mackenzie @ 2015-03-20 22:01 ` Stefan Monnier 0 siblings, 0 replies; 25+ messages in thread From: Stefan Monnier @ 2015-03-20 22:01 UTC (permalink / raw) To: Alan Mackenzie; +Cc: Daniel Colascione, emacs-devel > OK. I think I've got it now. jit-lock-fontify-now is expanding > backwards into a region which is already fontified, Whether it's already fontified is not the important point. The important point is that it's already been *rendered* (i.e. the buffer's content has already been read and turned into data in the glyph matrices used to draw on the screen). > but for which it doesn't "have permission", It has permission alright, but whatever it does will only affect that next redisplay (i.e. usually at the next command). > The abstract cause of all these difficulties seems to be Font Lock's not > distinguishing between a beginning of region to be used for analysis, and > the beginning of region to which face properties will be applied. Font lock indeed doesn't have a notion of "beginning of region to be used for analysis" at all, so it can't distinguish it from the notion of "beginning of region to which face properties will be applied". If your keyword rules need to look at some earlier text to get context, they're free to do so. > this understanding. Perhaps if "any part of the buffer prior to that" > had been "the initial part of the first line before `orig-start'", I > would have got it sooner. But that's very specific to your case. Usually the problem occurs when orig-start is at a BOL, but start is at an earlier line. Stefan ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. 2015-03-16 23:53 ` Daniel Colascione 2015-03-18 12:08 ` Alan Mackenzie @ 2015-03-30 16:44 ` Alan Mackenzie 1 sibling, 0 replies; 25+ messages in thread From: Alan Mackenzie @ 2015-03-30 16:44 UTC (permalink / raw) To: Daniel Colascione; +Cc: emacs-devel Hello, Daniel. On Mon, Mar 16, 2015 at 04:53:28PM -0700, Daniel Colascione wrote: > On 02/01/2015 01:22 PM, Alan Mackenzie wrote: > > branch: emacs-24 > > commit d69e9f1c24f37f36af33b0468c5a4e100dbb09b6 > > Author: Alan Mackenzie <acm@muc.de> > > Commit: Alan Mackenzie <acm@muc.de> > > CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669. > > cc-mode.el (c-font-lock-init): Setq font-lock-extend-region-functions to > > nil. > This change is driving me nuts. Consider the following buffer, with @ > representing point: > void foo() { > @ > } > If I type [f o r SPC], then depending on timing, I see only the "or", or > only the "r", fontified in font-lock-keyword-face. A moment later, > jit-lock kicks in and refontifies the entire chunk properly. > Can we please extend the start of the region back to at least the last `}'? I've just committed a fix (to the master branch) which should fix this whilst also leaving bug #19669 fixed. Could you try it out, please. -- Alan Mackenzie (Nuremberg, Germany). ^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2015-03-30 16:44 UTC | newest] Thread overview: 25+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- [not found] <20150201212213.17840.3710@vcs.savannah.gnu.org> [not found] ` <E1YI1y9-0004eO-Re@vcs.savannah.gnu.org> 2015-02-02 18:50 ` [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669 Stefan Monnier 2015-02-02 19:27 ` Alan Mackenzie 2015-02-03 2:12 ` Stefan Monnier 2015-02-02 18:54 ` Stefan Monnier 2015-02-02 21:39 ` Alan Mackenzie 2015-02-03 2:26 ` Stefan Monnier 2015-02-07 12:27 ` Alan Mackenzie 2015-02-07 15:29 ` Stefan Monnier 2015-02-07 15:40 ` Alan Mackenzie 2015-03-16 23:53 ` Daniel Colascione 2015-03-18 12:08 ` Alan Mackenzie 2015-03-19 3:34 ` Daniel Colascione 2015-03-19 9:31 ` Alan Mackenzie 2015-03-19 13:49 ` Stefan Monnier 2015-03-19 20:37 ` Alan Mackenzie 2015-03-19 20:56 ` Stefan Monnier 2015-03-20 16:30 ` Alan Mackenzie 2015-03-20 16:34 ` Daniel Colascione 2015-03-20 17:29 ` Alan Mackenzie 2015-03-20 21:25 ` Daniel Colascione 2015-03-20 22:30 ` Alan Mackenzie 2015-03-20 18:05 ` Stefan Monnier 2015-03-20 21:12 ` Alan Mackenzie 2015-03-20 22:01 ` Stefan Monnier 2015-03-30 16:44 ` Alan Mackenzie
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).